Compare commits

...
Author SHA1 Message Date
James Pine 60c0fe3b92 isolate shutdown unload calls so one failure doesn't block the other 2026-03-16 03:50:28 -07:00
James Pine c99828cf76 fix startup db session leak on error (rollback + close in finally) 2026-03-16 03:49:21 -07:00
James Pine 5c4b979480 suppress E402 for app.py (AMD env vars must precede torch import) 2026-03-16 03:47:56 -07:00
James Pine 2d1b0ae820 remove unused _get_cuda_dll_excludes function 2026-03-16 03:46:58 -07:00
James Pine 69486c2a77 handle null duration in story_items migration 2026-03-16 03:44:33 -07:00
James Pine e9f63d6c57 reject model migration to subdirectory of source cache 2026-03-16 03:43:37 -07:00
James Pine 8906bee23e fix docstring for find_voicebox_pid_on_port 2026-03-16 03:43:06 -07:00
James Pine 0dabb121c9 improve startup logging: version, platform, data dir, db stats
Replace verbose startup messages with a clean summary:
- App version, Python version, OS/arch
- Database path (fix None display), data directory
- Profile and generation counts
- Backend, GPU, model cache path
- Clean up stale loading_model status on startup
- Remove noisy progress manager log line
2026-03-16 03:42:39 -07:00
James Pine 944ba227ca soften select focus indicator opacity 2026-03-16 03:22:36 -07:00
James Pine 473bb3e9fb fix take-label race in regeneration, add accessible focus to select
- Use DB COUNT query instead of list length for take-N label to avoid
  TOCTOU race between list_versions and create_version
- Add focus:bg-muted to SelectTrigger for keyboard focus visibility
2026-03-16 03:22:05 -07:00
James Pine 0d0b62ea93 address CodeRabbit review: fix 4 critical + 12 major issues
Critical:
- Remove dead backend.utils.validation PyInstaller hidden import
- Fix story_items table rebuild to preserve track/trim/version columns
- Guard cache migration against same source/destination path
- Fix regeneration audio overwrite (use random uuid suffix per take)

Major:
- Engine selector: validate language on Qwen switch, clear stale modelSize
- Sync language validation regex between profile create and generate (22 langs)
- Guard CUDA download against duplicate concurrent requests
- Only set model_size for engines that support multiple sizes
- Fix 404 swallowed by generic except in history export
- Validate audio_path before FileResponse in export-audio
- Transcription: stream uploads in 1MB chunks, use robust cache check,
  call complete_download() on Whisper download success
- Set clean version as default when effects chain validation fails
- Return explicit error when Windows port occupied by non-voicebox process
2026-03-16 03:12:01 -07:00
James Pine 798cd40f05 delete stale planning docs 2026-03-16 02:59:24 -07:00
James Pine 3187344f01 add model loading status, effects preset dropdown, clean up UI
Backend:
- Generation service reports 'loading_model' status only when model
  is not yet in memory, then 'generating' once inference starts
- Migrate hf_offline_patch.py from print() to logging module
- Update ADDING_TTS_ENGINES.md for post-refactor file paths

Frontend:
- HistoryTable shows 'Loading model...' vs 'Generating...' based on step
- FloatingGenerateBox: replace instruct toggle + inline effects editor
  with an effects preset dropdown (third dropdown after language and engine)
- Instruct UI removed for now (form field preserved for future models)
- Remove focus ring from Select component globally
2026-03-16 02:58:41 -07:00
James Pine 8efcc95606 update Cargo.lock 2026-03-16 02:20:19 -07:00
James Pine 87cab9473d gitignore: stop tracking tauri/src-tauri/gen/Assets.car
Compiled Xcode asset catalog gets regenerated every build. No reason
to track it.
2026-03-16 02:20:02 -07:00
James Pine 7b0fbfb567 rewrite backend README, remove completed refactor plan, update style guide
Replace the outdated backend README (473 lines of stale API docs and
pre-refactor file tree) with a concise architecture document covering
module structure, request flow, backend selection, API domain overview,
and development commands.

Delete REFACTOR_PLAN.md -- all phases are complete.

Update STYLE_GUIDE.md to remove refactor plan references and replace
the verbose target layout with the current actual structure.
2026-03-16 02:18:34 -07:00
James Pine 7c1ea0a1e1 fix: replace netstat with TcpStream + PowerShell for port detection (#277)
On Windows, Voicebox shelled out to netstat.exe on startup to check for
existing server processes. On systems with corrupted DLLs, netstat fails
with 0xc0000142, causing an infinite loading loop.

Replace with:
- TcpStream::connect_timeout() for port-in-use checks (pure Rust)
- PowerShell Get-NetTCPConnection for port-to-PID lookup (built-in cmdlet)
- tasklist for process name verification (unchanged)

Closes #277
2026-03-16 02:15:26 -07:00
James Pine b3012ed10c move CRUD and service modules into services/, platform_detect into utils/
Move 9 business-logic modules from the backend root into services/:
channels, effects, history, profiles, stories, versions, export_import,
transcribe, tts. Move platform_detect.py into utils/.

Backend root now contains only infrastructure (app, main, config, server,
models, build_binary) and docs. All 94 routes verified.
2026-03-16 02:15:20 -07:00
James Pine 88536d27f7 extract routes from main.py into domain routers (Phase 4)
Split the 2,578-line main.py (90 routes) into 12 domain-specific router
modules under routes/. main.py is now a 45-line entry point.

New structure:
- app.py: FastAPI instance, CORS, startup/shutdown, safe_content_disposition
- routes/: health, profiles, channels, generations, history, transcription,
  stories, effects, audio, models, tasks, cuda
- services/cuda.py: moved from cuda_download.py

Also includes Phase 5 database/ package (from parallel agent):
- database/__init__.py re-exports all symbols for backward compat
- database/models.py, session.py, migrations.py, seed.py

All 90 routes verified registered and app imports cleanly.
2026-03-16 02:03:15 -07:00
James Pine 89d6e364d4 move pyproject 2026-03-16 01:48:26 -07:00
James Pine b7781951df comment cleanup 2026-03-16 01:46:19 -07:00
James Pine fe19a9ca47 add style guide, ruff config, generation service extraction, remove Makefile
- Add backend/STYLE_GUIDE.md covering formatting, imports, types, docstrings,
  comments, error handling, async, logging, and naming conventions
- Add pyproject.toml with ruff linter/formatter config (ERA, FIX, isort, pyupgrade)
- Extract generation service (Phase 3): unified run_generation() replaces three
  duplicated closures, serial queue moved to services/task_queue.py
- Delete Makefile in favor of justfile; update all references
- Add Python lint/format/test commands to justfile (check-python, fix-python, test)
- Install ruff, pytest, pytest-asyncio as dev tools in setup-python
- Update REFACTOR_PLAN.md with Phase 3 and Phase 7 completion
2026-03-16 01:35:59 -07:00
Jamie Pine 439fedcbf2 update refactor plan with phase 1+2 progress 2026-03-16 01:10:59 -07:00
Jamie Pine 0813a3d9d6 refactor: remove dead code, deduplicate backends
Phase 1 - delete dead code:
- studio.py, migrate_add_instruct.py, utils/validation.py
- duplicate _profile_to_response in main.py, duplicate asyncio import
- pointless _get_profiles_dir/_get_generations_dir wrappers
- duplicate LANGUAGE_CODE_TO_NAME and WHISPER_HF_REPOS constants

Phase 2 - extract backends/base.py with shared utilities:
- is_model_cached() replaces 7 copy-pasted HF cache checks
- get_torch_device() replaces 5 device detection methods
- combine_voice_prompts() replaces 5 identical implementations
- model_load_progress() ctx manager replaces progress boilerplate in all backends
- patch_chatterbox_f32() replaces identical monkey-patches in both chatterbox backends

net -1078 lines across the backend
2026-03-16 01:10:02 -07:00
Jamie Pine 9514c6596c migrations 2026-03-16 00:54:13 -07:00
Jamie Pine 4e84415da7 refactor start 2026-03-16 00:52:23 -07:00
Jamie Pine 82cd4bf2ef Add dynamic download redirect routes and update README links 2026-03-15 23:22:03 -07:00
Jamie Pine 3c30c5bec1 Update README for v0.2.x: multi-engine, effects, 23 languages, fix download links 2026-03-15 17:12:47 -07:00
Jamie Pine c9d7bc4f27 Fix macOS download links to use .dmg instead of .app.tar.gz 2026-03-15 17:03:17 -07:00
James Pine 34e17bd469 Fix LuxTTS + Chatterbox in prod: bundle espeak/perth data, fix multiprocessing
- collect-all piper_phonemize to bundle espeak-ng-data for LuxTTS phonemization
- Set ESPEAK_DATA_PATH in frozen builds so the C library finds bundled data
- collect-all perth to bundle pretrained watermark model for Chatterbox
- Add multiprocessing.freeze_support() to fix resource_tracker subprocess crash
2026-03-15 16:02:09 -07:00
James Pine aada13a5c9 Collect all inflect files for PyInstaller (fixes typeguard inspect.getsource) 2026-03-15 14:32:10 -07:00
James Pine de8558d197 Fix prod build: download progress, robust stderr, full tracebacks
- Force tqdm disable=False in TrackedTqdm so byte progress works in prod
  (huggingface_hub disables tqdm based on logger level, which prevents
  self.n from updating — our progress tracking needs the counter even
  though we don't render to terminal)
- Harden devnull redirect to test writability, not just None check
- Add full traceback logging to all backend error handlers
- Add chatterbox/luxtts/zipvoice hidden imports and metadata to spec
2026-03-15 14:23:11 -07:00
Jamie Pine 9d79ea367a Only use --noconsole on Windows, macOS/Linux need stdout for Tauri logs 2026-03-15 12:07:35 -07:00
Jamie Pine 04316f7adc Copy metadata for requests/transformers/huggingface-hub to fix PyInstaller metadata lookup 2026-03-15 11:35:05 -07:00
Jamie Pine 4e4361d350 Fix noconsole crash: redirect None stdout/stderr to devnull on Windows 2026-03-15 11:27:35 -07:00
Jamie Pine e9a249587c Collect all linacodec files for PyInstaller (fixes inspect.getsource in Vocos) 2026-03-15 11:06:13 -07:00
Jamie Pine d8a9ed7d15 Enable updater artifacts with v1Compatible for tauri-action sig generation 2026-03-15 10:54:12 -07:00
Jamie Pine 3dbf1c200e Revert "Bump version: 0.2.3 → 0.2.4"
This reverts commit 40fcb8d917.
2026-03-15 10:20:31 -07:00
Jamie Pine 40fcb8d917 Bump version: 0.2.3 → 0.2.4 2026-03-15 10:18:51 -07:00
Jamie Pine ad64d1c3d9 Collect all zipvoice files for PyInstaller (fixes source code error) 2026-03-15 10:18:40 -07:00
Jamie Pine f826e45250 Install chatterbox-tts in CI release workflow 2026-03-15 10:17:23 -07:00
Jamie Pine 3d53c06c5b Bump version: 0.2.2 → 0.2.3 2026-03-15 10:08:56 -07:00
James Pine 9835b9f6d4 fix: prevent stale release data by removing Next.js fetch cache
Replace next: { revalidate: 600 } with cache: 'no-store' on GitHub
API fetches so new releases show up within 5 minutes (in-memory cache
only, no Next.js/Vercel cache layer on top).
2026-03-15 10:07:50 -07:00
Jamie Pine a15dd30b1e Update tauri-action to v0.6 to fix updater JSON and signature generation 2026-03-15 10:05:36 -07:00
Jamie Pine 1d343ac071 Treat missing/draft releases as up-to-date instead of showing error 2026-03-15 09:52:17 -07:00
James Pine ca602de0ae fix: don't reset audio player when unmuting during playback 2026-03-15 09:29:44 -07:00
James Pine cdc0293ca8 feat: add /linux-install page with build-from-source instructions
Linux download card now links to /linux-install instead of a direct
binary download. The page explains the CI situation and gives
clone + setup + build commands.
2026-03-15 09:17:30 -07:00
Jamie Pine e7f749f082 Add luxtts/zipvoice hidden imports to PyInstaller build 2026-03-15 09:13:59 -07:00
Jamie Pine d42e926e5c Bump version: 0.2.1 → 0.2.2 2026-03-15 09:02:10 -07:00
Jamie Pine 32768ea874 Add chatterbox hidden imports to PyInstaller build 2026-03-15 09:00:13 -07:00
James Pine b585e18ccf fix: fade in hero background glow to avoid Safari rendering flash 2026-03-15 08:53:08 -07:00
Jamie Pine 655910457f Auto-update CUDA binary on app update: check version on startup, download if stale 2026-03-15 08:46:17 -07:00
James Pine d6984f1057 fix: remove mix-blend-lighten and drop-shadow causing boxes in Safari 2026-03-15 08:45:40 -07:00
James Pine a637aebe69 feat: show version and total download count on landing page
Fetches download counts across all GitHub releases (paginated) and
displays version, total downloads, and platform list below the CTA.
2026-03-15 08:37:23 -07:00
James Pine a5269d23db Fix keep-server-running on macOS: ignore SIGHUP, watchdog grace period, build script fixes 2026-03-15 08:22:35 -07:00
Jamie Pine fc450e5024 Hide console window for server binary on Windows 2026-03-15 07:57:58 -07:00
Jamie Pine a99c2b572d Show download progress bar for CUDA backend download 2026-03-15 07:50:23 -07:00
Jamie Pine 96289e95f1 Bump version: 0.2.0 → 0.2.1 2026-03-15 06:36:38 -07:00
Jamie PineandGitHub e316b0b4bb Merge pull request #274 from jamiepine/feat/landing-page-redesign
Landing page v0.2.0 redesign
2026-03-15 06:22:04 -07:00
Jamie PineandGitHub 732270b571 Merge pull request #272 from jamiepine/windows-support
Windows support: CUDA detection, cross-platform justfile, clean server shutdown
2026-03-15 06:20:46 -07:00
James Pine 0c6aa15746 Responsive polish: pointer-events-none on animations, sticky header with scroll fade, desktop scroll-to-active fix, iOS audio unlock, player and UI tweaks
- Add pointer-events-none/select-none to feature cards, voice creator, and ControlUI mock
- Sticky header with gradient fade overlay (matching real app 3-layer technique)
- Fix desktop scroll-to-active: separate mobile/desktop card refs to prevent mobile refs overwriting desktop
- Scroll selected card to 2nd row when outside safe zone above generate box
- iOS Safari audio unlock via WaveSurfer's actual media element
- Player: accent fill play/pause button, padding on volume slider, remove close button
- Profile cards: fixed 143px height, mobile edge fades with scroll-aware left fade
- Generate box: accent effect pill when active, white fill sparkle icon, edge-aligned on desktop
- Voice creator: animated waveform background with height-based bars
- 12 profiles (added Attenborough, Zendaya, Obama) for 4-row grid with scroll
2026-03-15 06:17:54 -07:00
Jamie Pine 410413dc57 Watchdog respects keep-server-running setting via /watchdog/disable endpoint 2026-03-15 06:05:17 -07:00
Jamie Pine e239be5bbb Review fixes: CUDA restore in finally, os._exit on Windows, taskkill /T for process tree, build-server-cuda error handling, db-init path 2026-03-15 05:43:26 -07:00
James Pine f80782a90a Landing page v0.2.0 updates: multi-engine copy, star count, model cards, voice creator section, responsive ControlUI, iOS audio fix
- Replace Qwen-specific copy with multi-engine messaging across hero, meta, and features
- Add GitHub star count fetched server-side via /api/stars with Spacedrive-style navbar badge
- Replace 'Why Voicebox exists' section with model cards for all 4 TTS engines
- Enable Linux download card (was 'Coming soon')
- Update GPU support copy to include ROCm, Intel Arc, DirectML
- Add Voice Creator section with animated 3-tab UI (upload, mic, system audio) and waveform background
- Make ControlUI responsive: horizontal scroll cards on mobile, stacked layout, scroll-to-active profile
- Fix iOS Safari audio autoplay (unlock AudioContext on user gesture)
- Fix hero logo square background with mix-blend-lighten
- Remove generation length green coloring, use gray with accent highlights
- Comment out grain overlay (visible tile seams)
- Remove player close button, stack waveform above controls on mobile
- Fixed-height profile cards (143px) with space between badges and buttons
2026-03-15 04:53:28 -07:00
Jamie Pine f1ba73a386 Address review: validate parent-pid, ensure binaries dir exists, fix Xcode typo 2026-03-15 04:09:49 -07:00
Jamie Pine f1963740b4 Fix server binary build, watchdog logging, pedalboard import, window close loop 2026-03-15 04:04:56 -07:00
Jamie Pine 4d6c976ad9 Windows support: CUDA detection, justfile cross-platform, clean server shutdown 2026-03-15 00:02:13 -07:00
Jamie Pine 8377152d86 Redesign landing page with animated ControlUI hero
New Spacedrive-inspired landing page with dark warm color system, glassmorphic navbar, feature cards with animated illustrations, and an interactive ControlUI mockup that cycles through voice generations with real audio playback via WaveSurfer.

The ControlUI demo script is fully data-driven - profiles, generation text, audio samples, and effects are all configurable from a single DEMO_SCRIPT array.

Includes 6 real voice samples (Jarvis, Morgan Freeman, Sam Altman, Samuel L. Jackson, Linus Tech Tips, Fireship) converted to webm opus.
2026-03-14 23:08:29 -07:00
Jamie PineandGitHub 7a511e3756 Merge pull request #271 from jamiepine/feat/post-processing-effects
Add post-processing audio effects system
2026-03-14 12:14:39 -07:00
Jamie PineandGitHub 6d261c44a1 Merge branch 'main' into feat/post-processing-effects 2026-03-14 12:14:26 -07:00
Jamie Pine 103e98b38f github runners suck 2026-03-14 12:13:45 -07:00
Jamie Pine 1c61b47a64 Glassmorphic active state for sidebar buttons with accent border shine 2026-03-14 12:11:07 -07:00
Jamie Pine 626e3740e1 Auto-select first story when navigating to Stories tab 2026-03-14 11:14:46 -07:00
Jamie Pine 310a4acb02 Add source version selection when applying effects, voices tab overhaul with inline inspector 2026-03-14 11:07:32 -07:00
Jamie Pine 899b90202b Add version control to track editor, restyle story list
- Story items can be pinned to a specific generation version via
  toolbar dropdown (shows when clip is selected and has >1 version)
- version_id column on story_items with migration, validated against
  the generation's versions before saving
- Split/duplicate preserve the source clip's pinned version
- Export and playback resolve version-specific audio paths
- Extracted _build_item_detail helper in stories.py (DRY cleanup)
- Story list restyled from rounded cards to flat rows with rounded
  hover/active states, gradient header fade, and dynamic bottom
  padding that accounts for track editor + generate box
2026-03-14 09:56:27 -07:00
Jamie Pine e8d54d52d3 Add favorites, effects badge on profiles, UI polish
- Add is_favorited column with toggle endpoint and star button on history
- Show sparkles icon on profile cards that have effects configured
- Gold ring on selected profile cards
- Smaller, gray action buttons with brighter hover
- Clamp player time to duration to prevent runaway playback
- Align profile card icon to top for wrapped names
- Flush bottom corners on history card when versions expanded
- Simplify .gitignore data/ rule
2026-03-14 09:10:56 -07:00
Jamie Pine 00c5b75ffb Regenerate as new version, UI polish, and bugfixes
- Add /generate/{id}/regenerate endpoint that creates a new take version
- Wire regenerate into history dropdown with SSE progress + autoplay
- Add normalize_audio to regenerate path
- Remove duplicate Regenerate menu item
- Show disabled ellipsis menu during generation instead of hiding it
- Fix sf.write format kwarg broken by asyncio.to_thread migration
- Rename clean version label to 'original', effects to 'version-N'
- Show effects chain names in version list instead of 'N fx'
- Include all versions in export package
- Clean up version panel padding
2026-03-14 08:34:58 -07:00
Jamie Pine 25134b4ba9 Fix player not loading new version after applying effects
Reload the player with the version-specific audio URL when effects are
applied to the currently playing generation. Also consolidate the
instruct/effects buttons into a single button with the effects editor
shown inline when instruct mode is open.
2026-03-14 08:01:45 -07:00
Jamie Pine 3d922ec846 Fix review findings: toggle logic, preset saving, version lookup, async audio ops
- Fix inverted effects toggle in FloatingGenerateBox
- Add Save button + API method for editing custom effect presets
- Return early on effects save failure in ProfileForm
- Fix no-op ternary in effectsStore
- Handle duplicate preset names with proper 400 response
- Use effects_chain is None instead of label for clean version lookup
- Move blocking audio ops to asyncio.to_thread in async endpoints
- Log warnings instead of silently swallowing parse errors
2026-03-14 07:47:06 -07:00
James Pine 638820c839 Add post-processing audio effects system
Adds a full effects pipeline powered by Spotify's pedalboard library,
enabling users to apply professional DSP effects (flanger, reverb, delay,
compressor, pitch shift, filters, gain) to generated audio.

Key features:
- Effects chain editor with drag-and-drop reordering (dnd-kit)
- Generation versions: clean copy always saved, processed versions created on top
- Built-in presets (Robotic, Radio, Echo Chamber, Deep Voice) + custom user presets
- Per-profile default effects chain (auto-applied to new generations)
- Per-generation effects override from the generation form
- Apply effects to existing generations from history (creates new version)
- Ephemeral preview endpoint for auditioning effects without persisting
- Dedicated Effects sidebar tab with preset management and live preview
- Version switcher in history cards with expandable panel
- Backward-compatible: existing generations backfilled as clean versions
2026-03-14 07:11:56 -07:00
Jamie Pine 7cbf5a1ded Use Namespace runner for Linux release build 2026-03-14 05:38:25 -07:00
Jamie Pine e89a7eb7e7 Windows dev experience: CUDA detection, GPU display cleanup, hide drag region, dev-mode update status 2026-03-14 03:24:39 -07:00
Jamie Pine e18757bab3 Skip AppImage on Linux, build deb/rpm only 2026-03-14 03:04:59 -07:00
Jamie Pine 0e6c678fc3 Add Windows support to justfile 2026-03-14 02:41:59 -07:00
Jamie Pine 942dabbcac Install CPU-only PyTorch before requirements.txt on Linux
The previous ordering let requirements.txt pull CUDA-enabled torch
with all nvidia deps first, then the CPU override only swapped the
torch wheel while leaving ~3GB of nvidia packages installed.
2026-03-14 00:55:43 -07:00
Jamie Pine b915825165 Add libasound2-dev to Linux release deps 2026-03-13 21:36:46 -07:00
Jamie Pine f3fc63942f Fix Linux release build exceeding 4GB PyInstaller limit
Install CPU-only PyTorch on Linux and exclude nvidia modules from
non-CUDA PyInstaller builds.
2026-03-13 21:19:24 -07:00
Jamie Pine 9044b986f3 Move tauri imports behind platform abstraction, merge CUDA build into release workflow
- Add openPath and pickDirectory to PlatformFilesystem interface
- Remove direct @tauri-apps/plugin-shell and plugin-dialog imports from app
- Merge build-cuda.yml into release.yml as a parallel job
2026-03-13 11:33:32 -07:00
Jamie Pine b01076b6b3 Bump version: 0.1.13 → 0.2.0 2026-03-13 11:06:40 -07:00
Jamie PineandGitHub 5121c76e39 Merge pull request #269 from jamiepine/feat/async-generation-queue
feat: async generation queue
2026-03-13 10:58:18 -07:00
157 changed files with 14750 additions and 10193 deletions
+1 -1
View File
@@ -1,5 +1,5 @@
[bumpversion] [bumpversion]
current_version = 0.1.13 current_version = 0.2.3
commit = True commit = True
tag = True tag = True
tag_name = v{new_version} tag_name = v{new_version}
-73
View File
@@ -1,73 +0,0 @@
name: Build CUDA Backend
on:
workflow_dispatch:
push:
tags:
- "v*"
jobs:
build-cuda-windows:
runs-on: windows-latest
permissions:
contents: write
steps:
- uses: actions/checkout@v4
- name: Setup Python
uses: actions/setup-python@v5
with:
python-version: "3.12"
cache: "pip"
- name: Install Python dependencies
run: |
python -m pip install --upgrade pip
pip install pyinstaller
pip install -r backend/requirements.txt
- name: Install PyTorch with CUDA 12.1
run: |
pip install torch --index-url https://download.pytorch.org/whl/cu121 --force-reinstall --no-deps
pip install torchaudio --index-url https://download.pytorch.org/whl/cu121
- name: Verify CUDA support in torch
run: |
python -c "import torch; print(f'CUDA available in build: {torch.cuda.is_available()}'); print(f'CUDA version: {torch.version.cuda}')"
- name: Build CUDA server binary
shell: bash
working-directory: backend
run: python build_binary.py --cuda
- name: Split binary for GitHub Releases
shell: bash
run: |
python scripts/split_binary.py \
backend/dist/voicebox-server-cuda.exe \
--output release-assets/
- name: Upload split parts to GitHub Release
if: startsWith(github.ref, 'refs/tags/')
uses: softprops/action-gh-release@v1
with:
files: |
release-assets/voicebox-server-cuda.part*.exe
release-assets/voicebox-server-cuda.sha256
release-assets/voicebox-server-cuda.manifest
draft: true
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: Upload binary as workflow artifact (for testing)
uses: actions/upload-artifact@v4
with:
name: voicebox-server-cuda-windows
path: backend/dist/voicebox-server-cuda.exe
retention-days: 7
# Linux CUDA build can be added later with:
# build-cuda-linux:
# runs-on: ubuntu-22.04
# ...
+71 -14
View File
@@ -22,10 +22,6 @@ jobs:
args: "--target x86_64-apple-darwin" args: "--target x86_64-apple-darwin"
python-version: "3.12" python-version: "3.12"
backend: "pytorch" backend: "pytorch"
- platform: "ubuntu-22.04"
args: ""
python-version: "3.12"
backend: "pytorch"
- platform: "windows-latest" - platform: "windows-latest"
args: "" args: ""
python-version: "3.12" python-version: "3.12"
@@ -37,10 +33,10 @@ jobs:
- uses: actions/checkout@v4 - uses: actions/checkout@v4
- name: Install dependencies (ubuntu only) - name: Install dependencies (ubuntu only)
if: matrix.platform == 'ubuntu-22.04' if: contains(matrix.platform, 'ubuntu') || contains(matrix.platform, 'namespace')
run: | run: |
sudo apt-get update sudo apt-get update
sudo apt-get install -y libwebkit2gtk-4.1-dev libappindicator3-dev librsvg2-dev patchelf llvm-dev sudo apt-get install -y libwebkit2gtk-4.1-dev libappindicator3-dev librsvg2-dev patchelf llvm-dev libasound2-dev
- name: Install LLVM (macOS) - name: Install LLVM (macOS)
if: matrix.platform == 'macos-latest' || matrix.platform == 'macos-15-intel' if: matrix.platform == 'macos-latest' || matrix.platform == 'macos-15-intel'
@@ -55,23 +51,23 @@ jobs:
python-version: ${{ matrix.python-version }} python-version: ${{ matrix.python-version }}
cache: "pip" cache: "pip"
- name: Install CPU-only PyTorch (Linux)
if: contains(matrix.platform, 'ubuntu') || contains(matrix.platform, 'namespace')
run: |
pip install torch torchaudio --index-url https://download.pytorch.org/whl/cpu
- name: Install Python dependencies - name: Install Python dependencies
run: | run: |
python -m pip install --upgrade pip python -m pip install --upgrade pip
pip install pyinstaller pip install pyinstaller
pip install -r backend/requirements.txt pip install -r backend/requirements.txt
pip install --no-deps chatterbox-tts
- name: Install MLX dependencies (Apple Silicon only) - name: Install MLX dependencies (Apple Silicon only)
if: matrix.backend == 'mlx' if: matrix.backend == 'mlx'
run: | run: |
pip install -r backend/requirements-mlx.txt pip install -r backend/requirements-mlx.txt
# - name: Install PyTorch with CUDA (Windows only)
# if: matrix.platform == 'windows-latest'
# run: |
# pip install torch --index-url https://download.pytorch.org/whl/cu121 --force-reinstall --no-deps
# pip install torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121
- name: Build Python server (Linux/macOS) - name: Build Python server (Linux/macOS)
if: matrix.platform != 'windows-latest' if: matrix.platform != 'windows-latest'
run: | run: |
@@ -127,7 +123,7 @@ jobs:
p12-file-base64: ${{ secrets.APPLE_CERTIFICATE }} p12-file-base64: ${{ secrets.APPLE_CERTIFICATE }}
p12-password: ${{ secrets.APPLE_CERTIFICATE_PASSWORD }} p12-password: ${{ secrets.APPLE_CERTIFICATE_PASSWORD }}
- uses: tauri-apps/tauri-action@v0 - uses: tauri-apps/tauri-action@v0.6
env: env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
TAURI_SIGNING_PRIVATE_KEY: ${{ secrets.TAURI_SIGNING_PRIVATE_KEY }} TAURI_SIGNING_PRIVATE_KEY: ${{ secrets.TAURI_SIGNING_PRIVATE_KEY }}
@@ -151,10 +147,71 @@ jobs:
- **macOS (Apple Silicon)**: Download the `aarch64.dmg` file - uses MLX for fast native inference - **macOS (Apple Silicon)**: Download the `aarch64.dmg` file - uses MLX for fast native inference
- **macOS (Intel)**: Download the `x64.dmg` file - uses PyTorch - **macOS (Intel)**: Download the `x64.dmg` file - uses PyTorch
- **Windows**: Download the `.msi` installer - **Windows**: Download the `.msi` installer
- **Linux**: Download the `.AppImage` or `.deb` package - **Linux**: Compile from source (see README)
The app includes automatic updates - future updates will be installed automatically. The app includes automatic updates - future updates will be installed automatically.
releaseDraft: true releaseDraft: true
prerelease: false prerelease: false
args: ${{ matrix.args }} args: ${{ matrix.args }}
includeUpdaterJson: true includeUpdaterJson: true
build-cuda-windows:
runs-on: windows-latest
permissions:
contents: write
steps:
- uses: actions/checkout@v4
- name: Setup Python
uses: actions/setup-python@v5
with:
python-version: "3.12"
cache: "pip"
- name: Install Python dependencies
run: |
python -m pip install --upgrade pip
pip install pyinstaller
pip install -r backend/requirements.txt
pip install --no-deps chatterbox-tts
- name: Install PyTorch with CUDA 12.1
run: |
pip install torch --index-url https://download.pytorch.org/whl/cu121 --force-reinstall --no-deps
pip install torchaudio --index-url https://download.pytorch.org/whl/cu121
- name: Verify CUDA support in torch
run: |
python -c "import torch; print(f'CUDA available in build: {torch.cuda.is_available()}'); print(f'CUDA version: {torch.version.cuda}')"
- name: Build CUDA server binary
shell: bash
working-directory: backend
run: python build_binary.py --cuda
- name: Split binary for GitHub Releases
shell: bash
run: |
python scripts/split_binary.py \
backend/dist/voicebox-server-cuda.exe \
--output release-assets/
- name: Upload split parts to GitHub Release
if: startsWith(github.ref, 'refs/tags/')
uses: softprops/action-gh-release@v1
with:
files: |
release-assets/voicebox-server-cuda.part*.exe
release-assets/voicebox-server-cuda.sha256
release-assets/voicebox-server-cuda.manifest
draft: true
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: Upload binary as workflow artifact
uses: actions/upload-artifact@v4
with:
name: voicebox-server-cuda-windows
path: backend/dist/voicebox-server-cuda.exe
retention-days: 7
+2 -4
View File
@@ -35,10 +35,7 @@ target/
Thumbs.db Thumbs.db
# Data (user-generated) # Data (user-generated)
data/profiles/* data/
data/generations/*
data/projects/*
data/voicebox.db
!data/.gitkeep !data/.gitkeep
# Logs # Logs
@@ -52,6 +49,7 @@ logs/
# Generated files # Generated files
app/openapi.json app/openapi.json
tauri/src-tauri/binaries/* tauri/src-tauri/binaries/*
tauri/src-tauri/gen/Assets.car
# Temporary # Temporary
tmp/ tmp/
+8 -6
View File
@@ -66,14 +66,16 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- OpenAPI client generator script now documents the local backend port and avoids an unused loop variable warning - OpenAPI client generator script now documents the local backend port and avoids an unused loop variable warning
### Added ### Added
- **Makefile** - Comprehensive development workflow automation with commands for setup, development, building, testing, and code quality checks - **justfile** - Comprehensive development workflow automation with commands for setup, development, building, testing, and code quality checks
- Includes Python version detection and compatibility warnings - Cross-platform support (macOS, Linux, Windows)
- Self-documenting help system with `make help` - Python version detection and compatibility warnings
- Colored output for better readability - Self-documenting help system with `just --list`
- Supports parallel development server execution
### Changed ### Changed
- **README** - Added Makefile reference and updated Quick Start with Makefile-based setup instructions alongside manual setup - **README** - Updated Quick Start with justfile-based setup instructions
### Removed
- **Makefile** - Replaced by justfile (cross-platform, simpler syntax)
--- ---
+36 -98
View File
@@ -33,101 +33,41 @@ Thank you for your interest in contributing to Voicebox! This document provides
### Development Setup ### Development Setup
**Using `just` (recommended):** Install [just](https://github.com/casey/just) (`brew install just`, `cargo install just`, or `winget install Casey.Just`), then:
Install [just](https://github.com/casey/just) (`brew install just` or `cargo install just`), then:
```bash ```bash
git clone https://github.com/YOUR_USERNAME/voicebox.git
cd voicebox
just setup # creates venv, installs Python + JS deps just setup # creates venv, installs Python + JS deps
just dev # starts backend + desktop app in one terminal just dev # starts backend + desktop app
``` ```
`just setup` handles everything automatically, including:
- Creating a Python virtual environment
- Installing Python dependencies (with CUDA PyTorch on Windows if an NVIDIA GPU is detected)
- Installing MLX dependencies on Apple Silicon
- Installing JavaScript dependencies
`just dev` starts the backend and desktop app together. If a backend is already running (e.g. from `just dev-backend` in another terminal), it detects it and only starts the frontend.
Other useful commands: Other useful commands:
```bash ```bash
just dev-web # backend + web app (no Tauri/Rust build) just dev-web # backend + web app (no Tauri/Rust build)
just dev-backend # backend only just dev-backend # backend only
just dev-frontend # Tauri app only (backend must be running)
just kill # stop all dev processes just kill # stop all dev processes
just clean-all # nuke everything and start fresh just clean-all # nuke everything and start fresh
just --list # see all available commands just --list # see all available commands
``` ```
**Using the Makefile:** Run `make setup` then `make dev`. See `make help` for all commands. > **Note:** In dev mode, the app connects to a manually-started Python server.
> The bundled server binary is only used in production builds.
**Manual setup (required for Windows):** #### Windows Notes
1. **Fork and clone the repository** The justfile works natively on Windows via PowerShell. No WSL or Git Bash required. On Windows with an NVIDIA GPU, `just setup` automatically installs CUDA-enabled PyTorch for GPU acceleration.
```bash
git clone https://github.com/YOUR_USERNAME/voicebox.git
cd voicebox
```
2. **Install JavaScript dependencies**
```bash
bun install
```
This installs dependencies for:
- `app/` - Shared React frontend
- `tauri/` - Tauri desktop wrapper
- `web/` - Web deployment wrapper
3. **Set up Python backend**
```bash
cd backend
# Create virtual environment
python -m venv venv
# Activate virtual environment
source venv/bin/activate # On macOS/Linux
# or
venv\Scripts\activate # On Windows
# Install Python dependencies
pip install -r requirements.txt
# Install MLX dependencies (Apple Silicon only - for faster inference)
# On Apple Silicon, this enables native Metal acceleration
if [[ $(uname -m) == "arm64" ]]; then
pip install -r requirements-mlx.txt
fi
# Install Qwen3-TTS (required for voice synthesis)
pip install git+https://github.com/QwenLM/Qwen3-TTS.git
```
4. **Start development servers**
Development requires two terminals: one for the Python backend, one for the Tauri app.
**Terminal 1: Backend server** (start this first)
```bash
cd backend
source venv/bin/activate # Activate venv if not already active
bun run dev:server
# Or manually: uvicorn main:app --reload --port 17493
```
Backend will be available at `http://localhost:17493`
**Terminal 2: Desktop app**
```bash
bun run dev
```
This will:
- Create a placeholder sidecar binary (for Tauri compilation)
- Start Vite dev server on port 5173
- Launch Tauri window pointing to localhost:5173
- Connect to the Python server you started in Terminal 1
- Enable hot reload
> **Note:** In dev mode, the app connects to your manually-started Python server.
> The bundled server binary is only used in production builds.
**Optional: Web app**
```bash
bun run dev:web
```
Web app will be available at `http://localhost:5174`
### Model Downloads ### Model Downloads
@@ -139,25 +79,30 @@ First-time usage will be slower due to model downloads, but subsequent runs will
### Building ### Building
**Build everything (recommended):** **Build production app:**
```bash ```bash
bun run build just build # Build CPU server binary + Tauri installer
``` ```
This automatically:
1. Builds the Python server binary (`./scripts/build-server.sh`) On Windows, to build with CUDA support for local testing:
2. Builds the Tauri desktop app (`cd tauri && bun run tauri build`)
```bash
just build-local # Build CPU + CUDA server binaries + Tauri installer
```
This builds the CPU sidecar (bundled with the app), the CUDA binary (placed in `%APPDATA%/com.voicebox.app/backends/` for runtime GPU switching), and the installable Tauri app.
Creates platform-specific installers (`.dmg`, `.msi`, `.AppImage`) in `tauri/src-tauri/target/release/bundle/`. Creates platform-specific installers (`.dmg`, `.msi`, `.AppImage`) in `tauri/src-tauri/target/release/bundle/`.
**Note:** The build process detects your platform and includes the appropriate backend (MLX for Apple Silicon, PyTorch for others). **Individual build targets:**
**Build server binary only:**
```bash ```bash
bun run build:server just build-server # CPU server binary only
# or just build-server-cuda # CUDA server binary only (Windows)
./scripts/build-server.sh just build-tauri # Tauri desktop app only
just build-web # Web app only
``` ```
Creates platform-specific binary in `tauri/src-tauri/binaries/`
**Building with local Qwen3-TTS development version:** **Building with local Qwen3-TTS development version:**
@@ -165,17 +110,10 @@ If you're actively developing or modifying the Qwen3-TTS library, set the `QWEN_
```bash ```bash
export QWEN_TTS_PATH=~/path/to/your/Qwen3-TTS export QWEN_TTS_PATH=~/path/to/your/Qwen3-TTS
bun run build:server just build-server
``` ```
This makes PyInstaller use your local qwen-tts version instead of the pip-installed package. Useful when testing changes to the TTS library before they're published to PyPI or when using an editable install (`pip install -e`). This makes PyInstaller use your local qwen-tts version instead of the pip-installed package.
**Build web app:**
```bash
cd web
bun run build
```
Output in `web/dist/`
### Generate OpenAPI Client ### Generate OpenAPI Client
-250
View File
@@ -1,250 +0,0 @@
# Voicebox Makefile
# Unix-only (macOS/Linux). Windows users should use WSL.
SHELL := /bin/bash
.DEFAULT_GOAL := help
# Directories
BACKEND_DIR := backend
TAURI_DIR := tauri
WEB_DIR := web
APP_DIR := app
# Python (prefer 3.12, fallback to 3.13, then python3)
PYTHON := $(shell command -v python3.12 2>/dev/null || command -v python3.13 2>/dev/null || echo python3)
VENV := $(CURDIR)/$(BACKEND_DIR)/venv
VENV_BIN := $(VENV)/bin
PIP := $(VENV_BIN)/pip
PYTHON_VENV := $(VENV_BIN)/python
# Colors for output
BLUE := \033[0;34m
GREEN := \033[0;32m
YELLOW := \033[0;33m
NC := \033[0m # No Color
.PHONY: help
help: ## Show this help message
@echo -e "$(BLUE)Voicebox$(NC) - Development Commands"
@echo ""
@grep -E '^[a-zA-Z_-]+:.*?## .*$$' $(MAKEFILE_LIST) | sort | \
awk 'BEGIN {FS = ":.*?## "}; {printf " $(GREEN)%-20s$(NC) %s\n", $$1, $$2}'
# =============================================================================
# SETUP
# =============================================================================
.PHONY: setup setup-js setup-python setup-rust
setup: setup-js setup-python ## Full project setup (all dependencies)
@echo -e "$(GREEN)✓ Setup complete!$(NC)"
@echo -e " Run $(YELLOW)make dev$(NC) to start development servers"
setup-js: ## Install JavaScript dependencies (bun)
@echo -e "$(BLUE)Installing JavaScript dependencies...$(NC)"
bun install
setup-python: $(VENV)/bin/activate ## Set up Python virtual environment and dependencies
@echo -e "$(BLUE)Installing Python dependencies...$(NC)"
$(PIP) install --upgrade pip
$(PIP) install -r $(BACKEND_DIR)/requirements.txt
$(PIP) install --no-deps chatterbox-tts
@if [ "$$(uname -m)" = "arm64" ] && [ "$$(uname)" = "Darwin" ]; then \
echo -e "$(BLUE)Detected Apple Silicon - installing MLX dependencies...$(NC)"; \
$(PIP) install -r $(BACKEND_DIR)/requirements-mlx.txt; \
echo -e "$(GREEN)✓ MLX backend enabled (native Metal acceleration)$(NC)"; \
fi
$(PIP) install git+https://github.com/QwenLM/Qwen3-TTS.git
@echo -e "$(GREEN)✓ Python environment ready$(NC)"
$(VENV)/bin/activate:
@echo -e "$(BLUE)Creating Python virtual environment...$(NC)"
@PY_MINOR=$$($(PYTHON) -c "import sys; print(sys.version_info[1])"); \
if [ "$$PY_MINOR" -gt 13 ]; then \
echo -e "$(YELLOW)Warning: Python 3.$$PY_MINOR detected. ML packages may not be compatible.$(NC)"; \
echo -e "$(YELLOW)Recommended: Use Python 3.12 or 3.13 (brew install [email protected])$(NC)"; \
fi
$(PYTHON) -m venv $(VENV)
setup-rust: ## Install Rust toolchain (if not present)
@command -v rustc >/dev/null 2>&1 || curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
# =============================================================================
# DEVELOPMENT
# =============================================================================
.PHONY: dev dev-backend dev-frontend dev-web kill-dev
dev: ## Start backend + desktop app (parallel)
@echo -e "$(BLUE)Starting development servers...$(NC)"
@echo -e "$(YELLOW)Note: If Tauri fails, run 'make build-server' first or use separate terminals$(NC)"
@trap 'kill 0' EXIT; \
$(MAKE) dev-backend & \
sleep 2 && if [ "$$(uname)" = "Linux" ] && lspci 2>/dev/null | grep -qi nvidia; then \
WEBKIT_DISABLE_DMABUF_RENDERER=1 $(MAKE) dev-frontend; \
else \
$(MAKE) dev-frontend; \
fi & \
wait
dev-backend: ## Start FastAPI backend server
@echo -e "$(BLUE)Starting backend server on http://localhost:17493$(NC)"
$(VENV_BIN)/uvicorn backend.main:app --reload --port 17493
dev-frontend: ## Start Tauri desktop app
@echo -e "$(BLUE)Starting Tauri desktop app...$(NC)"
bun run dev
dev-web: ## Start backend + web app (parallel)
@echo -e "$(BLUE)Starting web development servers...$(NC)"
@trap 'kill 0' EXIT; \
$(MAKE) dev-backend & \
sleep 2 && cd $(WEB_DIR) && bun run dev & \
wait
kill-dev: ## Kill all development processes
@echo -e "$(YELLOW)Killing development processes...$(NC)"
-pkill -f "uvicorn main:app" 2>/dev/null || true
-pkill -f "vite" 2>/dev/null || true
@echo -e "$(GREEN)✓ Processes killed$(NC)"
# =============================================================================
# BUILD
# =============================================================================
.PHONY: build build-server build-tauri build-web
build: build-server build-tauri ## Build everything (server binary + desktop app)
@echo -e "$(GREEN)✓ Build complete!$(NC)"
build-server: ## Build Python server binary
@echo -e "$(BLUE)Building server binary...$(NC)"
PATH="$(VENV_BIN):$$PATH" ./scripts/build-server.sh
build-tauri: ## Build Tauri desktop app
@echo -e "$(BLUE)Building Tauri desktop app...$(NC)"
cd $(TAURI_DIR) && bun run tauri build
build-web: ## Build web app
@echo -e "$(BLUE)Building web app...$(NC)"
cd $(WEB_DIR) && bun run build
@echo -e "$(GREEN)✓ Web build output in $(WEB_DIR)/dist/$(NC)"
# =============================================================================
# DATABASE & API
# =============================================================================
.PHONY: db-init db-reset generate-api
db-init: $(VENV)/bin/activate ## Initialize SQLite database
@echo -e "$(BLUE)Initializing database...$(NC)"
cd $(BACKEND_DIR) && $(PYTHON_VENV) -c "from database import init_db; init_db()"
@echo -e "$(GREEN)✓ Database created at $(BACKEND_DIR)/data/voicebox.db$(NC)"
db-reset: ## Reset database (delete and reinitialize)
@echo -e "$(YELLOW)Resetting database...$(NC)"
rm -f $(BACKEND_DIR)/data/voicebox.db
$(MAKE) db-init
generate-api: ## Generate TypeScript API client from OpenAPI schema
@echo -e "$(BLUE)Generating API client...$(NC)"
@echo -e "$(YELLOW)Note: Backend must be running (make dev-backend)$(NC)"
./scripts/generate-api.sh
@echo -e "$(GREEN)✓ API client generated in $(APP_DIR)/src/lib/api/$(NC)"
# =============================================================================
# CODE QUALITY
# =============================================================================
.PHONY: lint format typecheck check
lint: ## Run linter (Biome)
@echo -e "$(BLUE)Linting...$(NC)"
bun run lint
format: ## Format code (Biome)
@echo -e "$(BLUE)Formatting...$(NC)"
bun run format
typecheck: ## Run TypeScript type checking
@echo -e "$(BLUE)Type checking...$(NC)"
bun run tsc --noEmit
check: ## Run all checks (Biome lint + format + type check)
@echo -e "$(BLUE)Running all checks...$(NC)"
bun run check
@echo -e "$(GREEN)✓ All checks passed$(NC)"
# =============================================================================
# TESTING
# =============================================================================
.PHONY: test test-backend test-frontend
test: test-backend test-frontend ## Run all tests
@echo -e "$(GREEN)✓ All tests passed$(NC)"
test-backend: ## Run Python backend tests (requires pytest)
@echo -e "$(BLUE)Running backend tests...$(NC)"
@if [ -f "$(VENV_BIN)/pytest" ]; then \
cd $(BACKEND_DIR) && $(VENV_BIN)/pytest -v; \
else \
echo -e "$(YELLOW)pytest not installed. Run: $(PIP) install pytest$(NC)"; \
exit 1; \
fi
test-frontend: ## Run frontend tests (requires test script in package.json)
@echo -e "$(BLUE)Running frontend tests...$(NC)"
@if bun run test --help >/dev/null 2>&1; then \
bun run test; \
else \
echo -e "$(YELLOW)No test script configured$(NC)"; \
exit 1; \
fi
# =============================================================================
# LOGS & DEBUGGING
# =============================================================================
.PHONY: logs docs
logs: ## Tail backend logs
@echo -e "$(BLUE)Tailing logs (Ctrl+C to stop)...$(NC)"
tail -f $(BACKEND_DIR)/logs/*.log 2>/dev/null || echo "No log files found"
docs: ## Open API documentation (backend must be running)
@echo -e "$(BLUE)Opening API docs...$(NC)"
open http://localhost:17493/docs 2>/dev/null || xdg-open http://localhost:17493/docs
# =============================================================================
# CLEAN
# =============================================================================
.PHONY: clean clean-python clean-build clean-all
clean: ## Clean build artifacts
@echo -e "$(BLUE)Cleaning build artifacts...$(NC)"
rm -rf $(TAURI_DIR)/src-tauri/target/release
rm -rf $(WEB_DIR)/dist
rm -rf $(APP_DIR)/dist
@echo -e "$(GREEN)✓ Build artifacts cleaned$(NC)"
clean-python: ## Clean Python cache and virtual environment
@echo -e "$(BLUE)Cleaning Python files...$(NC)"
rm -rf $(VENV)
find $(BACKEND_DIR) -type d -name "__pycache__" -exec rm -rf {} + 2>/dev/null || true
find $(BACKEND_DIR) -type f -name "*.pyc" -delete 2>/dev/null || true
@echo -e "$(GREEN)✓ Python environment cleaned$(NC)"
clean-build: ## Clean Rust/Tauri build cache
@echo -e "$(BLUE)Cleaning Rust build cache...$(NC)"
cd $(TAURI_DIR)/src-tauri && cargo clean
@echo -e "$(GREEN)✓ Rust cache cleaned$(NC)"
clean-all: clean clean-python clean-build ## Nuclear clean (everything)
@echo -e "$(BLUE)Cleaning node_modules...$(NC)"
rm -rf node_modules
rm -rf $(APP_DIR)/node_modules
rm -rf $(TAURI_DIR)/node_modules
rm -rf $(WEB_DIR)/node_modules
@echo -e "$(GREEN)✓ Full clean complete$(NC)"
+4 -4
View File
@@ -31,7 +31,7 @@ Two-part fix:
## Testing ## Testing
To test this fix: To test this fix:
1. Build Voicebox from source: `make build` 1. Build Voicebox from source: `just build`
2. Disconnect from internet 2. Disconnect from internet
3. Try generating speech 3. Try generating speech
4. Should work without network requests 4. Should work without network requests
@@ -40,13 +40,13 @@ To test this fix:
```bash ```bash
# Install dependencies # Install dependencies
pip install -r requirements.txt just setup
# Build the app # Build the app
make build just build
# Or build just the server # Or build just the server
make build-server just build-server
``` ```
## Notes ## Notes
+121 -93
View File
@@ -6,7 +6,7 @@
<p align="center"> <p align="center">
<strong>The open-source voice synthesis studio.</strong><br/> <strong>The open-source voice synthesis studio.</strong><br/>
Clone voices. Generate speech. Build voice-powered apps.<br/> Clone voices. Generate speech. Apply effects. Build voice-powered apps.<br/>
All running locally on your machine. All running locally on your machine.
</p> </p>
@@ -59,96 +59,147 @@
## What is Voicebox? ## What is Voicebox?
Voicebox is a **local-first voice cloning studio** with DAW-like features for professional voice synthesis. Think of it as a **local, free and open-source alternative to ElevenLabs** — download models, clone voices, and generate speech entirely on your machine. Voicebox is a **local-first voice cloning studio** — a free and open-source alternative to ElevenLabs. Clone voices from a few seconds of audio, generate speech in 23 languages across 4 TTS engines, apply post-processing effects, and compose multi-voice projects with a timeline editor.
Unlike cloud services that lock your voice data behind subscriptions, Voicebox gives you:
- **Complete privacy** — models and voice data stay on your machine - **Complete privacy** — models and voice data stay on your machine
- **Professional tools** — multi-track timeline editor, audio trimming, conversation mixing - **4 TTS engines** — Qwen3-TTS, LuxTTS, Chatterbox Multilingual, and Chatterbox Turbo
- **Model flexibility** — currently powered by Qwen3-TTS, with support for XTTS, Bark, and other models coming soon - **23 languages** — from English to Arabic, Japanese, Hindi, Swahili, and more
- **API-first** — use the desktop app or integrate voice synthesis into your own projects - **Post-processing effects** — pitch shift, reverb, delay, chorus, compression, and filters
- **Expressive speech** — paralinguistic tags like `[laugh]`, `[sigh]`, `[gasp]` via Chatterbox Turbo
- **Unlimited length** — auto-chunking with crossfade for scripts, articles, and chapters
- **Stories editor** — multi-track timeline for conversations, podcasts, and narratives
- **API-first** — REST API for integrating voice synthesis into your own projects
- **Native performance** — built with Tauri (Rust), not Electron - **Native performance** — built with Tauri (Rust), not Electron
- **Super fast on Mac** — MLX backend with native Metal acceleration for 4-5x faster inference on Apple Silicon - **Runs everywhere** — macOS (MLX/Metal), Windows (CUDA), Linux, AMD ROCm, Intel Arc, Docker
Download a voice model, clone any voice from a few seconds of audio, and compose multi-voice projects with studio-grade editing tools. No Python install required, no cloud dependency, no limits.
--- ---
## Download ## Download
Voicebox is available now for macOS and Windows.
| Platform | Download | | Platform | Download |
|----------|----------| |----------|----------|
| macOS (Apple Silicon) | [Voicebox_aarch64.app.tar.gz](https://github.com/jamiepine/voicebox/releases/latest/download/Voicebox_aarch64.app.tar.gz) | | macOS (Apple Silicon) | [Download DMG](https://voicebox.sh/download/mac-arm) |
| macOS (Intel) | [Voicebox_x64.app.tar.gz](https://github.com/jamiepine/voicebox/releases/latest/download/Voicebox_x64.app.tar.gz) | | macOS (Intel) | [Download DMG](https://voicebox.sh/download/mac-intel) |
| Windows (MSI) | [Latest Windows MSI](https://github.com/jamiepine/voicebox/releases/latest) | | Windows | [Download MSI](https://voicebox.sh/download/windows) |
| Windows (Setup) | [Latest Windows Setup](https://github.com/jamiepine/voicebox/releases/latest) | | Docker | `docker compose up` |
> **Linux builds coming soon** — Currently blocked by GitHub runner disk space limitations. > **[View all binaries →](https://github.com/jamiepine/voicebox/releases/latest)**
> **Linux** — Pre-built binaries are not yet available. See [voicebox.sh/linux-install](https://voicebox.sh/linux-install) for build-from-source instructions.
--- ---
## Features ## Features
### Voice Cloning with Qwen3-TTS ### Multi-Engine Voice Cloning
Powered by Alibaba's **Qwen3-TTS** — a breakthrough model that achieves near-perfect voice cloning from just a few seconds of audio. Four TTS engines with different strengths, switchable per-generation:
- **Instant cloning** — Upload a sample, get a voice profile | Engine | Languages | Strengths |
- **High fidelity** — Natural prosody, emotion, and cadence |--------|-----------|-----------|
- **Multi-language** — English, Chinese, and more coming | **Qwen3-TTS** (0.6B / 1.7B) | 10 | High-quality multilingual cloning, delivery instructions ("speak slowly", "whisper") |
- **Lightning fast on Mac** — MLX backend leverages Apple Silicon's Neural Engine for super-fast generation | **LuxTTS** | English | Lightweight (~1GB VRAM), 48kHz output, 150x realtime on CPU |
| **Chatterbox Multilingual** | 23 | Broadest language coverage — Arabic, Danish, Finnish, Greek, Hebrew, Hindi, Malay, Norwegian, Polish, Swahili, Swedish, Turkish and more |
| **Chatterbox Turbo** | English | Fast 350M model with paralinguistic emotion/sound tags |
### Emotions & Paralinguistic Tags
Type `/` in the text input to insert expressive tags that the model synthesizes inline with speech (Chatterbox Turbo):
`[laugh]` `[chuckle]` `[gasp]` `[cough]` `[sigh]` `[groan]` `[sniff]` `[shush]` `[clear throat]`
### Post-Processing Effects
8 audio effects powered by Spotify's `pedalboard` library. Apply after generation, preview in real time, build reusable presets.
| Effect | Description |
|--------|-------------|
| Pitch Shift | Up or down by up to 12 semitones |
| Reverb | Configurable room size, damping, wet/dry mix |
| Delay | Echo with adjustable time, feedback, and mix |
| Chorus / Flanger | Modulated delay for metallic or lush textures |
| Compressor | Dynamic range compression |
| Gain | Volume adjustment (-40 to +40 dB) |
| High-Pass Filter | Remove low frequencies |
| Low-Pass Filter | Remove high frequencies |
Ships with 4 built-in presets (Robotic, Radio, Echo Chamber, Deep Voice) and supports custom presets. Effects can be assigned per-profile as defaults.
### Unlimited Generation Length
Text is automatically split at sentence boundaries and each chunk is generated independently, then crossfaded together. Works with all engines.
- Configurable auto-chunking limit (100–5,000 chars)
- Crossfade slider (0–200ms) for smooth transitions
- Max text length: 50,000 characters
- Smart splitting respects abbreviations, CJK punctuation, and `[tags]`
### Generation Versions
Every generation supports multiple versions with provenance tracking:
- **Original** — clean TTS output, always preserved
- **Effects versions** — apply different effects chains from any source version
- **Takes** — regenerate with a new seed for variation
- **Source tracking** — each version records its lineage
- **Favorites** — star generations for quick access
### Async Generation Queue
Generation is non-blocking. Submit and immediately start typing the next one.
- Serial execution queue prevents GPU contention
- Real-time SSE status streaming
- Failed generations can be retried
- Stale generations from crashes auto-recover on startup
### Voice Profile Management ### Voice Profile Management
- **Create profiles** from audio files or record directly in-app - Create profiles from audio files or record directly in-app
- **Import/Export** profiles to share or back up - Import/export profiles to share or back up
- **Multi-sample support** — combine multiple samples for higher quality cloning - Multi-sample support for higher quality cloning
- **Organize** with descriptions and language tags - Per-profile default effects chains
- Organize with descriptions and language tags
### Speech Generation
- **Text-to-speech** with any cloned voice
- **Batch generation** for long-form content
- **Smart caching** — regenerate instantly with voice prompt caching
### Stories Editor ### Stories Editor
Create multi-voice narratives, podcasts, and conversations with a timeline-based editor. Multi-voice timeline editor for conversations, podcasts, and narratives.
- **Multi-track composition** — arrange multiple voice tracks in a single project - Multi-track composition with drag-and-drop
- **Inline audio editing** — trim and split clips directly in the timeline - Inline audio trimming and splitting
- **Auto-playback** — preview stories with synchronized playhead - Auto-playback with synchronized playhead
- **Voice mixing** — build conversations with multiple participants - Version pinning per track clip
### Recording & Transcription ### Recording & Transcription
- **In-app recording** with waveform visualization - In-app recording with waveform visualization
- **System audio capture** — record desktop audio on macOS and Windows - System audio capture (macOS and Windows)
- **Automatic transcription** powered by Whisper - Automatic transcription powered by Whisper (including Whisper Turbo)
- **Export recordings** in multiple formats - Export recordings in multiple formats
### Generation History ### Model Management
- **Full history** of all generated audio - Per-model unload to free GPU memory without deleting downloads
- **Search & filter** by voice, text, or date - Custom models directory via `VOICEBOX_MODELS_DIR`
- **Re-generate** any past generation with one click - Model folder migration with progress tracking
- Download cancel/clear UI
### Flexible Deployment ### GPU Support
- **Local mode** — Everything runs on your machine | Platform | Backend | Notes |
- **Remote mode** — Connect to a GPU server on your network |----------|---------|-------|
- **One-click server** — Turn any machine into a Voicebox server | macOS (Apple Silicon) | MLX (Metal) | 4-5x faster via Neural Engine |
| Windows / Linux (NVIDIA) | PyTorch (CUDA) | Auto-downloads CUDA binary from within the app |
| Linux (AMD) | PyTorch (ROCm) | Auto-configures HSA_OVERRIDE_GFX_VERSION |
| Windows (any GPU) | DirectML | Universal Windows GPU support |
| Intel Arc | IPEX/XPU | Intel discrete GPU acceleration |
| Any | CPU | Works everywhere, just slower |
--- ---
## API ## API
Voicebox exposes a full REST API, so you can integrate voice synthesis into your own apps. Voicebox exposes a full REST API for integrating voice synthesis into your own apps.
For the current local app and development workflow, the backend is typically available at `http://localhost:17493`.
If you launch the backend manually with a different host or port, use that address instead.
```bash ```bash
# Generate speech # Generate speech
@@ -165,15 +216,9 @@ curl -X POST http://localhost:17493/profiles \
-d '{"name": "My Voice", "language": "en"}' -d '{"name": "My Voice", "language": "en"}'
``` ```
**Use cases:** **Use cases:** game dialogue, podcast production, accessibility tools, voice assistants, content automation.
- Game dialogue systems Full API documentation available at `http://localhost:17493/docs`.
- Podcast/video production pipelines
- Accessibility tools
- Voice assistants
- Content creation automation
Full API documentation is available at `http://localhost:17493/docs` in the default local workflow, or at `/docs` on whatever server address you configured.
--- ---
@@ -185,42 +230,24 @@ Full API documentation is available at `http://localhost:17493/docs` in the defa
| Frontend | React, TypeScript, Tailwind CSS | | Frontend | React, TypeScript, Tailwind CSS |
| State | Zustand, React Query | | State | Zustand, React Query |
| Backend | FastAPI (Python) | | Backend | FastAPI (Python) |
| Voice Model | Qwen3-TTS (PyTorch or MLX) | | TTS Engines | Qwen3-TTS, LuxTTS, Chatterbox, Chatterbox Turbo |
| Transcription | Whisper (PyTorch or MLX) | | Effects | Pedalboard (Spotify) |
| Inference Engine | MLX (Apple Silicon) / PyTorch (Windows/Linux/Intel) | | Transcription | Whisper / Whisper Turbo (PyTorch or MLX) |
| Inference | MLX (Apple Silicon) / PyTorch (CUDA/ROCm/XPU/CPU) |
| Database | SQLite | | Database | SQLite |
| Audio | WaveSurfer.js, librosa | | Audio | WaveSurfer.js, librosa |
**Why this stack?**
- **Tauri over Electron** — 10x smaller bundle, native performance, lower memory
- **FastAPI** — Async Python with automatic OpenAPI schema generation
- **Type-safe end-to-end** — Generated TypeScript client from OpenAPI spec
--- ---
## Roadmap ## Roadmap
Voicebox is the beginning of something bigger. Here's what's coming:
### Coming Soon
| Feature | Description | | Feature | Description |
|---------|-------------| |---------|-------------|
| **Real-time Synthesis** | Stream audio as it generates, word by word | | **Real-time Streaming** | Stream audio as it generates, word by word |
| **Conversation Mode** | Multi-speaker dialogues with automatic turn-taking | | **Voice Design** | Create new voices from text descriptions |
| **Voice Effects** | Pitch shift, reverb, M3GAN-style effects |
| **Timeline Editor** | Audio studio with word-level precision editing |
| **More Models** | XTTS, Bark, and other open-source voice models | | **More Models** | XTTS, Bark, and other open-source voice models |
| **Plugin Architecture** | Extend with custom models and effects |
### Future Vision | **Mobile Companion** | Control Voicebox from your phone |
- **Voice Design** — Create new voices from text descriptions
- **Project System** — Save and load complex multi-voice sessions
- **Plugin Architecture** — Extend with custom models and effects
- **Mobile Companion** — Control Voicebox from your phone
Voicebox aims to be the **one-stop shop for everything voice** — cloning, synthesis, editing, effects, and beyond.
--- ---
@@ -240,13 +267,14 @@ just dev # starts backend + desktop app
Install [just](https://github.com/casey/just): `brew install just` or `cargo install just`. Run `just --list` to see all commands. Install [just](https://github.com/casey/just): `brew install just` or `cargo install just`. Run `just --list` to see all commands.
Also available via Makefile: `make setup && make dev` (run `make help` for all commands). **Prerequisites:** [Bun](https://bun.sh), [Rust](https://rustup.rs), [Python 3.11+](https://python.org), [Tauri Prerequisites](https://v2.tauri.app/start/prerequisites/), and [Xcode](https://developer.apple.com/xcode/) on macOS.
**Prerequisites:** [Bun](https://bun.sh), [Rust](https://rustup.rs), [Python 3.11+](https://python.org), [XCode on macOS](https://developer.apple.com/xcode/), [Tauri Prerequisites](https://v2.tauri.app/start/prerequisites/). ### Building Locally
**Performance:** ```bash
- **Apple Silicon (M1/M2/M3)**: Uses MLX backend with native Metal acceleration for 4-5x faster inference just build # Build CPU server binary + Tauri app
- **Windows/Linux/Intel Mac**: Uses PyTorch backend (CUDA GPU recommended, CPU supported but slower) just build-local # (Windows) Build CPU + CUDA server binaries + Tauri app
```
### Project Structure ### Project Structure
+1 -1
View File
@@ -1,6 +1,6 @@
{ {
"name": "@voicebox/app", "name": "@voicebox/app",
"version": "0.1.13", "version": "0.2.3",
"private": true, "private": true,
"type": "module", "type": "module",
"scripts": { "scripts": {
+45 -134
View File
@@ -139,7 +139,11 @@ export function AudioPlayer() {
barRadius: 2, barRadius: 2,
height: 80, height: 80,
normalize: true, normalize: true,
backend: 'WebAudio', // Use MediaElement backend (default). Unlike the WebAudio backend,
// MediaElement uses a standard <audio> element for playback which
// benefits from the browser/webview's built-in audio session recovery.
// This prevents audio loss when another app steals audio output or
// the system audio session is interrupted.
interact: true, // Enable interaction (click to seek) interact: true, // Enable interaction (click to seek)
mediaControls: false, // Don't show native controls mediaControls: false, // Don't show native controls
}); });
@@ -157,8 +161,21 @@ export function AudioPlayer() {
const wavesurfer = wavesurferRef.current; const wavesurfer = wavesurferRef.current;
if (!wavesurfer) return; if (!wavesurfer) return;
// Update store when time changes // Update store when time changes, stop if past duration
wavesurfer.on('timeupdate', (time) => { wavesurfer.on('timeupdate', (time) => {
const dur = usePlayerStore.getState().duration;
if (dur > 0 && time >= dur) {
setCurrentTime(dur);
const loop = usePlayerStore.getState().isLooping;
if (loop) {
wavesurfer.seekTo(0);
wavesurfer.play();
} else {
wavesurfer.pause();
setIsPlaying(false);
}
return;
}
setCurrentTime(time); setCurrentTime(time);
}); });
@@ -176,15 +193,6 @@ export function AudioPlayer() {
const currentVolume = usePlayerStore.getState().volume; const currentVolume = usePlayerStore.getState().volume;
wavesurfer.setVolume(currentVolume); wavesurfer.setVolume(currentVolume);
// Get the underlying audio element and ensure it's not muted
// (unless we're using native playback, which will be set later)
const mediaElement = wavesurfer.getMediaElement();
if (mediaElement && !isUsingNativePlaybackRef.current) {
mediaElement.volume = currentVolume;
mediaElement.muted = false;
debug.log('Audio element volume:', mediaElement.volume, 'muted:', mediaElement.muted);
}
// Auto-play when ready - check if we should use native playback // Auto-play when ready - check if we should use native playback
// Get current values from the store and queries at runtime (not captured closure values) // Get current values from the store and queries at runtime (not captured closure values)
const currentAudioUrl = usePlayerStore.getState().audioUrl; const currentAudioUrl = usePlayerStore.getState().audioUrl;
@@ -251,21 +259,8 @@ export function AudioPlayer() {
debug.log('Should use native playback:', shouldUseNative); debug.log('Should use native playback:', shouldUseNative);
if (!shouldUseNative) { if (!shouldUseNative) {
debug.log('No custom devices assigned, falling back to WaveSurfer'); debug.log('No custom devices assigned, using standard playback');
// Reset native playback flag and unmute WaveSurfer
isUsingNativePlaybackRef.current = false; isUsingNativePlaybackRef.current = false;
const mediaElement = wavesurfer.getMediaElement();
if (mediaElement) {
const currentVolume = usePlayerStore.getState().volume;
mediaElement.volume = currentVolume;
mediaElement.muted = false;
debug.log(
'WaveSurfer unmuted for normal playback - volume:',
mediaElement.volume,
'muted:',
mediaElement.muted,
);
}
} else { } else {
const deviceIds = assignedChannels.flatMap((ch: any) => ch.device_ids); const deviceIds = assignedChannels.flatMap((ch: any) => ch.device_ids);
debug.log('Device IDs to play to:', deviceIds); debug.log('Device IDs to play to:', deviceIds);
@@ -286,19 +281,10 @@ export function AudioPlayer() {
// Mark that we're using native playback // Mark that we're using native playback
isUsingNativePlaybackRef.current = true; isUsingNativePlaybackRef.current = true;
// Mute WaveSurfer's audio element to prevent UI audio output // Mute WaveSurfer's audio output — native handles the actual sound
// Keep WaveSurfer running for visualization // Keep WaveSurfer running for waveform visualization
const mediaElement = wavesurfer.getMediaElement(); wavesurfer.setVolume(0);
if (mediaElement) { wavesurfer.setMuted(true);
mediaElement.volume = 0;
mediaElement.muted = true;
debug.log(
'WaveSurfer muted for native playback - volume:',
mediaElement.volume,
'muted:',
mediaElement.muted,
);
}
// Start WaveSurfer playback for visualization (muted) // Start WaveSurfer playback for visualization (muted)
wavesurfer.play().catch((error) => { wavesurfer.play().catch((error) => {
@@ -321,38 +307,15 @@ export function AudioPlayer() {
'Native playback failed during auto-play, falling back to WaveSurfer:', 'Native playback failed during auto-play, falling back to WaveSurfer:',
error, error,
); );
// Reset native playback flag and unmute WaveSurfer
isUsingNativePlaybackRef.current = false; isUsingNativePlaybackRef.current = false;
const mediaElement = wavesurfer.getMediaElement();
if (mediaElement) {
const currentVolume = usePlayerStore.getState().volume;
mediaElement.volume = currentVolume;
mediaElement.muted = false;
debug.log(
'WaveSurfer unmuted after native playback failure - volume:',
mediaElement.volume,
'muted:',
mediaElement.muted,
);
}
// Fall through to WaveSurfer playback // Fall through to WaveSurfer playback
} }
} else { }
debug.log('Not using native playback, using WaveSurfer');
// Reset native playback flag and unmute WaveSurfer // Standard playback path — ensure WaveSurfer is unmuted
isUsingNativePlaybackRef.current = false; if (!isUsingNativePlaybackRef.current) {
const mediaElement = wavesurfer.getMediaElement(); wavesurfer.setMuted(false);
if (mediaElement) { wavesurfer.setVolume(usePlayerStore.getState().volume);
const currentVolume = usePlayerStore.getState().volume;
mediaElement.volume = currentVolume;
mediaElement.muted = false;
debug.log(
'WaveSurfer unmuted for normal playback - volume:',
mediaElement.volume,
'muted:',
mediaElement.muted,
);
}
} }
// Only auto-play if shouldAutoPlay flag is set (user explicitly clicked to play) // Only auto-play if shouldAutoPlay flag is set (user explicitly clicked to play)
@@ -376,28 +339,6 @@ export function AudioPlayer() {
// Handle play/pause // Handle play/pause
wavesurfer.on('play', () => { wavesurfer.on('play', () => {
setIsPlaying(true); setIsPlaying(true);
// Ensure audio element volume is set correctly
const mediaElement = wavesurfer.getMediaElement();
if (mediaElement) {
// Double-check: if using native playback, keep WaveSurfer muted
// Otherwise, ensure it's unmuted
if (isUsingNativePlaybackRef.current) {
mediaElement.volume = 0;
mediaElement.muted = true;
debug.log('Playing (native mode) - WaveSurfer muted for visualization only');
} else {
// Ensure WaveSurfer is unmuted for normal playback
const currentVolume = usePlayerStore.getState().volume;
mediaElement.volume = currentVolume;
mediaElement.muted = false;
debug.log(
'Playing (normal mode) - volume:',
mediaElement.volume,
'muted:',
mediaElement.muted,
);
}
}
}); });
wavesurfer.on('pause', () => setIsPlaying(false)); wavesurfer.on('pause', () => setIsPlaying(false));
wavesurfer.on('finish', () => { wavesurfer.on('finish', () => {
@@ -479,11 +420,6 @@ export function AudioPlayer() {
if (wavesurferRef.current) { if (wavesurferRef.current) {
debug.log('Destroying WaveSurfer instance'); debug.log('Destroying WaveSurfer instance');
try { try {
const mediaElement = wavesurferRef.current.getMediaElement();
if (mediaElement) {
mediaElement.pause();
mediaElement.src = '';
}
wavesurferRef.current.destroy(); wavesurferRef.current.destroy();
} catch (error) { } catch (error) {
debug.error('Error destroying WaveSurfer:', error); debug.error('Error destroying WaveSurfer:', error);
@@ -524,13 +460,10 @@ export function AudioPlayer() {
} }
// Reset native playback flag when loading new audio // Reset native playback flag when loading new audio
// Also unmute WaveSurfer if it was muted // Unmute WaveSurfer if it was muted for native playback
if (isUsingNativePlaybackRef.current) { if (isUsingNativePlaybackRef.current) {
const mediaElement = wavesurfer.getMediaElement(); wavesurfer.setMuted(false);
if (mediaElement) { wavesurfer.setVolume(usePlayerStore.getState().volume);
mediaElement.muted = false;
mediaElement.volume = usePlayerStore.getState().volume;
}
} }
isUsingNativePlaybackRef.current = false; isUsingNativePlaybackRef.current = false;
@@ -546,16 +479,7 @@ export function AudioPlayer() {
wavesurfer.pause(); wavesurfer.pause();
} }
// Stop the media element explicitly // Use empty() to completely destroy the waveform and reset media
const mediaElement = wavesurfer.getMediaElement();
if (mediaElement) {
debug.log('Stopping media element');
mediaElement.pause();
mediaElement.currentTime = 0;
mediaElement.src = '';
}
// Use empty() to completely destroy the waveform and media element
debug.log('Calling wavesurfer.empty() to destroy audio'); debug.log('Calling wavesurfer.empty() to destroy audio');
wavesurfer.empty(); wavesurfer.empty();
} catch (error) { } catch (error) {
@@ -610,20 +534,13 @@ export function AudioPlayer() {
// Sync volume // Sync volume
useEffect(() => { useEffect(() => {
if (wavesurferRef.current) { if (wavesurferRef.current) {
wavesurferRef.current.setVolume(volume); // If using native playback, keep WaveSurfer muted regardless of volume setting
// Also ensure the underlying audio element volume is set if (isUsingNativePlaybackRef.current) {
const mediaElement = wavesurferRef.current.getMediaElement(); wavesurferRef.current.setVolume(0);
if (mediaElement) { debug.log('Volume sync: Using native playback, keeping WaveSurfer muted');
// If using native playback, keep WaveSurfer muted regardless of volume setting } else {
if (isUsingNativePlaybackRef.current) { wavesurferRef.current.setVolume(volume);
mediaElement.volume = 0; debug.log('Volume synced:', volume);
mediaElement.muted = true;
debug.log('Volume sync: Using native playback, keeping WaveSurfer muted');
} else {
mediaElement.volume = volume;
mediaElement.muted = volume === 0;
debug.log('Volume synced:', volume, 'muted:', mediaElement.muted);
}
} }
} }
}, [volume]); }, [volume]);
@@ -744,11 +661,8 @@ export function AudioPlayer() {
isUsingNativePlaybackRef.current = true; isUsingNativePlaybackRef.current = true;
// Mute WaveSurfer and start it for visualization // Mute WaveSurfer and start it for visualization
const mediaElement = wavesurferRef.current.getMediaElement(); wavesurferRef.current.setVolume(0);
if (mediaElement) { wavesurferRef.current.setMuted(true);
mediaElement.volume = 0;
mediaElement.muted = true;
}
// Start WaveSurfer for visualization (muted) // Start WaveSurfer for visualization (muted)
wavesurferRef.current.play().catch((error) => { wavesurferRef.current.play().catch((error) => {
@@ -772,11 +686,8 @@ export function AudioPlayer() {
} else { } else {
// Ensure WaveSurfer is not muted if not using native playback // Ensure WaveSurfer is not muted if not using native playback
if (!isUsingNativePlaybackRef.current) { if (!isUsingNativePlaybackRef.current) {
const mediaElement = wavesurferRef.current.getMediaElement(); wavesurferRef.current.setMuted(false);
if (mediaElement) { wavesurferRef.current.setVolume(volume);
mediaElement.muted = false;
mediaElement.volume = volume;
}
} }
wavesurferRef.current.play().catch((error) => { wavesurferRef.current.play().catch((error) => {
@@ -0,0 +1,377 @@
import {
closestCenter,
DndContext,
type DragEndEvent,
KeyboardSensor,
PointerSensor,
useSensor,
useSensors,
} from '@dnd-kit/core';
import {
arrayMove,
SortableContext,
sortableKeyboardCoordinates,
useSortable,
verticalListSortingStrategy,
} from '@dnd-kit/sortable';
import { CSS } from '@dnd-kit/utilities';
import { useQuery } from '@tanstack/react-query';
import { ChevronDown, ChevronRight, GripVertical, Plus, Power, Trash2 } from 'lucide-react';
import { useCallback, useMemo, useRef, useState } from 'react';
import { Button } from '@/components/ui/button';
import { Label } from '@/components/ui/label';
import {
Select,
SelectContent,
SelectItem,
SelectTrigger,
SelectValue,
} from '@/components/ui/select';
import { Slider } from '@/components/ui/slider';
import { apiClient } from '@/lib/api/client';
import type { AvailableEffect, EffectConfig, EffectPresetResponse } from '@/lib/api/types';
import { cn } from '@/lib/utils/cn';
// Each effect in the chain gets a stable ID for dnd-kit
interface EffectWithId extends EffectConfig {
_id: string;
}
let nextId = 0;
function makeId() {
return `fx-${++nextId}`;
}
interface EffectsChainEditorProps {
value: EffectConfig[];
onChange: (chain: EffectConfig[]) => void;
compact?: boolean;
showPresets?: boolean;
}
export function EffectsChainEditor({
value,
onChange,
compact = false,
showPresets = true,
}: EffectsChainEditorProps) {
const [expandedId, setExpandedId] = useState<string | null>(null);
// Maintain stable IDs for each effect across renders.
// We use a ref to map value items to IDs, rebuilding when length changes.
const idsRef = useRef<string[]>([]);
const items: EffectWithId[] = useMemo(() => {
// Grow ID array if effects were added
while (idsRef.current.length < value.length) {
idsRef.current.push(makeId());
}
// Shrink if effects were removed
if (idsRef.current.length > value.length) {
idsRef.current = idsRef.current.slice(0, value.length);
}
return value.map((e, i) => ({ ...e, _id: idsRef.current[i] }));
}, [value]);
const sensors = useSensors(
useSensor(PointerSensor, { activationConstraint: { distance: 5 } }),
useSensor(KeyboardSensor, { coordinateGetter: sortableKeyboardCoordinates }),
);
const { data: availableEffects } = useQuery({
queryKey: ['available-effects'],
queryFn: () => apiClient.getAvailableEffects(),
staleTime: Infinity,
});
const { data: presets } = useQuery({
queryKey: ['effect-presets'],
queryFn: () => apiClient.listEffectPresets(),
staleTime: 30_000,
});
const effectsMap = useMemo(() => {
const m = new Map<string, AvailableEffect>();
if (availableEffects) {
for (const e of availableEffects.effects) {
m.set(e.type, e);
}
}
return m;
}, [availableEffects]);
function addEffect(type: string) {
const def = effectsMap.get(type);
if (!def) return;
const params: Record<string, number> = {};
for (const [key, p] of Object.entries(def.params)) {
params[key] = p.default;
}
const newEffect: EffectConfig = { type, enabled: true, params };
const newId = makeId();
idsRef.current = [...idsRef.current, newId];
onChange([...value, newEffect]);
setExpandedId(newId);
}
const removeEffect = useCallback(
(index: number) => {
const removedId = idsRef.current[index];
idsRef.current = idsRef.current.filter((_, i) => i !== index);
onChange(value.filter((_, i) => i !== index));
if (expandedId === removedId) setExpandedId(null);
},
[value, onChange, expandedId],
);
const toggleEnabled = useCallback(
(index: number) => {
onChange(value.map((e, i) => (i === index ? { ...e, enabled: !e.enabled } : e)));
},
[value, onChange],
);
const updateParam = useCallback(
(index: number, paramName: string, paramValue: number) => {
onChange(
value.map((e, i) =>
i === index ? { ...e, params: { ...e.params, [paramName]: paramValue } } : e,
),
);
},
[value, onChange],
);
function loadPreset(preset: EffectPresetResponse) {
idsRef.current = preset.effects_chain.map(() => makeId());
onChange(preset.effects_chain);
setExpandedId(null);
}
function clearAll() {
idsRef.current = [];
onChange([]);
setExpandedId(null);
}
function handleDragEnd(event: DragEndEvent) {
const { active, over } = event;
if (!over || active.id === over.id) return;
const oldIndex = idsRef.current.indexOf(active.id as string);
const newIndex = idsRef.current.indexOf(over.id as string);
if (oldIndex === -1 || newIndex === -1) return;
idsRef.current = arrayMove(idsRef.current, oldIndex, newIndex);
onChange(arrayMove([...value], oldIndex, newIndex));
}
return (
<div className={cn('space-y-2', compact && 'text-sm')}>
{/* Preset selector row */}
{showPresets && (
<div className="flex items-center gap-2">
<Select
onValueChange={(id) => {
const preset = presets?.find((p) => p.id === id);
if (preset) loadPreset(preset);
}}
>
<SelectTrigger className="h-8 flex-1 text-xs focus:ring-0 focus:ring-offset-0">
<SelectValue placeholder="Load preset..." />
</SelectTrigger>
<SelectContent>
{presets?.map((p) => (
<SelectItem key={p.id} value={p.id}>
{p.name}
{p.description && (
<span className="ml-1 text-muted-foreground">- {p.description}</span>
)}
</SelectItem>
))}
</SelectContent>
</Select>
{value.length > 0 && (
<Button
variant="ghost"
size="sm"
className="h-8 px-2 text-xs text-muted-foreground"
onClick={clearAll}
>
Clear
</Button>
)}
</div>
)}
{/* Sortable effects chain */}
<DndContext sensors={sensors} collisionDetection={closestCenter} onDragEnd={handleDragEnd}>
<SortableContext items={items.map((i) => i._id)} strategy={verticalListSortingStrategy}>
{items.map((effect, index) => (
<SortableEffectItem
key={effect._id}
id={effect._id}
effect={effect}
index={index}
effectDef={effectsMap.get(effect.type)}
isExpanded={expandedId === effect._id}
onToggleExpand={() => setExpandedId(expandedId === effect._id ? null : effect._id)}
onRemove={() => removeEffect(index)}
onToggleEnabled={() => toggleEnabled(index)}
onUpdateParam={(paramName, paramValue) => updateParam(index, paramName, paramValue)}
/>
))}
</SortableContext>
</DndContext>
{/* Add effect */}
{availableEffects && (
<Select onValueChange={addEffect}>
<SelectTrigger className="h-8 border-dashed text-xs text-muted-foreground focus:ring-0 focus:ring-offset-0">
<Plus className="mr-1 h-3.5 w-3.5" />
<SelectValue placeholder="Add effect..." />
</SelectTrigger>
<SelectContent>
{availableEffects.effects.map((e) => (
<SelectItem key={e.type} value={e.type}>
{e.label}
</SelectItem>
))}
</SelectContent>
</Select>
)}
</div>
);
}
// ---------------------------------------------------------------------------
// Sortable effect item
// ---------------------------------------------------------------------------
interface SortableEffectItemProps {
id: string;
effect: EffectConfig;
index: number;
effectDef?: AvailableEffect;
isExpanded: boolean;
onToggleExpand: () => void;
onRemove: () => void;
onToggleEnabled: () => void;
onUpdateParam: (paramName: string, paramValue: number) => void;
}
function SortableEffectItem({
id,
effect,
effectDef,
isExpanded,
onToggleExpand,
onRemove,
onToggleEnabled,
onUpdateParam,
}: SortableEffectItemProps) {
const { attributes, listeners, setNodeRef, transform, transition, isDragging } = useSortable({
id,
});
const style = {
transform: CSS.Transform.toString(transform),
transition,
zIndex: isDragging ? 10 : undefined,
};
const label = effectDef?.label ?? effect.type;
return (
<div
ref={setNodeRef}
style={style}
className={cn(
'rounded-md border',
effect.enabled ? 'border-border bg-card' : 'border-border/50 bg-muted/30',
isDragging && 'opacity-80 shadow-lg',
)}
>
{/* Header */}
<div className="flex items-center gap-1 px-2 py-1.5">
<button
type="button"
className="p-0.5 text-muted-foreground hover:text-foreground"
onClick={onToggleExpand}
>
{isExpanded ? (
<ChevronDown className="h-3.5 w-3.5" />
) : (
<ChevronRight className="h-3.5 w-3.5" />
)}
</button>
<button
type="button"
className="p-0.5 text-muted-foreground/50 hover:text-muted-foreground cursor-grab active:cursor-grabbing touch-none"
{...attributes}
{...listeners}
>
<GripVertical className="h-3.5 w-3.5" />
</button>
<span
className={cn('flex-1 text-xs font-medium', !effect.enabled && 'text-muted-foreground')}
>
{label}
</span>
<button
type="button"
className={cn(
'p-0.5 transition-colors',
effect.enabled ? 'text-primary' : 'text-muted-foreground hover:text-foreground',
)}
onClick={onToggleEnabled}
title={effect.enabled ? 'Disable' : 'Enable'}
>
<Power className="h-3.5 w-3.5" />
</button>
<button
type="button"
className="p-0.5 text-muted-foreground hover:text-destructive"
onClick={onRemove}
title="Remove"
>
<Trash2 className="h-3.5 w-3.5" />
</button>
</div>
{/* Params */}
{isExpanded && effectDef && (
<div className="space-y-3 border-t px-3 py-2.5">
{Object.entries(effectDef.params).map(([paramName, paramDef]) => {
const currentValue = effect.params[paramName] ?? paramDef.default;
return (
<div key={paramName} className="space-y-1">
<div className="flex items-center justify-between">
<Label className="text-[11px] text-muted-foreground">
{paramDef.description}
</Label>
<span className="text-[11px] font-mono tabular-nums text-foreground">
{currentValue.toFixed(
paramDef.step < 1 ? Math.max(1, -Math.floor(Math.log10(paramDef.step))) : 0,
)}
</span>
</div>
<Slider
min={paramDef.min}
max={paramDef.max}
step={paramDef.step}
value={[currentValue]}
onValueChange={([v]) => onUpdateParam(paramName, v)}
/>
</div>
);
})}
</div>
)}
</div>
);
}
@@ -0,0 +1,103 @@
import { ChevronDown, Search } from 'lucide-react';
import { useMemo, useState } from 'react';
import { Button } from '@/components/ui/button';
import { Input } from '@/components/ui/input';
import { Popover, PopoverContent, PopoverTrigger } from '@/components/ui/popover';
import type { HistoryResponse } from '@/lib/api/types';
import { useHistory } from '@/lib/hooks/useHistory';
import { cn } from '@/lib/utils/cn';
interface GenerationPickerProps {
selectedId: string | null;
onSelect: (generation: HistoryResponse) => void;
className?: string;
}
export function GenerationPicker({ selectedId, onSelect, className }: GenerationPickerProps) {
const [open, setOpen] = useState(false);
const [searchQuery, setSearchQuery] = useState('');
const { data: historyData } = useHistory({ limit: 50 });
const completedGenerations = useMemo(() => {
if (!historyData?.items) return [];
return historyData.items.filter((gen) => gen.status === 'completed');
}, [historyData]);
const filtered = useMemo(() => {
if (!searchQuery) return completedGenerations;
const q = searchQuery.toLowerCase();
return completedGenerations.filter(
(gen) => gen.text.toLowerCase().includes(q) || gen.profile_name.toLowerCase().includes(q),
);
}, [completedGenerations, searchQuery]);
const selectedGeneration = completedGenerations.find((g) => g.id === selectedId);
return (
<Popover open={open} onOpenChange={setOpen}>
<PopoverTrigger asChild>
<Button
variant="outline"
size="sm"
className={cn('h-8 justify-between gap-2 text-xs font-normal', className)}
>
{selectedGeneration ? (
<span className="truncate">
<span className="font-medium">{selectedGeneration.profile_name}</span>
<span className="text-muted-foreground ml-1.5">
{selectedGeneration.text.length > 30
? `${selectedGeneration.text.substring(0, 30)}...`
: selectedGeneration.text}
</span>
</span>
) : (
<span className="text-muted-foreground">Select a generation...</span>
)}
<ChevronDown className="h-3.5 w-3.5 shrink-0 opacity-50" />
</Button>
</PopoverTrigger>
<PopoverContent className="w-80 p-0" align="start">
<div className="p-2 border-b">
<div className="relative">
<Search className="absolute left-2 top-1/2 -translate-y-1/2 h-3.5 w-3.5 text-muted-foreground" />
<Input
placeholder="Search by voice or text..."
value={searchQuery}
onChange={(e) => setSearchQuery(e.target.value)}
className="h-8 pl-7 text-xs"
/>
</div>
</div>
<div className="max-h-60 overflow-y-auto">
{filtered.length === 0 ? (
<div className="p-4 text-center text-xs text-muted-foreground">
No generations found
</div>
) : (
filtered.map((gen) => (
<button
key={gen.id}
type="button"
className={cn(
'w-full text-left px-3 py-2 hover:bg-muted/50 transition-colors border-b border-border/30 last:border-0',
gen.id === selectedId && 'bg-accent/10',
)}
onClick={() => {
onSelect(gen);
setOpen(false);
setSearchQuery('');
}}
>
<div className="font-medium text-sm">{gen.profile_name}</div>
<div className="text-xs text-muted-foreground truncate">
{gen.text.length > 60 ? `${gen.text.substring(0, 60)}...` : gen.text}
</div>
</button>
))
)}
</div>
</PopoverContent>
</Popover>
);
}
@@ -0,0 +1,422 @@
import { useQuery, useQueryClient } from '@tanstack/react-query';
import { Loader2, Play, Save, Trash2, Wand2 } from 'lucide-react';
import { useEffect, useRef, useState } from 'react';
import { EffectsChainEditor } from '@/components/Effects/EffectsChainEditor';
import { GenerationPicker } from '@/components/Effects/GenerationPicker';
import { Button } from '@/components/ui/button';
import {
Dialog,
DialogContent,
DialogDescription,
DialogFooter,
DialogHeader,
DialogTitle,
} from '@/components/ui/dialog';
import { Input } from '@/components/ui/input';
import { Label } from '@/components/ui/label';
import { Separator } from '@/components/ui/separator';
import { Textarea } from '@/components/ui/textarea';
import { useToast } from '@/components/ui/use-toast';
import { apiClient } from '@/lib/api/client';
import type { HistoryResponse } from '@/lib/api/types';
import { useHistory } from '@/lib/hooks/useHistory';
import { useEffectsStore } from '@/stores/effectsStore';
import { usePlayerStore } from '@/stores/playerStore';
export function EffectsDetail() {
const selectedPresetId = useEffectsStore((s) => s.selectedPresetId);
const isCreatingNew = useEffectsStore((s) => s.isCreatingNew);
const workingChain = useEffectsStore((s) => s.workingChain);
const setWorkingChain = useEffectsStore((s) => s.setWorkingChain);
const setSelectedPresetId = useEffectsStore((s) => s.setSelectedPresetId);
const setIsCreatingNew = useEffectsStore((s) => s.setIsCreatingNew);
const [name, setName] = useState('');
const [description, setDescription] = useState('');
const [saving, setSaving] = useState(false);
const [deleting, setDeleting] = useState(false);
// "Save as Custom" dialog state
const [saveAsDialogOpen, setSaveAsDialogOpen] = useState(false);
const [saveAsName, setSaveAsName] = useState('');
const [saveAsDescription, setSaveAsDescription] = useState('');
// Preview state
const [previewGenId, setPreviewGenId] = useState<string | null>(null);
const [previewLoading, setPreviewLoading] = useState(false);
const blobUrlRef = useRef<string | null>(null);
const setAudioWithAutoPlay = usePlayerStore((s) => s.setAudioWithAutoPlay);
const { toast } = useToast();
const queryClient = useQueryClient();
// Auto-select the most recent generation as preview source
const { data: historyData } = useHistory({ limit: 1 });
useEffect(() => {
if (!previewGenId && historyData?.items?.length) {
const first = historyData.items.find((g) => g.status === 'completed');
if (first) setPreviewGenId(first.id);
}
}, [historyData, previewGenId]);
const { data: preset } = useQuery({
queryKey: ['effect-preset', selectedPresetId],
queryFn: () =>
selectedPresetId
? apiClient
.listEffectPresets()
.then((all) => all.find((p) => p.id === selectedPresetId) ?? null)
: null,
enabled: !!selectedPresetId,
staleTime: 30_000,
});
// Sync name/description when selecting a preset
useEffect(() => {
if (preset) {
setName(preset.name);
setDescription(preset.description ?? '');
} else if (isCreatingNew) {
setName('');
setDescription('');
}
}, [preset, isCreatingNew]);
// Cleanup blob URL on unmount
useEffect(() => {
return () => {
if (blobUrlRef.current) {
URL.revokeObjectURL(blobUrlRef.current);
blobUrlRef.current = null;
}
};
}, []);
const isEditing = !!selectedPresetId || isCreatingNew;
const isBuiltIn = preset?.is_builtin ?? false;
async function handlePreview() {
if (!previewGenId || workingChain.length === 0) return;
setPreviewLoading(true);
try {
const blob = await apiClient.previewEffects(previewGenId, workingChain);
// Revoke old blob URL
if (blobUrlRef.current) {
URL.revokeObjectURL(blobUrlRef.current);
}
const url = URL.createObjectURL(blob);
blobUrlRef.current = url;
// Play through the main audio player
setAudioWithAutoPlay(url, `preview-${Date.now()}`, null, 'Effects Preview');
} catch (error) {
toast({
title: 'Preview failed',
description: error instanceof Error ? error.message : 'Unknown error',
variant: 'destructive',
});
} finally {
setPreviewLoading(false);
}
}
function handleSelectGeneration(gen: HistoryResponse) {
setPreviewGenId(gen.id);
}
async function handleSaveNew() {
if (!name.trim()) {
toast({ title: 'Name required', variant: 'destructive' });
return;
}
setSaving(true);
try {
const created = await apiClient.createEffectPreset({
name: name.trim(),
description: description.trim() || undefined,
effects_chain: workingChain,
});
queryClient.invalidateQueries({ queryKey: ['effect-presets'] });
setIsCreatingNew(false);
setSelectedPresetId(created.id);
toast({ title: 'Preset saved', description: `"${created.name}" has been created.` });
} catch (error) {
toast({
title: 'Failed to save',
description: error instanceof Error ? error.message : 'Unknown error',
variant: 'destructive',
});
} finally {
setSaving(false);
}
}
async function handleSaveExisting() {
if (!selectedPresetId || !name.trim()) return;
setSaving(true);
try {
await apiClient.updateEffectPreset(selectedPresetId, {
name: name.trim(),
description: description.trim() || undefined,
effects_chain: workingChain,
});
queryClient.invalidateQueries({ queryKey: ['effect-presets'] });
queryClient.invalidateQueries({ queryKey: ['effect-preset', selectedPresetId] });
toast({ title: 'Preset updated' });
} catch (error) {
toast({
title: 'Failed to save',
description: error instanceof Error ? error.message : 'Unknown error',
variant: 'destructive',
});
} finally {
setSaving(false);
}
}
function handleSaveAsNew() {
// Open the dialog with a suggested name based on the current preset
setSaveAsName(`${name} (Copy)`);
setSaveAsDescription(description);
setSaveAsDialogOpen(true);
}
async function handleSaveAsConfirm() {
if (!saveAsName.trim()) {
toast({ title: 'Name required', variant: 'destructive' });
return;
}
setSaving(true);
try {
const created = await apiClient.createEffectPreset({
name: saveAsName.trim(),
description: saveAsDescription.trim() || undefined,
effects_chain: workingChain,
});
queryClient.invalidateQueries({ queryKey: ['effect-presets'] });
setSaveAsDialogOpen(false);
setSelectedPresetId(created.id);
toast({ title: 'Preset saved', description: `"${created.name}" has been created.` });
} catch (error) {
toast({
title: 'Failed to save',
description: error instanceof Error ? error.message : 'Unknown error',
variant: 'destructive',
});
} finally {
setSaving(false);
}
}
async function handleDelete() {
if (!selectedPresetId) return;
setDeleting(true);
try {
await apiClient.deleteEffectPreset(selectedPresetId);
queryClient.invalidateQueries({ queryKey: ['effect-presets'] });
setSelectedPresetId(null);
setWorkingChain([]);
toast({ title: 'Preset deleted' });
} catch (error) {
toast({
title: 'Failed to delete',
description: error instanceof Error ? error.message : 'Unknown error',
variant: 'destructive',
});
} finally {
setDeleting(false);
}
}
if (!isEditing) {
return (
<div className="flex-1 flex items-center justify-center text-muted-foreground">
<div className="text-center space-y-2">
<Wand2 className="h-10 w-10 mx-auto opacity-30" />
<p className="text-sm">Select a preset or create a new one</p>
</div>
</div>
);
}
return (
<div className="flex flex-col h-full min-h-0">
{/* Header */}
<div className="flex items-center justify-between mb-4">
<h2 className="text-lg font-semibold">
{isCreatingNew ? 'New Preset' : isBuiltIn ? preset?.name : 'Edit Preset'}
</h2>
<div className="flex items-center gap-2">
{!isBuiltIn && !isCreatingNew && (
<>
<Button
variant="ghost"
size="sm"
className="h-8 text-destructive hover:text-destructive gap-1.5"
onClick={handleDelete}
disabled={deleting}
>
<Trash2 className="h-3.5 w-3.5" />
{deleting ? 'Deleting...' : 'Delete'}
</Button>
<Button
size="sm"
className="h-8 gap-1.5"
onClick={handleSaveExisting}
disabled={saving || workingChain.length === 0}
>
<Save className="h-3.5 w-3.5" />
{saving ? 'Saving...' : 'Save'}
</Button>
</>
)}
{isCreatingNew && (
<Button
size="sm"
className="h-8 gap-1.5"
onClick={handleSaveNew}
disabled={saving || workingChain.length === 0}
>
<Save className="h-3.5 w-3.5" />
{saving ? 'Saving...' : 'Save Preset'}
</Button>
)}
{isBuiltIn && (
<Button
size="sm"
variant="outline"
className="h-8 gap-1.5"
onClick={handleSaveAsNew}
disabled={saving}
>
<Save className="h-3.5 w-3.5" />
{saving ? 'Saving...' : 'Save as Custom'}
</Button>
)}
</div>
</div>
{/* Scrollable content */}
<div className="flex-1 min-h-0 overflow-y-auto space-y-5 pr-1">
{/* Name & description */}
{(isCreatingNew || !isBuiltIn) && (
<div className="space-y-3">
<div className="space-y-1.5">
<Label className="text-xs">Name</Label>
<Input
value={name}
onChange={(e) => setName(e.target.value)}
placeholder="My preset..."
className="h-9"
/>
</div>
<div className="space-y-1.5">
<Label className="text-xs">Description</Label>
<Textarea
value={description}
onChange={(e) => setDescription(e.target.value)}
placeholder="Describe what this preset does..."
className="min-h-[60px] resize-none"
/>
</div>
</div>
)}
{/* Built-in description (read-only) */}
{isBuiltIn && preset?.description && (
<p className="text-sm text-muted-foreground">{preset.description}</p>
)}
{/* Effects chain editor */}
<EffectsChainEditor value={workingChain} onChange={setWorkingChain} showPresets={false} />
<Separator />
{/* Preview section */}
<div className="space-y-3">
<Label className="text-xs">Preview</Label>
<div className="flex items-center gap-2">
<GenerationPicker
selectedId={previewGenId}
onSelect={handleSelectGeneration}
className="flex-1"
/>
<Button
variant="outline"
size="sm"
className="h-8 gap-1.5 shrink-0"
onClick={handlePreview}
disabled={!previewGenId || workingChain.length === 0 || previewLoading}
>
{previewLoading ? (
<>
<Loader2 className="h-3.5 w-3.5 animate-spin" />
Processing...
</>
) : (
<>
<Play className="h-3.5 w-3.5" />
Preview
</>
)}
</Button>
</div>
<p className="text-[11px] text-muted-foreground">
Preview applies effects to the clean version without saving.
</p>
</div>
</div>
{/* Save as Custom dialog */}
<Dialog open={saveAsDialogOpen} onOpenChange={setSaveAsDialogOpen}>
<DialogContent className="sm:max-w-md">
<DialogHeader>
<DialogTitle>Save as Custom Preset</DialogTitle>
<DialogDescription>
Create a new custom preset based on the current effects chain.
</DialogDescription>
</DialogHeader>
<div className="space-y-3 py-2">
<div className="space-y-1.5">
<Label className="text-xs">Name</Label>
<Input
value={saveAsName}
onChange={(e) => setSaveAsName(e.target.value)}
placeholder="My preset..."
className="h-9"
autoFocus
onKeyDown={(e) => {
if (e.key === 'Enter' && saveAsName.trim()) {
handleSaveAsConfirm();
}
}}
/>
</div>
<div className="space-y-1.5">
<Label className="text-xs">Description</Label>
<Textarea
value={saveAsDescription}
onChange={(e) => setSaveAsDescription(e.target.value)}
placeholder="Describe what this preset does..."
className="min-h-[60px] resize-none"
/>
</div>
</div>
<DialogFooter>
<Button variant="outline" onClick={() => setSaveAsDialogOpen(false)} disabled={saving}>
Cancel
</Button>
<Button onClick={handleSaveAsConfirm} disabled={saving || !saveAsName.trim()}>
<Save className="h-3.5 w-3.5 mr-1.5" />
{saving ? 'Saving...' : 'Save'}
</Button>
</DialogFooter>
</DialogContent>
</Dialog>
</div>
);
}
@@ -0,0 +1,165 @@
import { useQuery } from '@tanstack/react-query';
import { Loader2, Plus, Sparkles, Wand2 } from 'lucide-react';
import { Button } from '@/components/ui/button';
import { apiClient } from '@/lib/api/client';
import type { EffectPresetResponse } from '@/lib/api/types';
import { cn } from '@/lib/utils/cn';
import { useEffectsStore } from '@/stores/effectsStore';
export function EffectsList() {
const selectedPresetId = useEffectsStore((s) => s.selectedPresetId);
const setSelectedPresetId = useEffectsStore((s) => s.setSelectedPresetId);
const setWorkingChain = useEffectsStore((s) => s.setWorkingChain);
const setIsCreatingNew = useEffectsStore((s) => s.setIsCreatingNew);
const isCreatingNew = useEffectsStore((s) => s.isCreatingNew);
const { data: presets, isLoading } = useQuery({
queryKey: ['effect-presets'],
queryFn: () => apiClient.listEffectPresets(),
staleTime: 30_000,
});
const builtIn = presets?.filter((p) => p.is_builtin) ?? [];
const userPresets = presets?.filter((p) => !p.is_builtin) ?? [];
function handleSelect(preset: EffectPresetResponse) {
setSelectedPresetId(preset.id);
setWorkingChain(preset.effects_chain);
}
function handleCreateNew() {
setIsCreatingNew(true);
setWorkingChain([]);
}
if (isLoading) {
return (
<div className="flex items-center justify-center h-full">
<Loader2 className="h-6 w-6 animate-spin text-muted-foreground" />
</div>
);
}
return (
<div className="flex flex-col h-full min-h-0">
{/* Header */}
<div className="flex items-center justify-between mb-4">
<h2 className="text-lg font-semibold">Effects</h2>
<Button variant="outline" size="sm" className="h-8 gap-1.5" onClick={handleCreateNew}>
<Plus className="h-3.5 w-3.5" />
New Preset
</Button>
</div>
{/* Scrollable list */}
<div className="flex-1 min-h-0 overflow-y-auto space-y-4">
{/* Built-in presets */}
{builtIn.length > 0 && (
<div>
<div className="text-[11px] text-muted-foreground font-medium uppercase tracking-wider mb-2 px-1">
Built-in
</div>
<div className="space-y-1.5">
{builtIn.map((preset) => (
<PresetCard
key={preset.id}
preset={preset}
isSelected={selectedPresetId === preset.id && !isCreatingNew}
onSelect={() => handleSelect(preset)}
/>
))}
</div>
</div>
)}
{/* User presets */}
{userPresets.length > 0 && (
<div>
<div className="text-[11px] text-muted-foreground font-medium uppercase tracking-wider mb-2 px-1">
Custom
</div>
<div className="space-y-1.5">
{userPresets.map((preset) => (
<PresetCard
key={preset.id}
preset={preset}
isSelected={selectedPresetId === preset.id && !isCreatingNew}
onSelect={() => handleSelect(preset)}
/>
))}
</div>
</div>
)}
{/* New preset placeholder */}
{isCreatingNew && (
<div>
<div className="text-[11px] text-muted-foreground font-medium uppercase tracking-wider mb-2 px-1">
New
</div>
<div className="rounded-xl border-2 border-accent/40 bg-accent/5 p-3">
<div className="flex items-center gap-2">
<Sparkles className="h-4 w-4 text-accent" />
<span className="text-sm font-medium">Unsaved Preset</span>
</div>
<p className="text-xs text-muted-foreground mt-1">
Configure effects in the panel on the right.
</p>
</div>
</div>
)}
</div>
</div>
);
}
function PresetCard({
preset,
isSelected,
onSelect,
}: {
preset: EffectPresetResponse;
isSelected: boolean;
onSelect: () => void;
}) {
const effectCount = preset.effects_chain.length;
return (
<button
type="button"
className={cn(
'w-full text-left rounded-xl border p-3 h-[88px] transition-all duration-150',
isSelected
? 'border-accent/50 bg-accent/10'
: 'border-border bg-card hover:bg-muted/50 hover:border-border',
)}
onClick={onSelect}
>
<div className="flex items-center gap-2">
<Wand2
className={cn('h-4 w-4 shrink-0', isSelected ? 'text-accent' : 'text-muted-foreground')}
/>
<span className="text-sm font-medium truncate">{preset.name}</span>
{preset.is_builtin && (
<span className="text-[10px] bg-muted text-muted-foreground px-1.5 py-0.5 rounded-full shrink-0">
built-in
</span>
)}
</div>
<p className="text-xs text-muted-foreground mt-1 line-clamp-1 pl-6">
{preset.description || 'No description'}
</p>
<div className="flex items-center gap-2 mt-1.5 pl-6">
<span className="text-[10px] text-muted-foreground">
{effectCount} effect{effectCount !== 1 ? 's' : ''}
</span>
<span className="text-[10px] text-muted-foreground/50">
{preset.effects_chain
.filter((e) => e.enabled)
.map((e) => e.type)
.join(' → ')}
</span>
</div>
</button>
);
}
@@ -0,0 +1,20 @@
import {EffectsDetail} from "./EffectsDetail";
import {EffectsList} from "./EffectsList";
export function EffectsTab() {
return (
<div className="flex flex-col h-full min-h-0 overflow-hidden">
<div className="flex-1 min-h-0 flex gap-6 overflow-hidden">
{/* Left - Presets list */}
<div className="w-full max-w-[360px] shrink-0 flex flex-col min-h-0">
<EffectsList />
</div>
{/* Right - Detail / editor */}
<div className="flex-1 min-h-0 flex flex-col">
<EffectsDetail />
</div>
</div>
</div>
);
}
@@ -0,0 +1,103 @@
import type { UseFormReturn } from 'react-hook-form';
import { FormControl } from '@/components/ui/form';
import {
Select,
SelectContent,
SelectItem,
SelectTrigger,
SelectValue,
} from '@/components/ui/select';
import { getLanguageOptionsForEngine } from '@/lib/constants/languages';
import type { GenerationFormValues } from '@/lib/hooks/useGenerationForm';
/**
* Engine/model options and their display metadata.
* Adding a new engine means adding one entry here.
*/
const ENGINE_OPTIONS = [
{ value: 'qwen:1.7B', label: 'Qwen3-TTS 1.7B' },
{ value: 'qwen:0.6B', label: 'Qwen3-TTS 0.6B' },
{ value: 'luxtts', label: 'LuxTTS' },
{ value: 'chatterbox', label: 'Chatterbox' },
{ value: 'chatterbox_turbo', label: 'Chatterbox Turbo' },
] as const;
const ENGINE_DESCRIPTIONS: Record<string, string> = {
qwen: 'Multi-language, two sizes',
luxtts: 'Fast, English-focused',
chatterbox: '23 languages, incl. Hebrew',
chatterbox_turbo: 'English, [laugh] [cough] tags',
};
/** Engines that only support English and should force language to 'en' on select. */
const ENGLISH_ONLY_ENGINES = new Set(['luxtts', 'chatterbox_turbo']);
function getSelectValue(engine: string, modelSize?: string): string {
if (engine === 'qwen') return `qwen:${modelSize || '1.7B'}`;
return engine;
}
function handleEngineChange(form: UseFormReturn<GenerationFormValues>, value: string) {
if (value.startsWith('qwen:')) {
const [, modelSize] = value.split(':');
form.setValue('engine', 'qwen');
form.setValue('modelSize', modelSize as '1.7B' | '0.6B');
// Validate language is supported by Qwen
const currentLang = form.getValues('language');
const available = getLanguageOptionsForEngine('qwen');
if (!available.some((l) => l.value === currentLang)) {
form.setValue('language', available[0]?.value ?? 'en');
}
} else {
form.setValue('engine', value as GenerationFormValues['engine']);
form.setValue('modelSize', undefined as unknown as '1.7B' | '0.6B');
if (ENGLISH_ONLY_ENGINES.has(value)) {
form.setValue('language', 'en');
} else {
// If current language isn't supported by the new engine, reset to first available
const currentLang = form.getValues('language');
const available = getLanguageOptionsForEngine(value);
if (!available.some((l) => l.value === currentLang)) {
form.setValue('language', available[0]?.value ?? 'en');
}
}
}
}
interface EngineModelSelectorProps {
form: UseFormReturn<GenerationFormValues>;
compact?: boolean;
}
export function EngineModelSelector({ form, compact }: EngineModelSelectorProps) {
const engine = form.watch('engine') || 'qwen';
const modelSize = form.watch('modelSize');
const selectValue = getSelectValue(engine, modelSize);
const itemClass = compact ? 'text-xs text-muted-foreground' : undefined;
const triggerClass = compact
? 'h-8 text-xs bg-card border-border rounded-full hover:bg-background/50 transition-all'
: undefined;
return (
<Select value={selectValue} onValueChange={(v) => handleEngineChange(form, v)}>
<FormControl>
<SelectTrigger className={triggerClass}>
<SelectValue />
</SelectTrigger>
</FormControl>
<SelectContent>
{ENGINE_OPTIONS.map((opt) => (
<SelectItem key={opt.value} value={opt.value} className={itemClass}>
{opt.label}
</SelectItem>
))}
</SelectContent>
</Select>
);
}
/** Returns a human-readable description for the currently selected engine. */
export function getEngineDescription(engine: string): string {
return ENGINE_DESCRIPTIONS[engine] ?? '';
}
@@ -1,6 +1,7 @@
import { useQuery } from '@tanstack/react-query';
import { useMatchRoute } from '@tanstack/react-router'; import { useMatchRoute } from '@tanstack/react-router';
import { AnimatePresence, motion } from 'framer-motion'; import { AnimatePresence, motion } from 'framer-motion';
import { Loader2, SlidersHorizontal, Sparkles } from 'lucide-react'; import { Loader2, Sparkles } from 'lucide-react';
import { useEffect, useRef, useState } from 'react'; import { useEffect, useRef, useState } from 'react';
import { Button } from '@/components/ui/button'; import { Button } from '@/components/ui/button';
import { Form, FormControl, FormField, FormItem, FormMessage } from '@/components/ui/form'; import { Form, FormControl, FormField, FormItem, FormMessage } from '@/components/ui/form';
@@ -12,6 +13,7 @@ import {
SelectValue, SelectValue,
} from '@/components/ui/select'; } from '@/components/ui/select';
import { Textarea } from '@/components/ui/textarea'; import { Textarea } from '@/components/ui/textarea';
import { apiClient } from '@/lib/api/client';
import { getLanguageOptionsForEngine, type LanguageCode } from '@/lib/constants/languages'; import { getLanguageOptionsForEngine, type LanguageCode } from '@/lib/constants/languages';
import { useGenerationForm } from '@/lib/hooks/useGenerationForm'; import { useGenerationForm } from '@/lib/hooks/useGenerationForm';
import { useProfile, useProfiles } from '@/lib/hooks/useProfiles'; import { useProfile, useProfiles } from '@/lib/hooks/useProfiles';
@@ -20,6 +22,7 @@ import { cn } from '@/lib/utils/cn';
import { useGenerationStore } from '@/stores/generationStore'; import { useGenerationStore } from '@/stores/generationStore';
import { useStoryStore } from '@/stores/storyStore'; import { useStoryStore } from '@/stores/storyStore';
import { useUIStore } from '@/stores/uiStore'; import { useUIStore } from '@/stores/uiStore';
import { EngineModelSelector } from './EngineModelSelector';
import { ParalinguisticInput } from './ParalinguisticInput'; import { ParalinguisticInput } from './ParalinguisticInput';
interface FloatingGenerateBoxProps { interface FloatingGenerateBoxProps {
@@ -36,7 +39,7 @@ export function FloatingGenerateBox({
const { data: selectedProfile } = useProfile(selectedProfileId || ''); const { data: selectedProfile } = useProfile(selectedProfileId || '');
const { data: profiles } = useProfiles(); const { data: profiles } = useProfiles();
const [isExpanded, setIsExpanded] = useState(false); const [isExpanded, setIsExpanded] = useState(false);
const [isInstructMode, setIsInstructMode] = useState(false); const [selectedPresetId, setSelectedPresetId] = useState<string | null>(null);
const containerRef = useRef<HTMLDivElement>(null); const containerRef = useRef<HTMLDivElement>(null);
const textareaRef = useRef<HTMLTextAreaElement | null>(null); const textareaRef = useRef<HTMLTextAreaElement | null>(null);
const matchRoute = useMatchRoute(); const matchRoute = useMatchRoute();
@@ -46,17 +49,28 @@ export function FloatingGenerateBox({
const { data: currentStory } = useStory(selectedStoryId); const { data: currentStory } = useStory(selectedStoryId);
const addPendingStoryAdd = useGenerationStore((s) => s.addPendingStoryAdd); const addPendingStoryAdd = useGenerationStore((s) => s.addPendingStoryAdd);
// Fetch effect presets for the dropdown
const { data: effectPresets } = useQuery({
queryKey: ['effectPresets'],
queryFn: () => apiClient.listEffectPresets(),
});
// Calculate if track editor is visible (on stories route with items) // Calculate if track editor is visible (on stories route with items)
const hasTrackEditor = isStoriesRoute && currentStory && currentStory.items.length > 0; const hasTrackEditor = isStoriesRoute && currentStory && currentStory.items.length > 0;
const { form, handleSubmit, isPending } = useGenerationForm({ const { form, handleSubmit, isPending } = useGenerationForm({
onSuccess: async (generationId) => { onSuccess: async (generationId) => {
setIsExpanded(false); setIsExpanded(false);
// Defer the story add until TTS completes — useGenerationProgress handles it // Defer the story add until TTS completes -- useGenerationProgress handles it
if (isStoriesRoute && selectedStoryId && generationId) { if (isStoriesRoute && selectedStoryId && generationId) {
addPendingStoryAdd(generationId, selectedStoryId); addPendingStoryAdd(generationId, selectedStoryId);
} }
}, },
getEffectsChain: () => {
if (!selectedPresetId || !effectPresets) return undefined;
const preset = effectPresets.find((p) => p.id === selectedPresetId);
return preset?.effects_chain;
},
}); });
// Click away handler to collapse the box // Click away handler to collapse the box
@@ -184,111 +198,57 @@ export function FloatingGenerateBox({
<Form {...form}> <Form {...form}>
<form onSubmit={form.handleSubmit(onSubmit)}> <form onSubmit={form.handleSubmit(onSubmit)}>
<div className="flex gap-2"> <div className="flex gap-2">
<motion.div <motion.div className="flex-1" transition={{ duration: 0.3, ease: 'easeOut' }}>
className={cn('flex-1', isExpanded && 'mr-12')} <FormField
transition={{ duration: 0.3, ease: 'easeOut' }} control={form.control}
> name="text"
{/* Text field - hidden when in instruct mode */} render={({ field }) => (
<div style={{ display: isInstructMode ? 'none' : 'block' }}> <FormItem>
<FormField <FormControl>
control={form.control} <motion.div
name="text" animate={{
render={({ field }) => ( height: isExpanded ? 'auto' : '32px',
<FormItem> }}
<FormControl> transition={{ duration: 0.15, ease: 'easeOut' }}
<motion.div style={{ overflow: 'hidden' }}
animate={{ >
height: isExpanded ? 'auto' : '32px', {form.watch('engine') === 'chatterbox_turbo' ? (
}} <ParalinguisticInput
transition={{ duration: 0.15, ease: 'easeOut' }} value={field.value}
style={{ overflow: 'hidden' }} onChange={field.onChange}
> placeholder={
{form.watch('engine') === 'chatterbox_turbo' ? ( isStoriesRoute && currentStory
<ParalinguisticInput ? `Generate speech for "${currentStory.name}"... (type / for effects)`
value={field.value} : selectedProfile
onChange={field.onChange} ? `Type / for effects like [laugh], [sigh]...`
placeholder={ : 'Select a voice profile above...'
isStoriesRoute && currentStory }
? `Generate speech for "${currentStory.name}"... (type / for effects)` className="px-3 py-2 resize-none bg-transparent border-none focus-visible:ring-0 focus-visible:ring-offset-0 focus:outline-none focus:ring-0 outline-none ring-0 rounded-2xl text-sm w-full"
: selectedProfile style={{
? `Type / for effects like [laugh], [sigh]...` minHeight: isExpanded ? '100px' : '32px',
: 'Select a voice profile above...' maxHeight: '300px',
} overflowY: 'auto',
className="px-3 py-2 resize-none bg-transparent border-none focus-visible:ring-0 focus-visible:ring-offset-0 focus:outline-none focus:ring-0 outline-none ring-0 rounded-2xl text-sm w-full" }}
style={{ disabled={!selectedProfileId}
minHeight: isExpanded ? '100px' : '32px', onClick={() => setIsExpanded(true)}
maxHeight: '300px', onFocus={() => setIsExpanded(true)}
overflowY: 'auto', />
}} ) : (
disabled={!selectedProfileId}
onClick={() => setIsExpanded(true)}
onFocus={() => setIsExpanded(true)}
/>
) : (
<Textarea
{...field}
ref={(node: HTMLTextAreaElement | null) => {
// Store ref for auto-resize (only for active field)
if (!isInstructMode) {
textareaRef.current = node;
}
// Forward ref to react-hook-form
if (typeof field.ref === 'function') {
field.ref(node);
}
}}
placeholder={
isStoriesRoute && currentStory
? `Generate speech for "${currentStory.name}"...`
: selectedProfile
? `Generate speech using ${selectedProfile.name}...`
: 'Select a voice profile above...'
}
className="resize-none bg-transparent border-none focus-visible:ring-0 focus-visible:ring-offset-0 focus:outline-none focus:ring-0 outline-none ring-0 rounded-2xl text-sm placeholder:text-muted-foreground/60 w-full"
style={{
minHeight: isExpanded ? '100px' : '32px',
maxHeight: '300px',
}}
disabled={!selectedProfileId}
onClick={() => setIsExpanded(true)}
onFocus={() => setIsExpanded(true)}
/>
)}
</motion.div>
</FormControl>
<FormMessage className="text-xs" />
</FormItem>
)}
/>
</div>
{/* Instruct field - hidden when in text mode */}
<div style={{ display: isInstructMode ? 'block' : 'none' }}>
<FormField
control={form.control}
name="instruct"
render={({ field }) => (
<FormItem>
<FormControl>
<motion.div
animate={{
height: isExpanded ? 'auto' : '32px',
}}
transition={{ duration: 0.15, ease: 'easeOut' }}
style={{ overflow: 'hidden' }}
>
<Textarea <Textarea
{...field} {...field}
ref={(node: HTMLTextAreaElement | null) => { ref={(node: HTMLTextAreaElement | null) => {
// Store ref for auto-resize (only for active field) textareaRef.current = node;
if (isInstructMode) {
textareaRef.current = node;
}
// Forward ref to react-hook-form
if (typeof field.ref === 'function') { if (typeof field.ref === 'function') {
field.ref(node); field.ref(node);
} }
}} }}
placeholder="e.g. very happy and excited" placeholder={
isStoriesRoute && currentStory
? `Generate speech for "${currentStory.name}"...`
: selectedProfile
? `Generate speech using ${selectedProfile.name}...`
: 'Select a voice profile above...'
}
className="resize-none bg-transparent border-none focus-visible:ring-0 focus-visible:ring-offset-0 focus:outline-none focus:ring-0 outline-none ring-0 rounded-2xl text-sm placeholder:text-muted-foreground/60 w-full" className="resize-none bg-transparent border-none focus-visible:ring-0 focus-visible:ring-offset-0 focus:outline-none focus:ring-0 outline-none ring-0 rounded-2xl text-sm placeholder:text-muted-foreground/60 w-full"
style={{ style={{
minHeight: isExpanded ? '100px' : '32px', minHeight: isExpanded ? '100px' : '32px',
@@ -298,13 +258,13 @@ export function FloatingGenerateBox({
onClick={() => setIsExpanded(true)} onClick={() => setIsExpanded(true)}
onFocus={() => setIsExpanded(true)} onFocus={() => setIsExpanded(true)}
/> />
</motion.div> )}
</FormControl> </motion.div>
<FormMessage className="text-xs" /> </FormControl>
</FormItem> <FormMessage className="text-xs" />
)} </FormItem>
/> )}
</div> />
</motion.div> </motion.div>
<div className="relative shrink-0"> <div className="relative shrink-0">
@@ -336,40 +296,6 @@ export function FloatingGenerateBox({
: 'Generate speech'} : 'Generate speech'}
</span> </span>
</div> </div>
<AnimatePresence>
{isExpanded && form.watch('engine') === 'qwen' && (
<motion.div
initial={{ opacity: 0, scale: 0.8 }}
animate={{ opacity: 1, scale: 1 }}
exit={{ opacity: 0, scale: 0.8 }}
transition={{ duration: 0.2 }}
className="absolute top-0 right-[calc(100%+0.5rem)]"
>
<div className="group relative">
<Button
type="button"
variant="ghost"
size="icon"
onClick={() => setIsInstructMode(!isInstructMode)}
className={cn(
'h-10 w-10 rounded-full transition-all duration-200',
isInstructMode
? 'bg-accent text-accent-foreground border border-accent hover:bg-accent/90'
: 'bg-card border border-border hover:bg-background/50',
)}
aria-label={
isInstructMode ? 'Fine tune instructions, on' : 'Fine tune instructions'
}
>
<SlidersHorizontal className="h-4 w-4" />
</Button>
<span className="pointer-events-none absolute bottom-full left-1/2 -translate-x-1/2 mb-2 whitespace-nowrap rounded-md bg-popover px-3 py-1.5 text-xs text-popover-foreground border border-border opacity-0 transition-opacity group-hover:opacity-100 z-[9999]">
Fine tune instructions
</span>
</div>
</motion.div>
)}
</AnimatePresence>
</div> </div>
</div> </div>
@@ -431,57 +357,29 @@ export function FloatingGenerateBox({
}} }}
/> />
<FormItem className="flex-1 space-y-0">
<EngineModelSelector form={form} compact />
</FormItem>
<FormItem className="flex-1 space-y-0"> <FormItem className="flex-1 space-y-0">
<Select <Select
value={ value={selectedPresetId || 'none'}
form.watch('engine') === 'luxtts' onValueChange={(value) =>
? 'luxtts' setSelectedPresetId(value === 'none' ? null : value)
: form.watch('engine') === 'chatterbox'
? 'chatterbox'
: form.watch('engine') === 'chatterbox_turbo'
? 'chatterbox_turbo'
: `qwen:${form.watch('modelSize') || '1.7B'}`
} }
onValueChange={(value) => {
if (value === 'luxtts') {
form.setValue('engine', 'luxtts');
form.setValue('language', 'en');
} else if (value === 'chatterbox') {
form.setValue('engine', 'chatterbox');
} else if (value === 'chatterbox_turbo') {
form.setValue('engine', 'chatterbox_turbo');
form.setValue('language', 'en');
} else {
const [, modelSize] = value.split(':');
form.setValue('engine', 'qwen');
form.setValue('modelSize', modelSize as '1.7B' | '0.6B');
}
}}
> >
<FormControl> <SelectTrigger className="h-8 text-xs bg-card border-border rounded-full hover:bg-background/50 transition-all">
<SelectTrigger className="h-8 text-xs bg-card border-border rounded-full hover:bg-background/50 transition-all"> <SelectValue placeholder="No effects" />
<SelectValue /> </SelectTrigger>
</SelectTrigger>
</FormControl>
<SelectContent> <SelectContent>
<SelectItem value="qwen:1.7B" className="text-xs text-muted-foreground"> <SelectItem value="none" className="text-xs">
Qwen3-TTS 1.7B No effects
</SelectItem>
<SelectItem value="qwen:0.6B" className="text-xs text-muted-foreground">
Qwen3-TTS 0.6B
</SelectItem>
<SelectItem value="luxtts" className="text-xs text-muted-foreground">
LuxTTS
</SelectItem>
<SelectItem value="chatterbox" className="text-xs text-muted-foreground">
Chatterbox
</SelectItem>
<SelectItem
value="chatterbox_turbo"
className="text-xs text-muted-foreground"
>
Chatterbox Turbo
</SelectItem> </SelectItem>
{effectPresets?.map((preset) => (
<SelectItem key={preset.id} value={preset.id} className="text-xs">
{preset.name}
</SelectItem>
))}
</SelectContent> </SelectContent>
</Select> </Select>
</FormItem> </FormItem>
@@ -23,6 +23,7 @@ import { getLanguageOptionsForEngine } from '@/lib/constants/languages';
import { useGenerationForm } from '@/lib/hooks/useGenerationForm'; import { useGenerationForm } from '@/lib/hooks/useGenerationForm';
import { useProfile } from '@/lib/hooks/useProfiles'; import { useProfile } from '@/lib/hooks/useProfiles';
import { useUIStore } from '@/stores/uiStore'; import { useUIStore } from '@/stores/uiStore';
import { EngineModelSelector, getEngineDescription } from './EngineModelSelector';
import { ParalinguisticInput } from './ParalinguisticInput'; import { ParalinguisticInput } from './ParalinguisticInput';
export function GenerationForm() { export function GenerationForm() {
@@ -117,53 +118,9 @@ export function GenerationForm() {
<div className="grid gap-4 md:grid-cols-3"> <div className="grid gap-4 md:grid-cols-3">
<FormItem> <FormItem>
<FormLabel>Model</FormLabel> <FormLabel>Model</FormLabel>
<Select <EngineModelSelector form={form} />
value={
form.watch('engine') === 'luxtts'
? 'luxtts'
: form.watch('engine') === 'chatterbox'
? 'chatterbox'
: form.watch('engine') === 'chatterbox_turbo'
? 'chatterbox_turbo'
: `qwen:${form.watch('modelSize') || '1.7B'}`
}
onValueChange={(value) => {
if (value === 'luxtts') {
form.setValue('engine', 'luxtts');
form.setValue('language', 'en');
} else if (value === 'chatterbox') {
form.setValue('engine', 'chatterbox');
} else if (value === 'chatterbox_turbo') {
form.setValue('engine', 'chatterbox_turbo');
form.setValue('language', 'en');
} else {
const [, modelSize] = value.split(':');
form.setValue('engine', 'qwen');
form.setValue('modelSize', modelSize as '1.7B' | '0.6B');
}
}}
>
<FormControl>
<SelectTrigger>
<SelectValue />
</SelectTrigger>
</FormControl>
<SelectContent>
<SelectItem value="qwen:1.7B">Qwen3-TTS 1.7B</SelectItem>
<SelectItem value="qwen:0.6B">Qwen3-TTS 0.6B</SelectItem>
<SelectItem value="luxtts">LuxTTS</SelectItem>
<SelectItem value="chatterbox">Chatterbox</SelectItem>
<SelectItem value="chatterbox_turbo">Chatterbox Turbo</SelectItem>
</SelectContent>
</Select>
<FormDescription> <FormDescription>
{form.watch('engine') === 'luxtts' {getEngineDescription(form.watch('engine') || 'qwen')}
? 'Fast, English-focused'
: form.watch('engine') === 'chatterbox'
? '23 languages, incl. Hebrew'
: form.watch('engine') === 'chatterbox_turbo'
? 'English, [laugh] [cough] tags'
: 'Multi-language, two sizes'}
</FormDescription> </FormDescription>
</FormItem> </FormItem>
+419 -130
View File
@@ -1,15 +1,22 @@
import { useQueryClient } from '@tanstack/react-query'; import { useQueryClient } from '@tanstack/react-query';
import { AnimatePresence, motion } from 'framer-motion';
import { import {
AlignCenter,
AudioLines,
AudioWaveform,
Download, Download,
FileArchive, FileArchive,
Loader2, Loader2,
MoreHorizontal, MoreHorizontal,
Play, Play,
RotateCcw, RotateCcw,
Star,
Trash2, Trash2,
Wand2,
} from 'lucide-react'; } from 'lucide-react';
import { useEffect, useRef, useState } from 'react'; import { useEffect, useRef, useState } from 'react';
import Loader from 'react-loaders'; import Loader from 'react-loaders';
import { EffectsChainEditor } from '@/components/Effects/EffectsChainEditor';
import { Button } from '@/components/ui/button'; import { Button } from '@/components/ui/button';
import { import {
Dialog, Dialog,
@@ -25,10 +32,17 @@ import {
DropdownMenuItem, DropdownMenuItem,
DropdownMenuTrigger, DropdownMenuTrigger,
} from '@/components/ui/dropdown-menu'; } from '@/components/ui/dropdown-menu';
import {
Select,
SelectContent,
SelectItem,
SelectTrigger,
SelectValue,
} from '@/components/ui/select';
import { Textarea } from '@/components/ui/textarea'; import { Textarea } from '@/components/ui/textarea';
import { useToast } from '@/components/ui/use-toast'; import { useToast } from '@/components/ui/use-toast';
import { apiClient } from '@/lib/api/client'; import { apiClient } from '@/lib/api/client';
import type { HistoryResponse } from '@/lib/api/types'; import type { EffectConfig, GenerationVersionResponse, HistoryResponse } from '@/lib/api/types';
import { BOTTOM_SAFE_AREA_PADDING } from '@/lib/constants/ui'; import { BOTTOM_SAFE_AREA_PADDING } from '@/lib/constants/ui';
import { import {
useDeleteGeneration, useDeleteGeneration,
@@ -60,6 +74,15 @@ export function HistoryTable() {
const [generationToDelete, setGenerationToDelete] = useState<{ id: string; name: string } | null>( const [generationToDelete, setGenerationToDelete] = useState<{ id: string; name: string } | null>(
null, null,
); );
const [effectsDialogOpen, setEffectsDialogOpen] = useState(false);
const [effectsTargetId, setEffectsTargetId] = useState<string | null>(null);
const [effectsTargetVersions, setEffectsTargetVersions] = useState<GenerationVersionResponse[]>(
[],
);
const [effectsSourceVersionId, setEffectsSourceVersionId] = useState<string | null>(null);
const [effectsChain, setEffectsChain] = useState<EffectConfig[]>([]);
const [applyingEffects, setApplyingEffects] = useState(false);
const [expandedVersionsId, setExpandedVersionsId] = useState<string | null>(null);
const limit = 20; const limit = 20;
const { toast } = useToast(); const { toast } = useToast();
const queryClient = useQueryClient(); const queryClient = useQueryClient();
@@ -215,6 +238,106 @@ export function HistoryTable() {
} }
}; };
const handleRegenerate = async (generationId: string) => {
try {
await apiClient.regenerateGeneration(generationId);
addPendingGeneration(generationId);
queryClient.invalidateQueries({ queryKey: ['history'] });
} catch (error) {
toast({
title: 'Regenerate failed',
description: error instanceof Error ? error.message : 'Could not regenerate',
variant: 'destructive',
});
}
};
const handleToggleFavorite = async (generationId: string) => {
try {
await apiClient.toggleFavorite(generationId);
queryClient.invalidateQueries({ queryKey: ['history'] });
} catch (error) {
toast({
title: 'Failed to update favorite',
description: error instanceof Error ? error.message : 'Unknown error',
variant: 'destructive',
});
}
};
const handleApplyEffects = (generationId: string) => {
const gen = allHistory.find((g) => g.id === generationId);
const versions = gen?.versions ?? [];
setEffectsTargetId(generationId);
setEffectsTargetVersions(versions);
// Default to clean/original version (no effects chain)
const cleanVersion = versions.find((v) => !v.effects_chain || v.effects_chain.length === 0);
setEffectsSourceVersionId(cleanVersion?.id ?? null);
setEffectsChain([]);
setEffectsDialogOpen(true);
};
const handleApplyEffectsConfirm = async () => {
if (!effectsTargetId || effectsChain.length === 0) return;
setApplyingEffects(true);
try {
const newVersion = await apiClient.applyEffectsToGeneration(effectsTargetId, {
effects_chain: effectsChain,
source_version_id: effectsSourceVersionId ?? undefined,
set_as_default: true,
});
queryClient.invalidateQueries({ queryKey: ['history'] });
// If the player is currently on this generation, reload with the new version audio
if (currentAudioId === effectsTargetId) {
const gen = allHistory.find((g) => g.id === effectsTargetId);
if (gen) {
const versionUrl = apiClient.getVersionAudioUrl(newVersion.id);
setAudioWithAutoPlay(
versionUrl,
effectsTargetId,
gen.profile_id,
gen.text.substring(0, 50),
);
}
}
setEffectsDialogOpen(false);
toast({ title: 'Effects applied', description: 'A new version has been created.' });
} catch (error) {
toast({
title: 'Failed to apply effects',
description: error instanceof Error ? error.message : 'Unknown error',
variant: 'destructive',
});
} finally {
setApplyingEffects(false);
}
};
const handleSwitchVersion = async (generationId: string, versionId: string) => {
try {
await apiClient.setDefaultVersion(generationId, versionId);
queryClient.invalidateQueries({ queryKey: ['history'] });
} catch (error) {
toast({
title: 'Failed to switch version',
description: error instanceof Error ? error.message : 'Unknown error',
variant: 'destructive',
});
}
};
const handlePlayVersion = (
generationId: string,
versionId: string,
text: string,
profileId: string,
) => {
const audioUrl = apiClient.getVersionAudioUrl(versionId);
setAudioWithAutoPlay(audioUrl, generationId, profileId, text.substring(0, 50));
};
const handleImportConfirm = () => { const handleImportConfirm = () => {
if (selectedFile) { if (selectedFile) {
importGeneration.mutate(selectedFile, { importGeneration.mutate(selectedFile, {
@@ -271,154 +394,269 @@ export function HistoryTable() {
> >
{history.map((gen) => { {history.map((gen) => {
const isCurrentlyPlaying = currentAudioId === gen.id && isPlaying; const isCurrentlyPlaying = currentAudioId === gen.id && isPlaying;
const isGenerating = gen.status === 'generating'; const isInProgress = gen.status === 'loading_model' || gen.status === 'generating';
const isGenerating = isInProgress;
const isFailed = gen.status === 'failed'; const isFailed = gen.status === 'failed';
const isPlayable = !isGenerating && !isFailed; const isPlayable = !isGenerating && !isFailed;
const hasVersions = gen.versions && gen.versions.length > 1;
const isVersionsExpanded = expandedVersionsId === gen.id;
return ( return (
<div <div
key={gen.id} key={gen.id}
role={isPlayable ? 'button' : undefined}
tabIndex={isPlayable ? 0 : undefined}
className={cn( className={cn(
'flex items-stretch gap-4 h-26 border rounded-md p-3 bg-card transition-colors text-left w-full', 'border rounded-md bg-card transition-colors text-left w-full',
isPlayable && 'hover:bg-muted/70 cursor-pointer',
isCurrentlyPlaying && 'bg-muted/70', isCurrentlyPlaying && 'bg-muted/70',
)} )}
aria-label={
isGenerating
? `Generating speech for ${gen.profile_name}...`
: isFailed
? `Generation failed for ${gen.profile_name}`
: isCurrentlyPlaying
? `Sample from ${gen.profile_name}, ${formatDuration(gen.duration ?? 0)}, ${formatDate(gen.created_at)}. Playing. Press Enter to restart.`
: `Sample from ${gen.profile_name}, ${formatDuration(gen.duration ?? 0)}, ${formatDate(gen.created_at)}. Press Enter to play.`
}
onMouseDown={(e) => {
if (!isPlayable) return;
const target = e.target as HTMLElement;
if (target.closest('textarea') || window.getSelection()?.toString()) {
return;
}
handlePlay(gen.id, gen.text, gen.profile_id);
}}
onKeyDown={(e) => {
if (!isPlayable) return;
const target = e.target as HTMLElement;
if (target.closest('textarea') || target.closest('button')) return;
if (e.key === 'Enter' || e.key === ' ') {
e.preventDefault();
handlePlay(gen.id, gen.text, gen.profile_id);
}
}}
> >
{/* Status icon */} {/* Main row */}
<div className="flex items-center shrink-0 w-10 justify-center overflow-hidden"> <div
<div className="scale-50"> role={isPlayable ? 'button' : undefined}
<Loader tabIndex={isPlayable ? 0 : undefined}
type={isGenerating ? 'line-scale' : 'line-scale-pulse-out-rapid'} className={cn(
active={isGenerating || isCurrentlyPlaying} 'flex items-stretch gap-4 h-26 p-3',
isPlayable && 'hover:bg-muted/70 cursor-pointer rounded-md',
isVersionsExpanded && 'rounded-b-none',
)}
aria-label={
isGenerating
? `Generating speech for ${gen.profile_name}...`
: isFailed
? `Generation failed for ${gen.profile_name}`
: isCurrentlyPlaying
? `Sample from ${gen.profile_name}, ${formatDuration(gen.duration ?? 0)}, ${formatDate(gen.created_at)}. Playing. Press Enter to restart.`
: `Sample from ${gen.profile_name}, ${formatDuration(gen.duration ?? 0)}, ${formatDate(gen.created_at)}. Press Enter to play.`
}
onMouseDown={(e) => {
if (!isPlayable) return;
const target = e.target as HTMLElement;
if (target.closest('textarea') || window.getSelection()?.toString()) {
return;
}
handlePlay(gen.id, gen.text, gen.profile_id);
}}
onKeyDown={(e) => {
if (!isPlayable) return;
const target = e.target as HTMLElement;
if (target.closest('textarea') || target.closest('button')) return;
if (e.key === 'Enter' || e.key === ' ') {
e.preventDefault();
handlePlay(gen.id, gen.text, gen.profile_id);
}
}}
>
{/* Status icon */}
<div className="flex items-center shrink-0 w-10 justify-center overflow-hidden">
<div className="scale-50">
<Loader
type={isGenerating ? 'line-scale' : 'line-scale-pulse-out-rapid'}
active={isGenerating || isCurrentlyPlaying}
/>
</div>
</div>
{/* Left side - Meta information */}
<div className="flex flex-col gap-1.5 w-48 shrink-0 justify-center">
<div className="font-medium text-sm truncate" title={gen.profile_name}>
{gen.profile_name}
</div>
<div className="flex items-center gap-2">
<span className="text-xs text-muted-foreground">{gen.language}</span>
<span className="text-xs text-muted-foreground">
{formatEngineName(gen.engine, gen.model_size)}
</span>
{isFailed ? (
<span className="text-xs text-destructive">Failed</span>
) : !isGenerating ? (
<span className="text-xs text-muted-foreground">
{formatDuration(gen.duration ?? 0)}
</span>
) : null}
</div>
<div className="text-xs text-muted-foreground">
{isInProgress ? (
<span className="text-accent">
{gen.status === 'loading_model' ? 'Loading model...' : 'Generating...'}
</span>
) : (
formatDate(gen.created_at)
)}
</div>
</div>
{/* Right side - Transcript textarea */}
<div className="flex-1 min-w-0 flex">
<Textarea
value={gen.text}
className="flex-1 resize-none text-sm text-muted-foreground select-text"
readOnly
aria-label={`Transcript for sample from ${gen.profile_name}, ${formatDuration(gen.duration ?? 0)}`}
/> />
</div> </div>
</div>
{/* Left side - Meta information */} {/* Far right - Actions */}
<div className="flex flex-col gap-1.5 w-48 shrink-0 justify-center"> <div
<div className="font-medium text-sm truncate" title={gen.profile_name}> className="shrink-0 flex flex-col justify-center items-center gap-0.5"
{gen.profile_name} onMouseDown={(e) => e.stopPropagation()}
</div> onClick={(e) => e.stopPropagation()}
<div className="flex items-center gap-2"> >
<span className="text-xs text-muted-foreground">{gen.language}</span> <Button
<span className="text-xs text-muted-foreground"> variant="ghost"
{formatEngineName(gen.engine, gen.model_size)} size="icon"
</span> className={cn(
'h-6 w-6 text-muted-foreground/50 hover:bg-muted-foreground/20 hover:text-muted-foreground',
gen.is_favorited && 'text-accent hover:text-accent',
)}
aria-label={gen.is_favorited ? 'Unfavorite' : 'Favorite'}
onClick={() => handleToggleFavorite(gen.id)}
>
<Star
className="h-2 w-2"
fill={gen.is_favorited ? 'currentColor' : 'none'}
/>
</Button>
{hasVersions && (
<Button
variant="ghost"
size="icon"
className={cn(
'h-6 w-6 text-muted-foreground/50 hover:bg-muted-foreground/20 hover:text-muted-foreground',
isVersionsExpanded && 'text-accent hover:text-accent',
)}
aria-label="Toggle versions"
onClick={() => setExpandedVersionsId(isVersionsExpanded ? null : gen.id)}
>
<AudioLines className="h-2 w-2" />
</Button>
)}
{isFailed ? ( {isFailed ? (
<span className="text-xs text-destructive">Failed</span> <Button
) : !isGenerating ? ( variant="ghost"
<span className="text-xs text-muted-foreground"> size="icon"
{formatDuration(gen.duration ?? 0)} className="h-6 w-6 text-muted-foreground/50 hover:bg-muted-foreground/20 hover:text-muted-foreground"
</span> aria-label="Retry generation"
) : null} onClick={() => handleRetry(gen.id)}
</div> >
<div className="text-xs text-muted-foreground"> <RotateCcw className="h-2 w-2" />
{isGenerating ? ( </Button>
<span className="text-accent">Generating...</span>
) : ( ) : (
formatDate(gen.created_at) <>
<DropdownMenu>
<DropdownMenuTrigger asChild>
<Button
variant="ghost"
size="icon"
className="h-6 w-6 text-muted-foreground/50 hover:bg-muted-foreground/20 hover:text-muted-foreground"
aria-label="Actions"
disabled={isGenerating}
>
<MoreHorizontal className="h-2 w-2" />
</Button>
</DropdownMenuTrigger>
<DropdownMenuContent align="end">
<DropdownMenuItem
onClick={() => handlePlay(gen.id, gen.text, gen.profile_id)}
>
<Play className="mr-2 h-4 w-4" />
Play
</DropdownMenuItem>
<DropdownMenuItem
onClick={() => handleDownloadAudio(gen.id, gen.text)}
disabled={exportGenerationAudio.isPending}
>
<Download className="mr-2 h-4 w-4" />
Export Audio
</DropdownMenuItem>
<DropdownMenuItem
onClick={() => handleExportPackage(gen.id, gen.text)}
disabled={exportGeneration.isPending}
>
<FileArchive className="mr-2 h-4 w-4" />
Export Package
</DropdownMenuItem>
<DropdownMenuItem onClick={() => handleApplyEffects(gen.id)}>
<Wand2 className="mr-2 h-4 w-4" />
Apply Effects
</DropdownMenuItem>
<DropdownMenuItem onClick={() => handleRegenerate(gen.id)}>
<RotateCcw className="mr-2 h-4 w-4" />
Regenerate
</DropdownMenuItem>
<DropdownMenuItem
onClick={() => handleDeleteClick(gen.id, gen.profile_name)}
disabled={deleteGeneration.isPending}
// className="text-destructive focus:text-destructive"
>
<Trash2 className="mr-2 h-4 w-4" />
Delete
</DropdownMenuItem>
</DropdownMenuContent>
</DropdownMenu>
</>
)} )}
</div> </div>
</div> </div>
{/* Right side - Transcript textarea */} {/* Expandable versions panel */}
<div className="flex-1 min-w-0 flex"> <AnimatePresence>
<Textarea {isVersionsExpanded && gen.versions && (
value={gen.text} <motion.div
className="flex-1 resize-none text-sm text-muted-foreground select-text" initial={{ height: 0, opacity: 0 }}
readOnly animate={{ height: 'auto', opacity: 1 }}
aria-label={`Transcript for sample from ${gen.profile_name}, ${formatDuration(gen.duration ?? 0)}`} exit={{ height: 0, opacity: 0 }}
/> transition={{ duration: 0.2, ease: 'easeOut' }}
</div> className="overflow-hidden"
{/* Far right - Actions */}
<div
className="w-10 shrink-0 flex justify-end items-center"
onMouseDown={(e) => e.stopPropagation()}
onClick={(e) => e.stopPropagation()}
>
{isFailed ? (
<Button
variant="ghost"
size="icon"
className="h-8 w-8"
aria-label="Retry generation"
onClick={() => handleRetry(gen.id)}
> >
<RotateCcw className="h-4 w-4" /> <div className="border-t border-border/50">
</Button> <div className="divide-y divide-border/40">
) : isPlayable ? ( {gen.versions.map((v) => {
<DropdownMenu> // Show source provenance when effects were applied to a non-clean version
<DropdownMenuTrigger asChild> const sourceVersion = v.source_version_id
<Button ? gen.versions?.find((sv) => sv.id === v.source_version_id)
variant="ghost" : null;
size="icon" const showSource =
className="h-8 w-8" sourceVersion &&
aria-label="Actions" sourceVersion.effects_chain &&
> sourceVersion.effects_chain.length > 0;
<MoreHorizontal className="h-4 w-4" />
</Button> return (
</DropdownMenuTrigger> <button
<DropdownMenuContent align="end"> key={v.id}
<DropdownMenuItem type="button"
onClick={() => handlePlay(gen.id, gen.text, gen.profile_id)} className="flex items-center gap-2 w-full h-9 px-3 text-left hover:bg-muted/50 transition-colors"
> onClick={() => {
<Play className="mr-2 h-4 w-4" /> handlePlayVersion(gen.id, v.id, gen.text, gen.profile_id);
Play if (!v.is_default) {
</DropdownMenuItem> handleSwitchVersion(gen.id, v.id);
<DropdownMenuItem }
onClick={() => handleDownloadAudio(gen.id, gen.text)} }}
disabled={exportGenerationAudio.isPending} >
> <AudioLines className="h-3 w-3 shrink-0 text-muted-foreground" />
<Download className="mr-2 h-4 w-4" /> <span className="truncate text-xs font-medium">{v.label}</span>
Export Audio {v.effects_chain && v.effects_chain.length > 0 && (
</DropdownMenuItem> <span className="text-[10px] text-muted-foreground truncate">
<DropdownMenuItem {v.effects_chain.map((e) => e.type).join(' → ')}
onClick={() => handleExportPackage(gen.id, gen.text)} </span>
disabled={exportGeneration.isPending} )}
> {showSource && (
<FileArchive className="mr-2 h-4 w-4" /> <span className="text-[10px] text-muted-foreground/60 truncate">
Export Package from {sourceVersion.label}
</DropdownMenuItem> </span>
<DropdownMenuItem )}
onClick={() => handleDeleteClick(gen.id, gen.profile_name)} <span className="flex-1" />
disabled={deleteGeneration.isPending} {v.is_default && (
className="text-destructive focus:text-destructive" <span className="text-[10px] bg-accent/15 text-accent px-1.5 py-0.5 rounded-full">
> active
<Trash2 className="mr-2 h-4 w-4" /> </span>
Delete )}
</DropdownMenuItem> </button>
</DropdownMenuContent> );
</DropdownMenu> })}
) : null} </div>
</div> </div>
</motion.div>
)}
</AnimatePresence>
</div> </div>
); );
})} })}
@@ -500,6 +738,57 @@ export function HistoryTable() {
</DialogFooter> </DialogFooter>
</DialogContent> </DialogContent>
</Dialog> </Dialog>
<Dialog open={effectsDialogOpen} onOpenChange={setEffectsDialogOpen}>
<DialogContent className="max-w-md">
<DialogHeader>
<DialogTitle>Apply Effects</DialogTitle>
<DialogDescription>
Configure post-processing effects to apply to this generation. A new version will be
created.
</DialogDescription>
</DialogHeader>
{effectsTargetVersions.length > 1 && (
<div className="space-y-1.5">
<label className="text-xs font-medium text-muted-foreground">Source</label>
<Select
value={effectsSourceVersionId ?? ''}
onValueChange={(val) => setEffectsSourceVersionId(val || null)}
>
<SelectTrigger className="h-8 text-xs">
<SelectValue placeholder="Select source version" />
</SelectTrigger>
<SelectContent>
{effectsTargetVersions.map((v) => (
<SelectItem key={v.id} value={v.id} className="text-xs">
{v.label}
{v.effects_chain && v.effects_chain.length > 0 && (
<span className="text-muted-foreground ml-1.5">
({v.effects_chain.map((e) => e.type).join(' + ')})
</span>
)}
</SelectItem>
))}
</SelectContent>
</Select>
</div>
)}
<div className="py-2 max-h-80 overflow-y-auto">
<EffectsChainEditor value={effectsChain} onChange={setEffectsChain} />
</div>
<DialogFooter>
<Button variant="outline" onClick={() => setEffectsDialogOpen(false)}>
Cancel
</Button>
<Button
onClick={handleApplyEffectsConfirm}
disabled={applyingEffects || effectsChain.length === 0}
>
{applyingEffects ? 'Applying...' : 'Apply'}
</Button>
</DialogFooter>
</DialogContent>
</Dialog>
</div> </div>
); );
} }
@@ -113,7 +113,7 @@ export function ConnectionForm() {
<Badge variant={health.gpu_available ? 'default' : 'secondary'}> <Badge variant={health.gpu_available ? 'default' : 'secondary'}>
GPU: {health.gpu_available ? 'Available' : 'Not Available'} GPU: {health.gpu_available ? 'Available' : 'Not Available'}
</Badge> </Badge>
{health.vram_used_mb && ( {health.vram_used_mb != null && health.vram_used_mb > 0 && (
<Badge variant="outline">VRAM: {health.vram_used_mb.toFixed(0)} MB</Badge> <Badge variant="outline">VRAM: {health.vram_used_mb.toFixed(0)} MB</Badge>
)} )}
</div> </div>
@@ -26,7 +26,7 @@ export function GpuAcceleration() {
// Query CUDA backend status // Query CUDA backend status
const { const {
data: cudaStatus, data: cudaStatus,
isLoading: cudaStatusLoading, isLoading: _cudaStatusLoading,
refetch: refetchCudaStatus, refetch: refetchCudaStatus,
} = useQuery({ } = useQuery({
queryKey: ['cuda-status', serverUrl], queryKey: ['cuda-status', serverUrl],
@@ -218,43 +218,46 @@ export function GpuAcceleration() {
<CardTitle>GPU Acceleration</CardTitle> <CardTitle>GPU Acceleration</CardTitle>
</CardHeader> </CardHeader>
<CardContent className="space-y-4"> <CardContent className="space-y-4">
{/* Current status */} {/* GPU status */}
<div className="space-y-1"> <div className="space-y-1">
<div className="text-sm font-medium">Backend</div> {health.gpu_available && health.gpu_type ? (
<div className="text-sm text-muted-foreground"> <>
{isCurrentlyCuda <div className="text-sm font-medium">
? 'CUDA (GPU accelerated)' {health.gpu_type.replace(/^(CUDA|ROCm|MPS|Metal|XPU|DirectML)\s*\((.+)\)$/, '$2') ||
: hasNativeGpu health.gpu_type}
? `${health.backend_type === 'mlx' ? 'MLX' : 'PyTorch'} (GPU accelerated)`
: 'CPU'}
</div>
</div>
{/* GPU info from health */}
{health.gpu_type && (
<div className="space-y-1">
<div className="text-sm font-medium">GPU</div>
<div className="text-sm text-muted-foreground">{health.gpu_type}</div>
{health.vram_used_mb != null && (
<div className="text-xs text-muted-foreground">
VRAM: {health.vram_used_mb.toFixed(0)} MB used
</div> </div>
)} <div className="text-sm text-muted-foreground">
</div> {health.gpu_type.replace(/\s*\(.+\)$/, '')}
)} {health.vram_used_mb != null && health.vram_used_mb > 0
? ` \u00b7 ${health.vram_used_mb.toFixed(0)} MB VRAM used`
: ''}
</div>
</>
) : (
<>
<div className="text-sm font-medium">CPU</div>
<div className="text-sm text-muted-foreground">No GPU acceleration available</div>
</>
)}
</div>
{/* Native GPU detected - no CUDA download needed */} {/* Native GPU detected - no CUDA download needed */}
{/* CUDA download section - only show when native GPU is NOT detected (i.e., Windows/Linux NVIDIA users) */} {/* CUDA download section - only show when no GPU is active (native or CUDA) */}
{!hasNativeGpu && ( {!hasNativeGpu && !isCurrentlyCuda && (
<> <>
{/* Download progress */} {/* Download progress (manual download or auto-update) */}
{cudaDownloading && downloadProgress && ( {cudaDownloading && downloadProgress && (
<div className="space-y-2"> <div className="space-y-2">
<div className="flex items-center justify-between text-sm"> <div className="flex items-center justify-between text-sm">
<div className="flex items-center gap-2"> <div className="flex items-center gap-2">
<Loader2 className="h-4 w-4 animate-spin" /> <Loader2 className="h-4 w-4 animate-spin" />
<span>{downloadProgress.filename || 'Downloading CUDA backend...'}</span> <span>
{downloadProgress.filename ||
(cudaAvailable
? 'Updating CUDA backend...'
: 'Downloading CUDA backend...')}
</span>
</div> </div>
{downloadProgress.total > 0 && ( {downloadProgress.total > 0 && (
<span className="text-muted-foreground"> <span className="text-muted-foreground">
@@ -446,8 +446,7 @@ export function ModelManagement() {
className="text-xs text-muted-foreground h-7 px-2" className="text-xs text-muted-foreground h-7 px-2"
onClick={async () => { onClick={async () => {
try { try {
const { open } = await import('@tauri-apps/plugin-shell'); await platform.filesystem.openPath(cacheDir.path);
await open(cacheDir.path);
} catch { } catch {
toast({ title: 'Failed to open model folder', variant: 'destructive' }); toast({ title: 'Failed to open model folder', variant: 'destructive' });
} }
@@ -462,14 +461,9 @@ export function ModelManagement() {
className="text-xs text-muted-foreground h-7 px-2" className="text-xs text-muted-foreground h-7 px-2"
onClick={async () => { onClick={async () => {
try { try {
const { open: openDialog } = await import('@tauri-apps/plugin-dialog'); const newDir = await platform.filesystem.pickDirectory(
const selected = await openDialog({ 'Choose model storage folder',
directory: true, );
title: 'Choose model storage folder',
});
if (!selected) return;
const newDir =
typeof selected === 'string' ? selected : (selected as { path: string }).path;
if (!newDir) return; if (!newDir) return;
setPendingMigrateDir(newDir); setPendingMigrateDir(newDir);
} catch { } catch {
@@ -11,6 +11,7 @@ export function UpdateStatus() {
const platform = usePlatform(); const platform = usePlatform();
const { status, checkForUpdates, downloadAndInstall, restartAndInstall } = useAutoUpdater(false); const { status, checkForUpdates, downloadAndInstall, restartAndInstall } = useAutoUpdater(false);
const [currentVersion, setCurrentVersion] = useState<string>(''); const [currentVersion, setCurrentVersion] = useState<string>('');
const isDev = !import.meta.env?.PROD;
useEffect(() => { useEffect(() => {
platform.metadata platform.metadata
@@ -20,11 +21,7 @@ export function UpdateStatus() {
}, [platform]); }, [platform]);
return ( return (
<Card <Card role="region" aria-label="App Updates" tabIndex={0}>
role="region"
aria-label="App Updates"
tabIndex={0}
>
<CardHeader> <CardHeader>
<CardTitle>App Updates</CardTitle> <CardTitle>App Updates</CardTitle>
</CardHeader> </CardHeader>
@@ -32,97 +29,110 @@ export function UpdateStatus() {
<div className="flex items-center justify-between"> <div className="flex items-center justify-between">
<div className="space-y-1"> <div className="space-y-1">
<div className="text-sm font-medium">Current Version</div> <div className="text-sm font-medium">Current Version</div>
<div className="text-sm text-muted-foreground">v{currentVersion}</div> <div className="text-sm text-muted-foreground">
v{currentVersion}
{isDev ? ' (dev)' : ''}
</div>
</div> </div>
<Button {!isDev && (
onClick={checkForUpdates} <Button
disabled={status.checking || status.downloading || status.readyToInstall} onClick={checkForUpdates}
variant="outline" disabled={status.checking || status.downloading || status.readyToInstall}
size="sm" variant="outline"
> size="sm"
<RefreshCw className={`h-4 w-4 mr-2 ${status.checking ? 'animate-spin' : ''}`} /> >
Check for Updates <RefreshCw className={`h-4 w-4 mr-2 ${status.checking ? 'animate-spin' : ''}`} />
</Button> Check for Updates
</Button>
)}
</div> </div>
{status.checking && ( {isDev ? (
<div className="flex items-center gap-2 text-sm text-muted-foreground"> <div className="text-sm text-muted-foreground">
<RefreshCw className="h-4 w-4 animate-spin" /> Auto-updates are disabled in development mode.
Checking for updates...
</div> </div>
)} ) : (
<>
{status.error && ( {status.checking && (
<div className="flex items-center gap-2 text-sm text-destructive"> <div className="flex items-center gap-2 text-sm text-muted-foreground">
<AlertCircle className="h-4 w-4" /> <RefreshCw className="h-4 w-4 animate-spin" />
{status.error} Checking for updates...
</div>
)}
{status.available && !status.downloading && !status.readyToInstall && (
<div className="space-y-3 p-4 border rounded-lg bg-primary/5">
<div className="flex items-center justify-between">
<div>
<div className="font-semibold">Update Available</div>
<div className="text-sm text-muted-foreground">Version {status.version}</div>
</div> </div>
<Badge>New</Badge> )}
</div>
<Button onClick={downloadAndInstall} className="w-full" size="sm">
<Download className="h-4 w-4 mr-2" />
Download Update
</Button>
</div>
)}
{status.downloading && ( {status.error && (
<div className="space-y-2"> <div className="flex items-center gap-2 text-sm text-destructive">
<div className="flex items-center justify-between text-sm"> <AlertCircle className="h-4 w-4" />
<div className="flex items-center gap-2"> {status.error}
<Download className="h-4 w-4" />
Downloading update...
</div> </div>
{status.downloadProgress !== undefined && ( )}
<span className="text-muted-foreground">{status.downloadProgress}%</span>
)} {status.available && !status.downloading && !status.readyToInstall && (
</div> <div className="space-y-3 p-4 border rounded-lg bg-primary/5">
<Progress value={status.downloadProgress} /> <div className="flex items-center justify-between">
{status.downloadedBytes !== undefined && <div>
status.totalBytes !== undefined && <div className="font-semibold">Update Available</div>
status.totalBytes > 0 && ( <div className="text-sm text-muted-foreground">Version {status.version}</div>
<div className="text-xs text-muted-foreground"> </div>
{(status.downloadedBytes / 1024 / 1024).toFixed(1)} MB /{' '} <Badge>New</Badge>
{(status.totalBytes / 1024 / 1024).toFixed(1)} MB
</div> </div>
)} <Button onClick={downloadAndInstall} className="w-full" size="sm">
</div> <Download className="h-4 w-4 mr-2" />
)} Download Update
</Button>
</div>
)}
{status.readyToInstall && ( {status.downloading && (
<div className="space-y-3 p-4 border rounded-lg bg-accent/30 border-accent/50"> <div className="space-y-2">
<div className="flex items-center gap-2"> <div className="flex items-center justify-between text-sm">
<div> <div className="flex items-center gap-2">
<div className="font-semibold">Update Ready to Install</div> <Download className="h-4 w-4" />
Downloading update...
</div>
{status.downloadProgress !== undefined && (
<span className="text-muted-foreground">{status.downloadProgress}%</span>
)}
</div>
<Progress value={status.downloadProgress} />
{status.downloadedBytes !== undefined &&
status.totalBytes !== undefined &&
status.totalBytes > 0 && (
<div className="text-xs text-muted-foreground">
{(status.downloadedBytes / 1024 / 1024).toFixed(1)} MB /{' '}
{(status.totalBytes / 1024 / 1024).toFixed(1)} MB
</div>
)}
</div>
)}
{status.readyToInstall && (
<div className="space-y-3 p-4 border rounded-lg bg-accent/30 border-accent/50">
<div className="flex items-center gap-2">
<div>
<div className="font-semibold">Update Ready to Install</div>
<div className="text-sm text-muted-foreground">
Version {status.version} has been downloaded
</div>
</div>
</div>
<div className="text-sm text-muted-foreground"> <div className="text-sm text-muted-foreground">
Version {status.version} has been downloaded The app needs to restart to complete the installation. You can do this now or
later at your convenience.
</div> </div>
<Button onClick={restartAndInstall} className="w-full" size="sm">
<RefreshCw className="h-4 w-4 mr-2" />
Restart Now
</Button>
</div> </div>
</div> )}
<div className="text-sm text-muted-foreground">
The app needs to restart to complete the installation. You can do this now or later at
your convenience.
</div>
<Button onClick={restartAndInstall} className="w-full" size="sm">
<RefreshCw className="h-4 w-4 mr-2" />
Restart Now
</Button>
</div>
)}
{!status.available && !status.checking && !status.error && status.checking === false && ( {!status.available && !status.checking && !status.error && (
<div className="flex items-center gap-2 text-sm text-muted-foreground"> <div className="flex items-center gap-2 text-sm text-muted-foreground">
You're up to date You're up to date
</div> </div>
)}
</>
)} )}
</CardContent> </CardContent>
</Card> </Card>
+48 -10
View File
@@ -1,7 +1,10 @@
import { Link, useMatchRoute } from '@tanstack/react-router'; import { Link, useMatchRoute } from '@tanstack/react-router';
import { BookOpen, Box, Mic, Server, Speaker, Volume2 } from 'lucide-react'; import { AudioLines, Box, Mic, Server, Speaker, Volume2, Wand2 } from 'lucide-react';
import { useEffect, useState } from 'react';
import voiceboxLogo from '@/assets/voicebox-logo.png'; import voiceboxLogo from '@/assets/voicebox-logo.png';
import { cn } from '@/lib/utils/cn'; import { cn } from '@/lib/utils/cn';
import { usePlatform } from '@/platform/PlatformContext';
import type { UpdateStatus } from '@/platform/types';
import { usePlayerStore } from '@/stores/playerStore'; import { usePlayerStore } from '@/stores/playerStore';
import { version } from '../../package.json'; import { version } from '../../package.json';
@@ -11,8 +14,9 @@ interface SidebarProps {
const tabs = [ const tabs = [
{ id: 'main', path: '/', icon: Volume2, label: 'Generate' }, { id: 'main', path: '/', icon: Volume2, label: 'Generate' },
{ id: 'stories', path: '/stories', icon: BookOpen, label: 'Stories' }, { id: 'stories', path: '/stories', icon: AudioLines, label: 'Stories' },
{ id: 'voices', path: '/voices', icon: Mic, label: 'Voices' }, { id: 'voices', path: '/voices', icon: Mic, label: 'Voices' },
{ id: 'effects', path: '/effects', icon: Wand2, label: 'Effects' },
{ id: 'audio', path: '/audio', icon: Speaker, label: 'Audio' }, { id: 'audio', path: '/audio', icon: Speaker, label: 'Audio' },
{ id: 'models', path: '/models', icon: Box, label: 'Models' }, { id: 'models', path: '/models', icon: Box, label: 'Models' },
{ id: 'server', path: '/server', icon: Server, label: 'Server' }, { id: 'server', path: '/server', icon: Server, label: 'Server' },
@@ -21,6 +25,10 @@ const tabs = [
export function Sidebar({ isMacOS }: SidebarProps) { export function Sidebar({ isMacOS }: SidebarProps) {
const matchRoute = useMatchRoute(); const matchRoute = useMatchRoute();
const isPlayerOpen = !!usePlayerStore((s) => s.audioUrl); const isPlayerOpen = !!usePlayerStore((s) => s.audioUrl);
const platform = usePlatform();
const [updateStatus, setUpdateStatus] = useState<UpdateStatus>(platform.updater.getStatus());
useEffect(() => platform.updater.subscribe(setUpdateStatus), [platform.updater]);
return ( return (
<div <div
@@ -31,30 +39,52 @@ export function Sidebar({ isMacOS }: SidebarProps) {
> >
{/* Logo */} {/* Logo */}
<div className="mb-2"> <div className="mb-2">
<img src={voiceboxLogo} alt="Voicebox" className="w-12 h-12 object-contain" /> <img
src={voiceboxLogo}
alt="Voicebox"
className="w-12 h-12 object-contain"
style={{
filter:
'drop-shadow(0 0 6px hsl(var(--accent) / 0.5)) drop-shadow(0 0 14px hsl(var(--accent) / 0.35)) drop-shadow(0 0 28px hsl(var(--accent) / 0.2))',
}}
/>
</div> </div>
{/* Navigation Buttons */} {/* Navigation Buttons */}
<div className="flex flex-col gap-3"> <div className="flex flex-col gap-3">
{tabs.map((tab) => { {tabs.map((tab, index) => {
const Icon = tab.icon; const Icon = tab.icon;
// For index route, use exact match; for others, use default matching // For index route, use exact match; for others, use default matching
const isActive = const isActive =
tab.path === '/' ? matchRoute({ to: '/', exact: true }) : matchRoute({ to: tab.path }); tab.path === '/' ? matchRoute({ to: '/', exact: true }) : matchRoute({ to: tab.path });
// Accent fades as buttons get further from the logo
const accentOpacity = Math.max(0.08, 0.5 - index * 0.07);
return ( return (
<Link <Link
key={tab.id} key={tab.id}
to={tab.path} to={tab.path}
className={cn( className={cn(
'w-12 h-12 rounded-full flex items-center justify-center transition-all duration-200', 'relative w-12 h-12 rounded-full flex items-center justify-center transition-all duration-200 overflow-hidden',
'hover:bg-muted/50', isActive
isActive ? 'bg-muted/50 text-foreground shadow-lg' : 'text-muted-foreground', ? 'bg-white/[0.07] text-foreground shadow-lg backdrop-blur-sm border border-white/[0.08]'
: 'text-muted-foreground hover:bg-muted/50',
)} )}
title={tab.label} title={tab.label}
aria-label={tab.label} aria-label={tab.label}
> >
<Icon className="h-5 w-5" /> {isActive && (
<div
className="absolute inset-0 rounded-full pointer-events-none"
style={{
maskImage: 'linear-gradient(to bottom, black, transparent 60%)',
WebkitMaskImage: 'linear-gradient(to bottom, black, transparent 60%)',
border: `1px solid hsl(var(--accent) / ${accentOpacity})`,
}}
/>
)}
<Icon className="h-5 w-5 relative z-10" />
</Link> </Link>
); );
})} })}
@@ -62,10 +92,18 @@ export function Sidebar({ isMacOS }: SidebarProps) {
{/* Version */} {/* Version */}
<div <div
className="mt-auto text-[10px] text-muted-foreground/50 transition-all duration-300" className="mt-auto flex flex-col items-center gap-1.5 transition-all duration-300"
style={{ paddingBottom: isPlayerOpen ? '7rem' : undefined }} style={{ paddingBottom: isPlayerOpen ? '7rem' : undefined }}
> >
v{version} <span className="text-[10px] text-muted-foreground/50">v{version}</span>
{updateStatus.available && (
<Link
to="/server"
className="text-[9px] font-semibold tracking-wide uppercase px-2 py-0.5 rounded-full bg-accent/15 text-accent hover:bg-accent/25 transition-colors"
>
Update
</Link>
)}
</div> </div>
</div> </div>
); );
+96 -79
View File
@@ -1,5 +1,5 @@
import { Plus, BookOpen, MoreHorizontal, Pencil, Trash2 } from 'lucide-react'; import { BookOpen, MoreHorizontal, Pencil, Plus, Trash2 } from 'lucide-react';
import { useState } from 'react'; import { useEffect, useState } from 'react';
import { import {
AlertDialog, AlertDialog,
AlertDialogAction, AlertDialogAction,
@@ -29,7 +29,13 @@ import { Input } from '@/components/ui/input';
import { Label } from '@/components/ui/label'; import { Label } from '@/components/ui/label';
import { Textarea } from '@/components/ui/textarea'; import { Textarea } from '@/components/ui/textarea';
import { useToast } from '@/components/ui/use-toast'; import { useToast } from '@/components/ui/use-toast';
import { useStories, useCreateStory, useUpdateStory, useDeleteStory } from '@/lib/hooks/useStories'; import {
useCreateStory,
useDeleteStory,
useStories,
useStory,
useUpdateStory,
} from '@/lib/hooks/useStories';
import { cn } from '@/lib/utils/cn'; import { cn } from '@/lib/utils/cn';
import { formatDate } from '@/lib/utils/format'; import { formatDate } from '@/lib/utils/format';
import { useStoryStore } from '@/stores/storyStore'; import { useStoryStore } from '@/stores/storyStore';
@@ -38,6 +44,8 @@ export function StoryList() {
const { data: stories, isLoading } = useStories(); const { data: stories, isLoading } = useStories();
const selectedStoryId = useStoryStore((state) => state.selectedStoryId); const selectedStoryId = useStoryStore((state) => state.selectedStoryId);
const setSelectedStoryId = useStoryStore((state) => state.setSelectedStoryId); const setSelectedStoryId = useStoryStore((state) => state.setSelectedStoryId);
const trackEditorHeight = useStoryStore((state) => state.trackEditorHeight);
const { data: selectedStory } = useStory(selectedStoryId);
const createStory = useCreateStory(); const createStory = useCreateStory();
const updateStory = useUpdateStory(); const updateStory = useUpdateStory();
const deleteStory = useDeleteStory(); const deleteStory = useDeleteStory();
@@ -54,6 +62,13 @@ export function StoryList() {
const [newStoryDescription, setNewStoryDescription] = useState(''); const [newStoryDescription, setNewStoryDescription] = useState('');
const { toast } = useToast(); const { toast } = useToast();
// Auto-select the first story when the list loads with no selection
useEffect(() => {
if (!selectedStoryId && stories && stories.length > 0) {
setSelectedStoryId(stories[0].id);
}
}, [selectedStoryId, stories, setSelectedStoryId]);
const handleCreateStory = () => { const handleCreateStory = () => {
if (!newStoryName.trim()) { if (!newStoryName.trim()) {
toast({ toast({
@@ -170,20 +185,29 @@ export function StoryList() {
} }
const storyList = stories || []; const storyList = stories || [];
const hasTrackEditor = selectedStoryId && selectedStory && selectedStory.items.length > 0;
return ( return (
<div className="flex flex-col h-full min-h-0"> <div className="h-full flex flex-col relative overflow-hidden">
{/* Header */} {/* Scroll Mask */}
<div className="flex items-center justify-between mb-4 px-1"> <div className="absolute top-0 left-0 right-0 h-16 bg-gradient-to-b from-background to-transparent z-10 pointer-events-none" />
<h2 className="text-2xl font-bold">Stories</h2>
<Button onClick={() => setCreateDialogOpen(true)} size="sm"> {/* Fixed Header */}
<Plus className="mr-2 h-4 w-4" /> <div className="absolute top-0 left-0 right-0 z-20">
New Story <div className="flex items-center justify-between mb-4 px-1">
</Button> <h2 className="text-2xl font-bold">Stories</h2>
<Button onClick={() => setCreateDialogOpen(true)} size="sm">
<Plus className="mr-2 h-4 w-4" />
New Story
</Button>
</div>
</div> </div>
{/* Story List */} {/* Scrollable Story List */}
<div className="flex-1 min-h-0 overflow-y-auto space-y-2"> <div
className="flex-1 overflow-y-auto pt-14 relative z-0"
style={{ paddingBottom: hasTrackEditor ? `${trackEditorHeight + 140}px` : '170px' }}
>
{storyList.length === 0 ? ( {storyList.length === 0 ? (
<div className="text-center py-12 px-5 border-2 border-dashed border-muted rounded-2xl text-muted-foreground"> <div className="text-center py-12 px-5 border-2 border-dashed border-muted rounded-2xl text-muted-foreground">
<BookOpen className="h-12 w-12 mx-auto mb-4 opacity-50" /> <BookOpen className="h-12 w-12 mx-auto mb-4 opacity-50" />
@@ -191,75 +215,68 @@ export function StoryList() {
<p className="text-xs mt-2">Create your first story to get started</p> <p className="text-xs mt-2">Create your first story to get started</p>
</div> </div>
) : ( ) : (
storyList.map((story) => ( <div className="space-y-0.5">
<div {storyList.map((story) => (
key={story.id} <div
role="button" key={story.id}
tabIndex={0} role="button"
className={cn( tabIndex={0}
'h-24 p-4 border rounded-2xl transition-colors group flex items-center cursor-pointer', className={cn(
selectedStoryId === story.id && 'bg-muted border-primary', 'px-5 py-3 rounded-lg transition-colors group flex items-center cursor-pointer',
)} selectedStoryId === story.id ? 'bg-muted' : 'hover:bg-muted/50',
aria-label={ )}
selectedStoryId === story.id aria-label={`Story ${story.name}, ${story.item_count} ${story.item_count === 1 ? 'item' : 'items'}, ${formatDate(story.updated_at)}`}
? `Story ${story.name}, ${story.item_count} ${story.item_count === 1 ? 'item' : 'items'}, ${formatDate(story.updated_at)}. Selected. Press Enter to select.` aria-pressed={selectedStoryId === story.id}
: `Story ${story.name}, ${story.item_count} ${story.item_count === 1 ? 'item' : 'items'}, ${formatDate(story.updated_at)}. Press Enter to select.` onClick={() => setSelectedStoryId(story.id)}
} onKeyDown={(e) => {
aria-pressed={selectedStoryId === story.id} if (e.target !== e.currentTarget) return;
onClick={() => setSelectedStoryId(story.id)} if (e.key === 'Enter' || e.key === ' ') {
onKeyDown={(e) => { e.preventDefault();
if (e.target !== e.currentTarget) return; setSelectedStoryId(story.id);
if (e.key === 'Enter' || e.key === ' ') { }
e.preventDefault(); }}
setSelectedStoryId(story.id); >
} <div className="flex items-start justify-between gap-2 w-full min-w-0">
}} <div className="flex-1 min-w-0 text-left overflow-hidden">
> <h3 className="text-sm font-medium truncate">{story.name}</h3>
<div className="flex items-start justify-between gap-2 w-full min-w-0"> <div className="flex items-center gap-2 mt-1 text-xs text-muted-foreground">
<div className="flex-1 min-w-0 text-left overflow-hidden"> <span>
<h3 className="font-medium truncate">{story.name}</h3> {story.item_count} {story.item_count === 1 ? 'item' : 'items'}
{story.description && ( </span>
<p className="text-sm text-muted-foreground mt-1 truncate"> <span>·</span>
{story.description} <span>{formatDate(story.updated_at)}</span>
</p> </div>
)}
<div className="flex items-center gap-3 mt-2 text-xs text-muted-foreground">
<span>
{story.item_count} {story.item_count === 1 ? 'item' : 'items'}
</span>
<span>•</span>
<span>{formatDate(story.updated_at)}</span>
</div> </div>
<DropdownMenu>
<DropdownMenuTrigger asChild>
<Button
variant="ghost"
size="icon"
className="h-7 w-7 opacity-0 group-hover:opacity-100 transition-opacity"
onClick={(e) => e.stopPropagation()}
aria-label={`Actions for ${story.name}`}
>
<MoreHorizontal className="h-3.5 w-3.5" />
</Button>
</DropdownMenuTrigger>
<DropdownMenuContent align="end">
<DropdownMenuItem onClick={() => handleEditClick(story)}>
<Pencil className="mr-2 h-4 w-4" />
Edit
</DropdownMenuItem>
<DropdownMenuItem
onClick={() => handleDeleteClick(story.id)}
className="text-destructive focus:text-destructive"
>
<Trash2 className="mr-2 h-4 w-4" />
Delete
</DropdownMenuItem>
</DropdownMenuContent>
</DropdownMenu>
</div> </div>
<DropdownMenu>
<DropdownMenuTrigger asChild>
<Button
variant="ghost"
size="icon"
className="h-8 w-8 opacity-0 group-hover:opacity-100 transition-opacity"
onClick={(e) => e.stopPropagation()}
aria-label={`Actions for ${story.name}`}
>
<MoreHorizontal className="h-4 w-4" />
</Button>
</DropdownMenuTrigger>
<DropdownMenuContent align="end">
<DropdownMenuItem onClick={() => handleEditClick(story)}>
<Pencil className="mr-2 h-4 w-4" />
Edit
</DropdownMenuItem>
<DropdownMenuItem
onClick={() => handleDeleteClick(story.id)}
className="text-destructive focus:text-destructive"
>
<Trash2 className="mr-2 h-4 w-4" />
Delete
</DropdownMenuItem>
</DropdownMenuContent>
</DropdownMenu>
</div> </div>
</div> ))}
)) </div>
)} )}
</div> </div>
@@ -1,5 +1,7 @@
import { import {
Check,
Copy, Copy,
GalleryVerticalEnd,
GripHorizontal, GripHorizontal,
Minus, Minus,
Pause, Pause,
@@ -12,6 +14,12 @@ import {
import { useCallback, useEffect, useMemo, useRef, useState } from 'react'; import { useCallback, useEffect, useMemo, useRef, useState } from 'react';
import WaveSurfer from 'wavesurfer.js'; import WaveSurfer from 'wavesurfer.js';
import { Button } from '@/components/ui/button'; import { Button } from '@/components/ui/button';
import {
DropdownMenu,
DropdownMenuContent,
DropdownMenuItem,
DropdownMenuTrigger,
} from '@/components/ui/dropdown-menu';
import { useToast } from '@/components/ui/use-toast'; import { useToast } from '@/components/ui/use-toast';
import { apiClient } from '@/lib/api/client'; import { apiClient } from '@/lib/api/client';
import type { StoryItemDetail } from '@/lib/api/types'; import type { StoryItemDetail } from '@/lib/api/types';
@@ -19,6 +27,7 @@ import {
useDuplicateStoryItem, useDuplicateStoryItem,
useMoveStoryItem, useMoveStoryItem,
useRemoveStoryItem, useRemoveStoryItem,
useSetStoryItemVersion,
useSplitStoryItem, useSplitStoryItem,
useTrimStoryItem, useTrimStoryItem,
} from '@/lib/hooks/useStories'; } from '@/lib/hooks/useStories';
@@ -28,12 +37,14 @@ import { useStoryStore } from '@/stores/storyStore';
// Clip waveform component with trim support // Clip waveform component with trim support
function ClipWaveform({ function ClipWaveform({
generationId, generationId,
versionId,
width, width,
trimStartMs, trimStartMs,
trimEndMs, trimEndMs,
duration, duration,
}: { }: {
generationId: string; generationId: string;
versionId?: string;
width: number; width: number;
trimStartMs: number; trimStartMs: number;
trimEndMs: number; trimEndMs: number;
@@ -79,7 +90,9 @@ function ClipWaveform({
wavesurferRef.current = wavesurfer; wavesurferRef.current = wavesurfer;
const audioUrl = apiClient.getAudioUrl(generationId); const audioUrl = versionId
? apiClient.getVersionAudioUrl(versionId)
: apiClient.getAudioUrl(generationId);
wavesurfer.load(audioUrl).catch(() => { wavesurfer.load(audioUrl).catch(() => {
// Ignore load errors // Ignore load errors
}); });
@@ -88,7 +101,7 @@ function ClipWaveform({
wavesurfer.destroy(); wavesurfer.destroy();
wavesurferRef.current = null; wavesurferRef.current = null;
}; };
}, [generationId, fullWaveformWidth]); }, [generationId, versionId, fullWaveformWidth]);
return ( return (
<div className="w-full h-full opacity-60 overflow-hidden"> <div className="w-full h-full opacity-60 overflow-hidden">
@@ -135,12 +148,57 @@ export function StoryTrackEditor({ storyId, items }: StoryTrackEditorProps) {
const splitItem = useSplitStoryItem(); const splitItem = useSplitStoryItem();
const duplicateItem = useDuplicateStoryItem(); const duplicateItem = useDuplicateStoryItem();
const removeItem = useRemoveStoryItem(); const removeItem = useRemoveStoryItem();
const setItemVersion = useSetStoryItemVersion();
const { toast } = useToast(); const { toast } = useToast();
// Selection state // Selection state
const selectedClipId = useStoryStore((state) => state.selectedClipId); const selectedClipId = useStoryStore((state) => state.selectedClipId);
const setSelectedClipId = useStoryStore((state) => state.setSelectedClipId); const setSelectedClipId = useStoryStore((state) => state.setSelectedClipId);
// Selected clip item (for version picker)
const selectedItem = useMemo(
() => (selectedClipId ? items.find((i) => i.id === selectedClipId) : undefined),
[selectedClipId, items],
);
const selectedItemVersions = selectedItem?.versions;
const hasMultipleVersions = selectedItemVersions && selectedItemVersions.length > 1;
// Determine which version label is active for the selected clip
const activeVersionLabel = useMemo(() => {
if (!selectedItem || !selectedItemVersions) return null;
// If the item has a pinned version_id, find its label
if (selectedItem.version_id) {
const pinned = selectedItemVersions.find((v) => v.id === selectedItem.version_id);
return pinned?.label ?? null;
}
// Otherwise use the generation's default version
const defaultVersion = selectedItemVersions.find((v) => v.is_default);
return defaultVersion?.label ?? null;
}, [selectedItem, selectedItemVersions]);
const handleSetVersion = useCallback(
(versionId: string | null) => {
if (!selectedClipId) return;
setItemVersion.mutate(
{
storyId,
itemId: selectedClipId,
data: { version_id: versionId },
},
{
onError: (error) => {
toast({
title: 'Failed to set version',
description: error instanceof Error ? error.message : String(error),
variant: 'destructive',
});
},
},
);
},
[selectedClipId, storyId, setItemVersion, toast],
);
// Trim state // Trim state
const [trimmingItem, setTrimmingItem] = useState<string | null>(null); const [trimmingItem, setTrimmingItem] = useState<string | null>(null);
const [trimSide, setTrimSide] = useState<'start' | 'end' | null>(null); const [trimSide, setTrimSide] = useState<'start' | 'end' | null>(null);
@@ -788,6 +846,49 @@ export function StoryTrackEditor({ storyId, items }: StoryTrackEditorProps) {
> >
<Trash2 className="h-4 w-4" /> <Trash2 className="h-4 w-4" />
</Button> </Button>
{hasMultipleVersions && (
<>
<div className="w-px h-4 bg-border mx-1" />
<DropdownMenu>
<DropdownMenuTrigger asChild>
<Button
variant="ghost"
className="h-7 gap-1.5 px-2 text-xs"
title="Change version/take"
>
<GalleryVerticalEnd className="h-3.5 w-3.5" />
<span className="max-w-[80px] truncate">
{activeVersionLabel ?? 'default'}
</span>
</Button>
</DropdownMenuTrigger>
<DropdownMenuContent align="center" className="min-w-[160px]">
{selectedItemVersions.map((version) => {
const isActive = selectedItem?.version_id
? version.id === selectedItem.version_id
: version.is_default;
return (
<DropdownMenuItem
key={version.id}
onClick={() => handleSetVersion(version.id)}
className="gap-2 text-xs"
>
<Check
className={cn('h-3 w-3', isActive ? 'opacity-100' : 'opacity-0')}
/>
<span className="truncate">{version.label}</span>
{version.effects_chain && version.effects_chain.length > 0 && (
<span className="text-muted-foreground ml-auto text-[10px]">
{version.effects_chain.length} fx
</span>
)}
</DropdownMenuItem>
);
})}
</DropdownMenuContent>
</DropdownMenu>
</>
)}
</div> </div>
)} )}
@@ -958,6 +1059,7 @@ export function StoryTrackEditor({ storyId, items }: StoryTrackEditorProps) {
<div className="absolute inset-0 top-3"> <div className="absolute inset-0 top-3">
<ClipWaveform <ClipWaveform
generationId={item.generation_id} generationId={item.generation_id}
versionId={item.version_id}
width={clipWidth} width={clipWidth}
trimStartMs={displayTrimStart} trimStartMs={displayTrimStart}
trimEndMs={displayTrimEnd} trimEndMs={displayTrimEnd}
+5 -6
View File
@@ -1,8 +1,7 @@
const isWindows = navigator.userAgent.includes('Windows');
export function TitleBarDragRegion() { export function TitleBarDragRegion() {
return ( if (isWindows) return null;
<div
data-tauri-drag-region return <div data-tauri-drag-region className="fixed top-0 left-0 right-0 h-12 z-[9999]" />;
className="fixed top-0 left-0 right-0 h-12 z-[9999]"
/>
);
} }
@@ -1,4 +1,4 @@
import { Download, Edit, Mic, Trash2 } from 'lucide-react'; import { Download, Edit, Sparkles, Trash2 } from 'lucide-react';
import { useState } from 'react'; import { useState } from 'react';
import { Badge } from '@/components/ui/badge'; import { Badge } from '@/components/ui/badge';
import { Button } from '@/components/ui/button'; import { Button } from '@/components/ui/button';
@@ -15,7 +15,6 @@ import {
import type { VoiceProfileResponse } from '@/lib/api/types'; import type { VoiceProfileResponse } from '@/lib/api/types';
import { useDeleteProfile, useExportProfile } from '@/lib/hooks/useProfiles'; import { useDeleteProfile, useExportProfile } from '@/lib/hooks/useProfiles';
import { cn } from '@/lib/utils/cn'; import { cn } from '@/lib/utils/cn';
import { useServerStore } from '@/stores/serverStore';
import { useUIStore } from '@/stores/uiStore'; import { useUIStore } from '@/stores/uiStore';
interface ProfileCardProps { interface ProfileCardProps {
@@ -24,19 +23,16 @@ interface ProfileCardProps {
export function ProfileCard({ profile }: ProfileCardProps) { export function ProfileCard({ profile }: ProfileCardProps) {
const [deleteDialogOpen, setDeleteDialogOpen] = useState(false); const [deleteDialogOpen, setDeleteDialogOpen] = useState(false);
const [avatarError, setAvatarError] = useState(false);
const deleteProfile = useDeleteProfile(); const deleteProfile = useDeleteProfile();
const exportProfile = useExportProfile(); const exportProfile = useExportProfile();
const setEditingProfileId = useUIStore((state) => state.setEditingProfileId); const setEditingProfileId = useUIStore((state) => state.setEditingProfileId);
const setProfileDialogOpen = useUIStore((state) => state.setProfileDialogOpen); const setProfileDialogOpen = useUIStore((state) => state.setProfileDialogOpen);
const selectedProfileId = useUIStore((state) => state.selectedProfileId); const selectedProfileId = useUIStore((state) => state.selectedProfileId);
const setSelectedProfileId = useUIStore((state) => state.setSelectedProfileId); const setSelectedProfileId = useUIStore((state) => state.setSelectedProfileId);
const serverUrl = useServerStore((state) => state.serverUrl);
const isSelected = selectedProfileId === profile.id; const isSelected = selectedProfileId === profile.id;
const avatarUrl = profile.avatar_path ? `${serverUrl}/profiles/${profile.id}/avatar` : null;
const handleSelect = () => { const handleSelect = () => {
setSelectedProfileId(isSelected ? null : profile.id); setSelectedProfileId(isSelected ? null : profile.id);
}; };
@@ -79,7 +75,7 @@ export function ProfileCard({ profile }: ProfileCardProps) {
<Card <Card
className={cn( className={cn(
'cursor-pointer hover:shadow-md transition-all flex flex-col h-[162px]', 'cursor-pointer hover:shadow-md transition-all flex flex-col h-[162px]',
isSelected && 'ring-2 ring-primary shadow-md', isSelected && 'ring-2 ring-accent shadow-md',
)} )}
onClick={handleSelect} onClick={handleSelect}
tabIndex={0} tabIndex={0}
@@ -89,22 +85,7 @@ export function ProfileCard({ profile }: ProfileCardProps) {
onKeyDown={handleKeyDown} onKeyDown={handleKeyDown}
> >
<CardHeader className="p-3 pb-2"> <CardHeader className="p-3 pb-2">
<CardTitle className="flex items-center gap-1.5 text-base font-medium"> <CardTitle className="text-base font-medium">
<div className="h-6 w-6 rounded-full bg-muted flex items-center justify-center shrink-0 overflow-hidden">
{avatarUrl && !avatarError ? (
<img
src={avatarUrl}
alt={`${profile.name} avatar`}
className={cn(
'h-full w-full object-cover transition-all duration-200',
!isSelected && 'grayscale',
)}
onError={() => setAvatarError(true)}
/>
) : (
<Mic className="h-3.5 w-3.5 text-muted-foreground" />
)}
</div>
<span className="break-words">{profile.name}</span> <span className="break-words">{profile.name}</span>
</CardTitle> </CardTitle>
</CardHeader> </CardHeader>
@@ -112,10 +93,13 @@ export function ProfileCard({ profile }: ProfileCardProps) {
<p className="text-xs text-muted-foreground mb-1.5 line-clamp-2 leading-relaxed"> <p className="text-xs text-muted-foreground mb-1.5 line-clamp-2 leading-relaxed">
{profile.description || 'No description'} {profile.description || 'No description'}
</p> </p>
<div className="mb-2"> <div className="mb-2 flex items-center gap-1.5">
<Badge variant="outline" className="text-xs h-5 px-1.5 text-muted-foreground"> <Badge variant="outline" className="text-xs h-5 px-1.5 text-muted-foreground">
{profile.language} {profile.language}
</Badge> </Badge>
{profile.effects_chain && profile.effects_chain.length > 0 && (
<Sparkles className="h-3.5 w-3.5 text-accent fill-accent" />
)}
</div> </div>
<div className="flex gap-0.5 justify-end items-end mt-auto"> <div className="flex gap-0.5 justify-end items-end mt-auto">
<CircleButton <CircleButton
@@ -3,6 +3,7 @@ import { Edit2, Mic, Monitor, Upload, X } from 'lucide-react';
import { useEffect, useRef, useState } from 'react'; import { useEffect, useRef, useState } from 'react';
import { useForm } from 'react-hook-form'; import { useForm } from 'react-hook-form';
import * as z from 'zod'; import * as z from 'zod';
import { EffectsChainEditor } from '@/components/Effects/EffectsChainEditor';
import { Button } from '@/components/ui/button'; import { Button } from '@/components/ui/button';
import { import {
Dialog, Dialog,
@@ -30,6 +31,8 @@ import {
import { Tabs, TabsContent, TabsList, TabsTrigger } from '@/components/ui/tabs'; import { Tabs, TabsContent, TabsList, TabsTrigger } from '@/components/ui/tabs';
import { Textarea } from '@/components/ui/textarea'; import { Textarea } from '@/components/ui/textarea';
import { useToast } from '@/components/ui/use-toast'; import { useToast } from '@/components/ui/use-toast';
import { apiClient } from '@/lib/api/client';
import type { EffectConfig } from '@/lib/api/types';
import { LANGUAGE_CODES, LANGUAGE_OPTIONS, type LanguageCode } from '@/lib/constants/languages'; import { LANGUAGE_CODES, LANGUAGE_OPTIONS, type LanguageCode } from '@/lib/constants/languages';
import { useAudioPlayer } from '@/lib/hooks/useAudioPlayer'; import { useAudioPlayer } from '@/lib/hooks/useAudioPlayer';
import { useAudioRecording } from '@/lib/hooks/useAudioRecording'; import { useAudioRecording } from '@/lib/hooks/useAudioRecording';
@@ -125,6 +128,8 @@ export function ProfileForm() {
const { isPlaying, playPause, cleanup: cleanupAudio } = useAudioPlayer(); const { isPlaying, playPause, cleanup: cleanupAudio } = useAudioPlayer();
const isCreating = !editingProfileId; const isCreating = !editingProfileId;
const serverUrl = useServerStore((state) => state.serverUrl); const serverUrl = useServerStore((state) => state.serverUrl);
const [profileEffectsChain, setProfileEffectsChain] = useState<EffectConfig[]>([]);
const [effectsDirty, setEffectsDirty] = useState(false);
const form = useForm<ProfileFormValues>({ const form = useForm<ProfileFormValues>({
resolver: zodResolver(profileSchema), resolver: zodResolver(profileSchema),
@@ -280,6 +285,8 @@ export function ProfileForm() {
referenceText: undefined, referenceText: undefined,
avatarFile: undefined, avatarFile: undefined,
}); });
setProfileEffectsChain(editingProfile.effects_chain ?? []);
setEffectsDirty(false);
} else if (profileFormDraft && open) { } else if (profileFormDraft && open) {
// Restore from draft when opening in create mode // Restore from draft when opening in create mode
form.reset({ form.reset({
@@ -435,6 +442,24 @@ export function ProfileForm() {
} }
} }
// Save effects chain if changed
if (effectsDirty) {
try {
await apiClient.updateProfileEffects(
editingProfileId,
profileEffectsChain.length > 0 ? profileEffectsChain : null,
);
} catch (fxError) {
toast({
title: 'Effects update failed',
description:
fxError instanceof Error ? fxError.message : 'Failed to save effects chain',
variant: 'destructive',
});
return;
}
}
toast({ toast({
title: 'Voice updated', title: 'Voice updated',
description: `"${data.name}" has been updated successfully.`, description: `"${data.name}" has been updated successfully.`,
@@ -898,6 +923,23 @@ export function ProfileForm() {
</FormItem> </FormItem>
)} )}
/> />
{editingProfileId && (
<div className="space-y-2">
<FormLabel>Default Effects</FormLabel>
<p className="text-xs text-muted-foreground">
Effects applied automatically to all new generations with this voice.
</p>
<EffectsChainEditor
value={profileEffectsChain}
onChange={(chain) => {
setProfileEffectsChain(chain);
setEffectsDirty(true);
}}
compact
/>
</div>
)}
</div> </div>
</div> </div>
@@ -0,0 +1,340 @@
import { zodResolver } from '@hookform/resolvers/zod';
import { Edit2, Mic, X } from 'lucide-react';
import { useEffect, useRef, useState } from 'react';
import { useForm } from 'react-hook-form';
import * as z from 'zod';
import { EffectsChainEditor } from '@/components/Effects/EffectsChainEditor';
import { Button } from '@/components/ui/button';
import {
Form,
FormControl,
FormField,
FormItem,
FormLabel,
FormMessage,
} from '@/components/ui/form';
import { Input } from '@/components/ui/input';
import {
Select,
SelectContent,
SelectItem,
SelectTrigger,
SelectValue,
} from '@/components/ui/select';
import { Textarea } from '@/components/ui/textarea';
import { useToast } from '@/components/ui/use-toast';
import { SampleList } from '@/components/VoiceProfiles/SampleList';
import { apiClient } from '@/lib/api/client';
import type { EffectConfig } from '@/lib/api/types';
import { LANGUAGE_CODES, LANGUAGE_OPTIONS, type LanguageCode } from '@/lib/constants/languages';
import { BOTTOM_SAFE_AREA_PADDING } from '@/lib/constants/ui';
import {
useDeleteAvatar,
useProfile,
useUpdateProfile,
useUploadAvatar,
} from '@/lib/hooks/useProfiles';
import { cn } from '@/lib/utils/cn';
import { usePlayerStore } from '@/stores/playerStore';
import { useServerStore } from '@/stores/serverStore';
const profileSchema = z.object({
name: z.string().min(1, 'Name is required').max(100),
description: z.string().max(500).optional(),
language: z.enum(LANGUAGE_CODES as [LanguageCode, ...LanguageCode[]]),
});
type ProfileFormValues = z.infer<typeof profileSchema>;
interface VoiceInspectorProps {
profileId: string;
}
export function VoiceInspector({ profileId }: VoiceInspectorProps) {
const { data: profile } = useProfile(profileId);
const audioUrl = usePlayerStore((state) => state.audioUrl);
const isPlayerVisible = !!audioUrl;
const updateProfile = useUpdateProfile();
const uploadAvatar = useUploadAvatar();
const deleteAvatar = useDeleteAvatar();
const serverUrl = useServerStore((state) => state.serverUrl);
const { toast } = useToast();
const [avatarPreview, setAvatarPreview] = useState<string | null>(null);
const [avatarError, setAvatarError] = useState(false);
const avatarInputRef = useRef<HTMLInputElement>(null);
const [effectsChain, setEffectsChain] = useState<EffectConfig[]>([]);
const [effectsDirty, setEffectsDirty] = useState(false);
const form = useForm<ProfileFormValues>({
resolver: zodResolver(profileSchema),
defaultValues: {
name: '',
description: '',
language: 'en',
},
});
// Populate form when profile loads
useEffect(() => {
if (profile) {
form.reset({
name: profile.name,
description: profile.description || '',
language: profile.language as LanguageCode,
});
setEffectsChain(profile.effects_chain ?? []);
setEffectsDirty(false);
}
}, [profile, form]);
// Avatar preview
useEffect(() => {
if (profile?.avatar_path) {
setAvatarPreview(`${serverUrl}/profiles/${profile.id}/avatar`);
} else {
setAvatarPreview(null);
}
setAvatarError(false);
}, [profile, serverUrl]);
function handleAvatarFileChange(e: React.ChangeEvent<HTMLInputElement>) {
const file = e.target.files?.[0];
if (!file) return;
if (!file.type.startsWith('image/')) {
toast({
title: 'Invalid file type',
description: 'Please select PNG, JPG, or WebP',
variant: 'destructive',
});
return;
}
if (file.size > 5 * 1024 * 1024) {
toast({
title: 'File too large',
description: 'Image must be less than 5MB',
variant: 'destructive',
});
return;
}
// Upload immediately
uploadAvatar.mutate(
{ profileId, file },
{
onSuccess: () => {
setAvatarPreview(URL.createObjectURL(file));
toast({ title: 'Avatar updated' });
},
onError: (err) => {
toast({
title: 'Avatar upload failed',
description: err instanceof Error ? err.message : 'Unknown error',
variant: 'destructive',
});
},
},
);
}
async function handleRemoveAvatar() {
if (profile?.avatar_path) {
try {
await deleteAvatar.mutateAsync(profileId);
toast({ title: 'Avatar removed' });
} catch (err) {
toast({
title: 'Failed to remove avatar',
description: err instanceof Error ? err.message : 'Unknown error',
variant: 'destructive',
});
}
}
setAvatarPreview(null);
if (avatarInputRef.current) avatarInputRef.current.value = '';
}
async function onSubmit(data: ProfileFormValues) {
try {
await updateProfile.mutateAsync({
profileId,
data: {
name: data.name,
description: data.description,
language: data.language,
},
});
if (effectsDirty) {
try {
await apiClient.updateProfileEffects(
profileId,
effectsChain.length > 0 ? effectsChain : null,
);
setEffectsDirty(false);
} catch (fxError) {
toast({
title: 'Effects update failed',
description: fxError instanceof Error ? fxError.message : 'Failed to save effects',
variant: 'destructive',
});
return;
}
}
toast({ title: 'Voice updated', description: `"${data.name}" saved.` });
} catch (error) {
toast({
title: 'Error',
description: error instanceof Error ? error.message : 'Failed to save profile',
variant: 'destructive',
});
}
}
if (!profile) {
return (
<div className="flex items-center justify-center h-full text-muted-foreground text-sm">
Loading...
</div>
);
}
const isDirty = form.formState.isDirty || effectsDirty;
return (
<div className="h-full flex flex-col overflow-hidden">
<div className={cn('flex-1 overflow-y-auto', isPlayerVisible && BOTTOM_SAFE_AREA_PADDING)}>
<Form {...form}>
<form onSubmit={form.handleSubmit(onSubmit)} className="space-y-0">
{/* Avatar */}
<div className="flex justify-center pt-5 pb-3">
<div className="relative group">
<div className="h-20 w-20 rounded-full bg-muted flex items-center justify-center shrink-0 overflow-hidden border-2 border-border">
{avatarPreview && !avatarError ? (
<img
src={avatarPreview}
alt={profile.name}
className="h-full w-full object-cover"
onError={() => setAvatarError(true)}
/>
) : (
<Mic className="h-8 w-8 text-muted-foreground" />
)}
</div>
<button
type="button"
onClick={() => avatarInputRef.current?.click()}
className="absolute inset-0 rounded-full bg-accent/60 opacity-0 group-hover:opacity-100 transition-opacity flex items-center justify-center cursor-pointer"
>
<Edit2 className="h-5 w-5 text-accent-foreground" />
</button>
{avatarPreview && (
<button
type="button"
onClick={handleRemoveAvatar}
disabled={deleteAvatar.isPending}
className="absolute bottom-0 right-0 h-5 w-5 rounded-full bg-background/60 backdrop-blur-sm text-muted-foreground flex items-center justify-center hover:bg-background/80 hover:text-foreground transition-colors shadow-sm border border-border/50"
>
<X className="h-3 w-3" />
</button>
)}
</div>
<input
ref={avatarInputRef}
type="file"
accept="image/png,image/jpeg,image/webp"
onChange={handleAvatarFileChange}
className="hidden"
/>
</div>
{/* Fields */}
<div className="space-y-3 px-5">
<FormField
control={form.control}
name="name"
render={({ field }) => (
<FormItem>
<FormLabel>Name</FormLabel>
<FormControl>
<Input placeholder="My Voice" {...field} />
</FormControl>
<FormMessage />
</FormItem>
)}
/>
<FormField
control={form.control}
name="description"
render={({ field }) => (
<FormItem>
<FormLabel>Description</FormLabel>
<FormControl>
<Textarea placeholder="Describe this voice..." rows={2} {...field} />
</FormControl>
<FormMessage />
</FormItem>
)}
/>
<FormField
control={form.control}
name="language"
render={({ field }) => (
<FormItem>
<FormLabel>Language</FormLabel>
<Select onValueChange={field.onChange} value={field.value}>
<FormControl>
<SelectTrigger>
<SelectValue />
</SelectTrigger>
</FormControl>
<SelectContent>
{LANGUAGE_OPTIONS.map((lang) => (
<SelectItem key={lang.value} value={lang.value}>
{lang.label}
</SelectItem>
))}
</SelectContent>
</Select>
<FormMessage />
</FormItem>
)}
/>
{/* Effects */}
<div className="space-y-2">
<FormLabel>Default Effects</FormLabel>
<p className="text-xs text-muted-foreground">
Applied automatically to new generations with this voice.
</p>
<EffectsChainEditor
value={effectsChain}
onChange={(chain) => {
setEffectsChain(chain);
setEffectsDirty(true);
}}
compact
/>
</div>
{/* Save */}
{isDirty && (
<Button type="submit" className="w-full" disabled={updateProfile.isPending}>
{updateProfile.isPending ? 'Saving...' : 'Save Changes'}
</Button>
)}
</div>
{/* Samples */}
<div className="px-5 pb-5">
<SampleList profileId={profileId} />
</div>
</form>
</Form>
</div>
</div>
);
}
+141 -124
View File
@@ -1,13 +1,9 @@
import { useQuery, useQueryClient } from '@tanstack/react-query'; import { useQuery, useQueryClient } from '@tanstack/react-query';
import { Edit, MoreHorizontal, Plus, Trash2, Mic } from 'lucide-react'; import { Mic, Plus, Search, Sparkles } from 'lucide-react';
import { useMemo, useRef } from 'react'; import { useEffect, useMemo, useRef, useState } from 'react';
import { Button } from '@/components/ui/button'; import { Button } from '@/components/ui/button';
import { import { Input } from '@/components/ui/input';
DropdownMenu,
DropdownMenuContent,
DropdownMenuItem,
DropdownMenuTrigger,
} from '@/components/ui/dropdown-menu';
import { MultiSelect } from '@/components/ui/multi-select'; import { MultiSelect } from '@/components/ui/multi-select';
import { import {
Table, Table,
@@ -21,33 +17,46 @@ import { ProfileForm } from '@/components/VoiceProfiles/ProfileForm';
import { apiClient } from '@/lib/api/client'; import { apiClient } from '@/lib/api/client';
import type { VoiceProfileResponse } from '@/lib/api/types'; import type { VoiceProfileResponse } from '@/lib/api/types';
import { BOTTOM_SAFE_AREA_PADDING } from '@/lib/constants/ui'; import { BOTTOM_SAFE_AREA_PADDING } from '@/lib/constants/ui';
import { useHistory } from '@/lib/hooks/useHistory'; import { useProfiles } from '@/lib/hooks/useProfiles';
import { useDeleteProfile, useProfileSamples, useProfiles } from '@/lib/hooks/useProfiles';
import { cn } from '@/lib/utils/cn'; import { cn } from '@/lib/utils/cn';
import { usePlayerStore } from '@/stores/playerStore'; import { usePlayerStore } from '@/stores/playerStore';
import { useServerStore } from '@/stores/serverStore';
import { useUIStore } from '@/stores/uiStore'; import { useUIStore } from '@/stores/uiStore';
import { VoiceInspector } from './VoiceInspector';
export function VoicesTab() { export function VoicesTab() {
const { data: profiles, isLoading } = useProfiles(); const { data: profiles, isLoading } = useProfiles();
const { data: historyData } = useHistory({ limit: 1000 });
const queryClient = useQueryClient(); const queryClient = useQueryClient();
const setDialogOpen = useUIStore((state) => state.setProfileDialogOpen); const setDialogOpen = useUIStore((state) => state.setProfileDialogOpen);
const setEditingProfileId = useUIStore((state) => state.setEditingProfileId); const selectedVoiceId = useUIStore((state) => state.selectedVoiceId);
const deleteProfile = useDeleteProfile(); const setSelectedVoiceId = useUIStore((state) => state.setSelectedVoiceId);
const scrollRef = useRef<HTMLDivElement>(null); const scrollRef = useRef<HTMLDivElement>(null);
const audioUrl = usePlayerStore((state) => state.audioUrl); const audioUrl = usePlayerStore((state) => state.audioUrl);
const isPlayerVisible = !!audioUrl; const isPlayerVisible = !!audioUrl;
const [search, setSearch] = useState('');
// Get generation counts per profile const filteredProfiles = useMemo(() => {
const generationCounts = useMemo(() => { if (!profiles) return [];
const counts: Record<string, number> = {}; if (!search.trim()) return profiles;
if (historyData?.items) { const q = search.toLowerCase();
historyData.items.forEach((item) => { return profiles.filter(
counts[item.profile_id] = (counts[item.profile_id] || 0) + 1; (p) =>
}); p.name.toLowerCase().includes(q) ||
p.description?.toLowerCase().includes(q) ||
p.language.toLowerCase().includes(q),
);
}, [profiles, search]);
// Auto-select first profile if none selected
useEffect(() => {
if (!selectedVoiceId && profiles && profiles.length > 0) {
setSelectedVoiceId(profiles[0].id);
} }
return counts; // Clear selection if selected profile was deleted
}, [historyData]); if (selectedVoiceId && profiles && !profiles.find((p) => p.id === selectedVoiceId)) {
setSelectedVoiceId(profiles.length > 0 ? profiles[0].id : null);
}
}, [profiles, selectedVoiceId, setSelectedVoiceId]);
// Get channel assignments for each profile // Get channel assignments for each profile
const { data: channelAssignments } = useQuery({ const { data: channelAssignments } = useQuery({
@@ -74,17 +83,6 @@ export function VoicesTab() {
queryFn: () => apiClient.listChannels(), queryFn: () => apiClient.listChannels(),
}); });
const handleEdit = (profileId: string) => {
setEditingProfileId(profileId);
setDialogOpen(true);
};
const handleProfileDelete = async (profileId: string) => {
if (await confirm('Are you sure you want to delete this profile?')) {
deleteProfile.mutate(profileId);
}
};
const handleChannelChange = async (profileId: string, channelIds: string[]) => { const handleChannelChange = async (profileId: string, channelIds: string[]) => {
try { try {
await apiClient.setProfileChannels(profileId, channelIds); await apiClient.setProfileChannels(profileId, channelIds);
@@ -103,56 +101,76 @@ export function VoicesTab() {
} }
return ( return (
<div className="h-full flex flex-col relative overflow-hidden"> <div className="h-full flex gap-0 overflow-hidden -mx-8">
{/* Scroll Mask - Always visible, behind content */} {/* Left: Table */}
<div className="absolute top-0 left-0 right-0 h-16 bg-gradient-to-b from-background to-transparent z-10 pointer-events-none" /> <div className="flex-1 min-w-0 flex flex-col relative overflow-hidden">
{/* Scroll Mask */}
<div className="absolute top-0 left-0 right-0 h-16 bg-gradient-to-b from-background to-transparent z-10 pointer-events-none" />
{/* Fixed Header */} {/* Fixed Header */}
<div className="absolute top-0 left-0 right-0 z-20"> <div className="absolute top-0 left-0 right-0 z-20 pl-8 pr-8">
<div className="flex items-center justify-between mb-6"> <div className="flex items-center gap-3 mb-6">
<h1 className="text-2xl font-bold">Voices</h1> <h1 className="text-2xl font-bold">Voices</h1>
<Button onClick={() => setDialogOpen(true)}> <div className="flex-1" />
<Plus className="h-4 w-4 mr-2" /> <div className="relative w-[240px]">
New Voice <Search className="absolute left-2.5 top-1/2 -translate-y-1/2 h-3.5 w-3.5 text-muted-foreground" />
</Button> <Input
placeholder="Search voices..."
value={search}
onChange={(e) => setSearch(e.target.value)}
className="h-10 pl-8 text-sm rounded-full focus-visible:ring-0 focus-visible:ring-offset-0"
/>
</div>
<Button onClick={() => setDialogOpen(true)}>
<Plus className="h-4 w-4 mr-2" />
New Voice
</Button>
</div>
</div>
{/* Scrollable Content */}
<div
ref={scrollRef}
className={cn(
'flex-1 overflow-y-auto overflow-x-hidden pt-16 relative z-0',
isPlayerVisible && BOTTOM_SAFE_AREA_PADDING,
)}
>
<Table className="table-fixed [&_td:first-child]:pl-8 [&_th:first-child]:pl-8">
<TableHeader>
<TableRow>
<TableHead className="w-[30%]">Name</TableHead>
<TableHead className="w-[10%]">Language</TableHead>
<TableHead className="w-[10%]">Generations</TableHead>
<TableHead className="w-[8%]">Samples</TableHead>
<TableHead className="w-[8%]">Effects</TableHead>
<TableHead className="w-[24%]">Channels</TableHead>
<TableHead className="w-6"></TableHead>
</TableRow>
</TableHeader>
<TableBody>
{filteredProfiles.map((profile) => (
<VoiceRow
key={profile.id}
profile={profile}
isSelected={selectedVoiceId === profile.id}
onSelect={() => setSelectedVoiceId(profile.id)}
channelIds={channelAssignments?.[profile.id] || []}
channels={channels || []}
onChannelChange={(channelIds) => handleChannelChange(profile.id, channelIds)}
/>
))}
</TableBody>
</Table>
</div> </div>
</div> </div>
{/* Scrollable Content */} {/* Right: Inspector */}
<div {selectedVoiceId && (
ref={scrollRef} <div className="w-[340px] shrink-0 border-l border-t rounded-tl-xl bg-muted/30">
className={cn( <VoiceInspector key={selectedVoiceId} profileId={selectedVoiceId} />
'flex-1 overflow-y-auto pt-16 relative z-0', </div>
isPlayerVisible && BOTTOM_SAFE_AREA_PADDING, )}
)}
>
<Table>
<TableHeader>
<TableRow>
<TableHead>Name</TableHead>
<TableHead>Language</TableHead>
<TableHead>Generations</TableHead>
<TableHead>Samples</TableHead>
<TableHead>Channels</TableHead>
<TableHead className="w-[50px]"></TableHead>
</TableRow>
</TableHeader>
<TableBody>
{profiles?.map((profile) => (
<VoiceRow
key={profile.id}
profile={profile}
generationCount={generationCounts[profile.id] || 0}
channelIds={channelAssignments?.[profile.id] || []}
channels={channels || []}
onChannelChange={(channelIds) => handleChannelChange(profile.id, channelIds)}
onEdit={() => handleEdit(profile.id)}
onDelete={() => handleProfileDelete(profile.id)}
/>
))}
</TableBody>
</Table>
</div>
<ProfileForm /> <ProfileForm />
</div> </div>
@@ -161,42 +179,46 @@ export function VoicesTab() {
interface VoiceRowProps { interface VoiceRowProps {
profile: VoiceProfileResponse; profile: VoiceProfileResponse;
generationCount: number; isSelected: boolean;
onSelect: () => void;
channelIds: string[]; channelIds: string[];
channels: Array<{ id: string; name: string; is_default: boolean }>; channels: Array<{ id: string; name: string; is_default: boolean }>;
onChannelChange: (channelIds: string[]) => void; onChannelChange: (channelIds: string[]) => void;
onEdit: () => void;
onDelete: () => void;
} }
function VoiceRow({ function VoiceRow({
profile, profile,
generationCount, isSelected,
onSelect,
channelIds, channelIds,
channels, channels,
onChannelChange, onChannelChange,
onEdit,
onDelete,
}: VoiceRowProps) { }: VoiceRowProps) {
const { data: samples } = useProfileSamples(profile.id); const serverUrl = useServerStore((state) => state.serverUrl);
const sampleCount = samples?.length || 0; const [avatarError, setAvatarError] = useState(false);
const avatarUrl = profile.avatar_path ? `${serverUrl}/profiles/${profile.id}/avatar` : null;
const rowLabel = `${profile.name}, ${profile.language}, ${generationCount} generations, ${sampleCount} samples. Press Enter to edit.`; const enabledEffects = profile.effects_chain?.filter((e) => e.enabled) ?? [];
const effectsSummary = enabledEffects.map((e) => e.type).join(' → ');
return ( return (
<TableRow className="cursor-pointer" onClick={onEdit}> <TableRow
className={cn('cursor-pointer', isSelected ? 'bg-muted/50' : 'hover:bg-muted/50')}
onClick={onSelect}
>
<TableCell> <TableCell>
<button <div className="flex w-full min-w-0 items-center gap-2">
type="button" <div className="h-8 w-8 rounded-full bg-muted flex items-center justify-center shrink-0 overflow-hidden">
className="flex w-full min-w-0 items-center gap-2 text-left focus:outline-none focus-visible:ring-2 focus-visible:ring-ring focus-visible:ring-offset-2 rounded" {avatarUrl && !avatarError ? (
aria-label={rowLabel} <img
onClick={(e) => { src={avatarUrl}
e.stopPropagation(); alt={`${profile.name} avatar`}
onEdit(); className="h-full w-full object-cover"
}} onError={() => setAvatarError(true)}
> />
<div className="h-8 w-8 rounded-lg bg-muted flex items-center justify-center shrink-0"> ) : (
<Mic className="h-4 w-4 text-muted-foreground" /> <Mic className="h-4 w-4 text-muted-foreground" />
)}
</div> </div>
<div className="min-w-0"> <div className="min-w-0">
<div className="font-medium truncate">{profile.name}</div> <div className="font-medium truncate">{profile.name}</div>
@@ -204,11 +226,24 @@ function VoiceRow({
<div className="text-sm text-muted-foreground truncate">{profile.description}</div> <div className="text-sm text-muted-foreground truncate">{profile.description}</div>
)} )}
</div> </div>
</button> </div>
</TableCell>
<TableCell>{profile.language}</TableCell>
<TableCell>{profile.generation_count}</TableCell>
<TableCell>{profile.sample_count}</TableCell>
<TableCell>
{enabledEffects.length > 0 ? (
<span
className="inline-flex items-center gap-1 text-xs text-accent"
title={effectsSummary}
>
<Sparkles className="h-3 w-3 fill-accent" />
{enabledEffects.length}
</span>
) : (
<span className="text-xs text-muted-foreground">—</span>
)}
</TableCell> </TableCell>
<TableCell onClick={(e) => e.stopPropagation()}>{profile.language}</TableCell>
<TableCell onClick={(e) => e.stopPropagation()}>{generationCount}</TableCell>
<TableCell onClick={(e) => e.stopPropagation()}>{sampleCount}</TableCell>
<TableCell onClick={(e) => e.stopPropagation()}> <TableCell onClick={(e) => e.stopPropagation()}>
<MultiSelect <MultiSelect
options={channels.map((ch) => ({ options={channels.map((ch) => ({
@@ -218,28 +253,10 @@ function VoiceRow({
value={channelIds} value={channelIds}
onChange={onChannelChange} onChange={onChannelChange}
placeholder="Select channels..." placeholder="Select channels..."
className="min-w-[200px]" className="w-full"
/> />
</TableCell> </TableCell>
<TableCell onClick={(e) => e.stopPropagation()}> <TableCell />
<DropdownMenu>
<DropdownMenuTrigger asChild>
<Button variant="ghost" size="icon" aria-label={`Actions for ${profile.name}`}>
<MoreHorizontal className="h-4 w-4" />
</Button>
</DropdownMenuTrigger>
<DropdownMenuContent>
<DropdownMenuItem onClick={onEdit}>
<Edit className="h-4 w-4 mr-2" />
Edit
</DropdownMenuItem>
<DropdownMenuItem onClick={onDelete} className="text-destructive">
<Trash2 className="h-4 w-4 mr-2" />
Delete
</DropdownMenuItem>
</DropdownMenuContent>
</DropdownMenu>
</TableCell>
</TableRow> </TableRow>
); );
} }
+5 -3
View File
@@ -1,6 +1,6 @@
import * as React from 'react';
import * as DropdownMenuPrimitive from '@radix-ui/react-dropdown-menu'; import * as DropdownMenuPrimitive from '@radix-ui/react-dropdown-menu';
import { MoreHorizontal } from 'lucide-react'; import { MoreHorizontal } from 'lucide-react';
import * as React from 'react';
import { cn } from '@/lib/utils/cn'; import { cn } from '@/lib/utils/cn';
const DropdownMenu = DropdownMenuPrimitive.Root; const DropdownMenu = DropdownMenuPrimitive.Root;
@@ -73,7 +73,7 @@ const DropdownMenuItem = React.forwardRef<
<DropdownMenuPrimitive.Item <DropdownMenuPrimitive.Item
ref={ref} ref={ref}
className={cn( className={cn(
'relative flex cursor-default select-none items-center rounded-sm px-2 py-1.5 text-sm outline-none transition-colors focus:bg-accent focus:text-accent-foreground data-[disabled]:pointer-events-none data-[disabled]:opacity-50', 'relative flex cursor-default select-none items-center rounded-sm px-2 py-1.5 text-sm outline-none focus:bg-accent focus:text-accent-foreground data-[disabled]:pointer-events-none data-[disabled]:opacity-50',
inset && 'pl-8', inset && 'pl-8',
className, className,
)} )}
@@ -154,7 +154,9 @@ const DropdownMenuSeparator = React.forwardRef<
DropdownMenuSeparator.displayName = DropdownMenuPrimitive.Separator.displayName; DropdownMenuSeparator.displayName = DropdownMenuPrimitive.Separator.displayName;
const DropdownMenuShortcut = ({ className, ...props }: React.HTMLAttributes<HTMLSpanElement>) => { const DropdownMenuShortcut = ({ className, ...props }: React.HTMLAttributes<HTMLSpanElement>) => {
return <span className={cn('ml-auto text-xs tracking-widest opacity-60', className)} {...props} />; return (
<span className={cn('ml-auto text-xs tracking-widest opacity-60', className)} {...props} />
);
}; };
DropdownMenuShortcut.displayName = 'DropdownMenuShortcut'; DropdownMenuShortcut.displayName = 'DropdownMenuShortcut';
+2 -2
View File
@@ -16,7 +16,7 @@ const SelectTrigger = React.forwardRef<
<SelectPrimitive.Trigger <SelectPrimitive.Trigger
ref={ref} ref={ref}
className={cn( className={cn(
'flex h-10 w-full items-center justify-between rounded-md border border-input bg-background px-3 py-2 text-sm ring-offset-background placeholder:text-muted-foreground focus:outline-none focus:ring-2 focus:ring-ring focus:ring-offset-2 disabled:cursor-not-allowed disabled:opacity-50 [&>span]:line-clamp-1', 'flex h-10 w-full items-center justify-between rounded-md border border-input bg-background px-3 py-2 text-sm ring-offset-background placeholder:text-muted-foreground focus:outline-none focus:bg-muted/50 disabled:cursor-not-allowed disabled:opacity-50 [&>span]:line-clamp-1',
className, className,
)} )}
{...props} {...props}
@@ -108,7 +108,7 @@ const SelectItem = React.forwardRef<
<SelectPrimitive.Item <SelectPrimitive.Item
ref={ref} ref={ref}
className={cn( className={cn(
'relative flex w-full cursor-default select-none items-center rounded-sm py-1.5 pl-8 pr-2 text-sm outline-none focus:bg-accent focus:text-accent-foreground data-[disabled]:pointer-events-none data-[disabled]:opacity-50', 'relative flex w-full cursor-default select-none items-center rounded-sm py-1.5 pl-8 pr-2 text-sm outline-none focus:bg-accent focus:text-accent-foreground focus:[&_*]:text-accent-foreground data-[disabled]:pointer-events-none data-[disabled]:opacity-50',
className, className,
)} )}
{...props} {...props}
+6 -5
View File
@@ -14,7 +14,11 @@ const TableHeader = React.forwardRef<
HTMLTableSectionElement, HTMLTableSectionElement,
React.HTMLAttributes<HTMLTableSectionElement> React.HTMLAttributes<HTMLTableSectionElement>
>(({ className, ...props }, ref) => ( >(({ className, ...props }, ref) => (
<thead ref={ref} className={cn('[&_tr]:border-b', className)} {...props} /> <thead
ref={ref}
className={cn('[&_tr]:border-b [&_tr]:hover:bg-transparent', className)}
{...props}
/>
)); ));
TableHeader.displayName = 'TableHeader'; TableHeader.displayName = 'TableHeader';
@@ -42,10 +46,7 @@ const TableRow = React.forwardRef<HTMLTableRowElement, React.HTMLAttributes<HTML
({ className, ...props }, ref) => ( ({ className, ...props }, ref) => (
<tr <tr
ref={ref} ref={ref}
className={cn( className={cn('border-b hover:bg-muted/50 data-[state=selected]:bg-muted', className)}
'border-b transition-colors hover:bg-muted/50 data-[state=selected]:bg-muted',
className,
)}
{...props} {...props}
/> />
), ),
+127
View File
@@ -2,9 +2,15 @@ import type { LanguageCode } from '@/lib/constants/languages';
import { useServerStore } from '@/stores/serverStore'; import { useServerStore } from '@/stores/serverStore';
import type { import type {
ActiveTasksResponse, ActiveTasksResponse,
ApplyEffectsRequest,
AvailableEffectsResponse,
CudaStatus, CudaStatus,
EffectConfig,
EffectPresetCreate,
EffectPresetResponse,
GenerationRequest, GenerationRequest,
GenerationResponse, GenerationResponse,
GenerationVersionResponse,
HealthResponse, HealthResponse,
HistoryListResponse, HistoryListResponse,
HistoryQuery, HistoryQuery,
@@ -21,6 +27,7 @@ import type {
StoryItemReorder, StoryItemReorder,
StoryItemSplit, StoryItemSplit,
StoryItemTrim, StoryItemTrim,
StoryItemVersionUpdate,
StoryResponse, StoryResponse,
TranscriptionResponse, TranscriptionResponse,
VoiceProfileCreate, VoiceProfileCreate,
@@ -206,6 +213,18 @@ class ApiClient {
}); });
} }
async regenerateGeneration(generationId: string): Promise<GenerationResponse> {
return this.request<GenerationResponse>(`/generate/${generationId}/regenerate`, {
method: 'POST',
});
}
async toggleFavorite(generationId: string): Promise<{ is_favorited: boolean }> {
return this.request<{ is_favorited: boolean }>(`/history/${generationId}/favorite`, {
method: 'POST',
});
}
// History // History
async listHistory(query?: HistoryQuery): Promise<HistoryListResponse> { async listHistory(query?: HistoryQuery): Promise<HistoryListResponse> {
const params = new URLSearchParams(); const params = new URLSearchParams();
@@ -570,6 +589,17 @@ class ApiClient {
}); });
} }
async setStoryItemVersion(
storyId: string,
itemId: string,
data: StoryItemVersionUpdate,
): Promise<StoryItemDetail> {
return this.request<StoryItemDetail>(`/stories/${storyId}/items/${itemId}/version`, {
method: 'PUT',
body: JSON.stringify(data),
});
}
async exportStoryAudio(storyId: string): Promise<Blob> { async exportStoryAudio(storyId: string): Promise<Blob> {
const url = `${this.getBaseUrl()}/stories/${storyId}/export-audio`; const url = `${this.getBaseUrl()}/stories/${storyId}/export-audio`;
const response = await fetch(url); const response = await fetch(url);
@@ -583,6 +613,103 @@ class ApiClient {
return response.blob(); return response.blob();
} }
// Effects & Versions
async getAvailableEffects(): Promise<AvailableEffectsResponse> {
return this.request<AvailableEffectsResponse>('/effects/available');
}
async listEffectPresets(): Promise<EffectPresetResponse[]> {
return this.request<EffectPresetResponse[]>('/effects/presets');
}
async createEffectPreset(data: EffectPresetCreate): Promise<EffectPresetResponse> {
return this.request<EffectPresetResponse>('/effects/presets', {
method: 'POST',
body: JSON.stringify(data),
});
}
async updateEffectPreset(
presetId: string,
data: { name?: string; description?: string; effects_chain?: EffectConfig[] },
): Promise<EffectPresetResponse> {
return this.request<EffectPresetResponse>(`/effects/presets/${presetId}`, {
method: 'PUT',
body: JSON.stringify(data),
});
}
async deleteEffectPreset(presetId: string): Promise<void> {
await this.request<void>(`/effects/presets/${presetId}`, {
method: 'DELETE',
});
}
async listGenerationVersions(generationId: string): Promise<GenerationVersionResponse[]> {
return this.request<GenerationVersionResponse[]>(`/generations/${generationId}/versions`);
}
async applyEffectsToGeneration(
generationId: string,
data: ApplyEffectsRequest,
): Promise<GenerationVersionResponse> {
return this.request<GenerationVersionResponse>(
`/generations/${generationId}/versions/apply-effects`,
{
method: 'POST',
body: JSON.stringify(data),
},
);
}
async setDefaultVersion(
generationId: string,
versionId: string,
): Promise<GenerationVersionResponse> {
return this.request<GenerationVersionResponse>(
`/generations/${generationId}/versions/${versionId}/set-default`,
{ method: 'PUT' },
);
}
async deleteGenerationVersion(generationId: string, versionId: string): Promise<void> {
await this.request<void>(`/generations/${generationId}/versions/${versionId}`, {
method: 'DELETE',
});
}
getVersionAudioUrl(versionId: string): string {
return `${this.getBaseUrl()}/audio/version/${versionId}`;
}
async updateProfileEffects(
profileId: string,
effectsChain: EffectConfig[] | null,
): Promise<VoiceProfileResponse> {
return this.request<VoiceProfileResponse>(`/profiles/${profileId}/effects`, {
method: 'PUT',
body: JSON.stringify({ effects_chain: effectsChain }),
});
}
async previewEffects(generationId: string, effectsChain: EffectConfig[]): Promise<Blob> {
const url = `${this.getBaseUrl()}/effects/preview/${generationId}`;
const response = await fetch(url, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ effects_chain: effectsChain }),
});
if (!response.ok) {
const error = await response.json().catch(() => ({
detail: response.statusText,
}));
throw new Error(error.detail || `HTTP error! status: ${response.status}`);
}
return response.blob();
}
} }
export const apiClient = new ApiClient(); export const apiClient = new ApiClient();
+83 -1
View File
@@ -13,6 +13,9 @@ export interface VoiceProfileResponse {
description?: string; description?: string;
language: string; language: string;
avatar_path?: string; avatar_path?: string;
effects_chain?: EffectConfig[];
generation_count: number;
sample_count: number;
created_at: string; created_at: string;
updated_at: string; updated_at: string;
} }
@@ -28,6 +31,12 @@ export interface ProfileSampleResponse {
reference_text: string; reference_text: string;
} }
export interface EffectConfig {
type: string;
enabled: boolean;
params: Record<string, number>;
}
export interface GenerationRequest { export interface GenerationRequest {
profile_id: string; profile_id: string;
text: string; text: string;
@@ -39,6 +48,18 @@ export interface GenerationRequest {
max_chunk_chars?: number; max_chunk_chars?: number;
crossfade_ms?: number; crossfade_ms?: number;
normalize?: boolean; normalize?: boolean;
effects_chain?: EffectConfig[];
}
export interface GenerationVersionResponse {
id: string;
generation_id: string;
label: string;
audio_path: string;
effects_chain?: EffectConfig[];
source_version_id?: string;
is_default: boolean;
created_at: string;
} }
export interface GenerationResponse { export interface GenerationResponse {
@@ -52,9 +73,12 @@ export interface GenerationResponse {
instruct?: string; instruct?: string;
engine?: string; engine?: string;
model_size?: string; model_size?: string;
status: 'generating' | 'completed' | 'failed'; status: 'loading_model' | 'generating' | 'completed' | 'failed';
error?: string; error?: string;
is_favorited?: boolean;
created_at: string; created_at: string;
versions?: GenerationVersionResponse[];
active_version_id?: string;
} }
export interface HistoryQuery { export interface HistoryQuery {
@@ -66,6 +90,8 @@ export interface HistoryQuery {
export interface HistoryResponse extends GenerationResponse { export interface HistoryResponse extends GenerationResponse {
profile_name: string; profile_name: string;
versions?: GenerationVersionResponse[];
active_version_id?: string;
} }
export interface HistoryListResponse { export interface HistoryListResponse {
@@ -199,6 +225,7 @@ export interface StoryItemDetail {
id: string; id: string;
story_id: string; story_id: string;
generation_id: string; generation_id: string;
version_id?: string;
start_time_ms: number; start_time_ms: number;
track: number; track: number;
trim_start_ms: number; trim_start_ms: number;
@@ -213,6 +240,12 @@ export interface StoryItemDetail {
seed?: number; seed?: number;
instruct?: string; instruct?: string;
generation_created_at: string; generation_created_at: string;
versions?: GenerationVersionResponse[];
active_version_id?: string;
}
export interface StoryItemVersionUpdate {
version_id: string | null;
} }
export interface StoryDetailResponse { export interface StoryDetailResponse {
@@ -256,3 +289,52 @@ export interface StoryItemTrim {
export interface StoryItemSplit { export interface StoryItemSplit {
split_time_ms: number; split_time_ms: number;
} }
// Effects
export interface EffectPresetResponse {
id: string;
name: string;
description?: string;
effects_chain: EffectConfig[];
is_builtin: boolean;
created_at: string;
}
export interface EffectPresetCreate {
name: string;
description?: string;
effects_chain: EffectConfig[];
}
export interface EffectPresetUpdate {
name?: string;
description?: string;
effects_chain?: EffectConfig[];
}
export interface AvailableEffectParam {
default: number;
min: number;
max: number;
step: number;
description: string;
}
export interface AvailableEffect {
type: string;
label: string;
description: string;
params: Record<string, AvailableEffectParam>;
}
export interface AvailableEffectsResponse {
effects: AvailableEffect[];
}
export interface ApplyEffectsRequest {
effects_chain: EffectConfig[];
source_version_id?: string;
label?: string;
set_as_default?: boolean;
}
+5 -2
View File
@@ -2,11 +2,14 @@
* UI layout constants for safe area padding * UI layout constants for safe area padding
*/ */
const isWindows = typeof navigator !== 'undefined' && navigator.userAgent.includes('Windows');
/** /**
* Top safe area padding - height of the drag region bar * Top safe area padding - height of the drag region bar
* Corresponds to Tailwind's pt-12 (3rem / 48px) * On macOS this accounts for the overlay titlebar (48px).
* On Windows the native title bar is outside the webview, so no padding is needed.
*/ */
export const TOP_SAFE_AREA_PADDING = 'pt-12'; export const TOP_SAFE_AREA_PADDING = isWindows ? 'pt-8' : 'pt-12';
/** /**
* Bottom safe area padding - height of the audio player * Bottom safe area padding - height of the audio player
+5 -1
View File
@@ -4,6 +4,7 @@ import { useForm } from 'react-hook-form';
import * as z from 'zod'; import * as z from 'zod';
import { useToast } from '@/components/ui/use-toast'; import { useToast } from '@/components/ui/use-toast';
import { apiClient } from '@/lib/api/client'; import { apiClient } from '@/lib/api/client';
import type { EffectConfig } from '@/lib/api/types';
import { LANGUAGE_CODES, type LanguageCode } from '@/lib/constants/languages'; import { LANGUAGE_CODES, type LanguageCode } from '@/lib/constants/languages';
import { useGeneration } from '@/lib/hooks/useGeneration'; import { useGeneration } from '@/lib/hooks/useGeneration';
import { useModelDownloadToast } from '@/lib/hooks/useModelDownloadToast'; import { useModelDownloadToast } from '@/lib/hooks/useModelDownloadToast';
@@ -11,7 +12,7 @@ import { useGenerationStore } from '@/stores/generationStore';
import { useServerStore } from '@/stores/serverStore'; import { useServerStore } from '@/stores/serverStore';
const generationSchema = z.object({ const generationSchema = z.object({
text: z.string().min(1, 'Text is required').max(50000), text: z.string().min(1, '').max(50000),
language: z.enum(LANGUAGE_CODES as [LanguageCode, ...LanguageCode[]]), language: z.enum(LANGUAGE_CODES as [LanguageCode, ...LanguageCode[]]),
seed: z.number().int().optional(), seed: z.number().int().optional(),
modelSize: z.enum(['1.7B', '0.6B']).optional(), modelSize: z.enum(['1.7B', '0.6B']).optional(),
@@ -24,6 +25,7 @@ export type GenerationFormValues = z.infer<typeof generationSchema>;
interface UseGenerationFormOptions { interface UseGenerationFormOptions {
onSuccess?: (generationId: string) => void; onSuccess?: (generationId: string) => void;
defaultValues?: Partial<GenerationFormValues>; defaultValues?: Partial<GenerationFormValues>;
getEffectsChain?: () => EffectConfig[] | undefined;
} }
export function useGenerationForm(options: UseGenerationFormOptions = {}) { export function useGenerationForm(options: UseGenerationFormOptions = {}) {
@@ -103,6 +105,7 @@ export function useGenerationForm(options: UseGenerationFormOptions = {}) {
} }
const isQwen = engine === 'qwen'; const isQwen = engine === 'qwen';
const effectsChain = options.getEffectsChain?.();
// This now returns immediately with status="generating" // This now returns immediately with status="generating"
const result = await generation.mutateAsync({ const result = await generation.mutateAsync({
profile_id: selectedProfileId, profile_id: selectedProfileId,
@@ -115,6 +118,7 @@ export function useGenerationForm(options: UseGenerationFormOptions = {}) {
max_chunk_chars: maxChunkChars, max_chunk_chars: maxChunkChars,
crossfade_ms: crossfadeMs, crossfade_ms: crossfadeMs,
normalize: normalizeAudio, normalize: normalizeAudio,
effects_chain: effectsChain?.length ? effectsChain : undefined,
}); });
// Track this generation for SSE status updates // Track this generation for SSE status updates
+1 -1
View File
@@ -8,7 +8,7 @@ import { useServerStore } from '@/stores/serverStore';
interface GenerationStatusEvent { interface GenerationStatusEvent {
id: string; id: string;
status: 'generating' | 'completed' | 'failed' | 'not_found'; status: 'loading_model' | 'generating' | 'completed' | 'failed' | 'not_found';
duration?: number; duration?: number;
error?: string; error?: string;
} }
+61 -8
View File
@@ -1,6 +1,15 @@
import { useMutation, useQuery, useQueryClient } from '@tanstack/react-query'; import { useMutation, useQuery, useQueryClient } from '@tanstack/react-query';
import { apiClient } from '@/lib/api/client'; import { apiClient } from '@/lib/api/client';
import type { StoryCreate, StoryItemCreate, StoryItemBatchUpdate, StoryItemReorder, StoryItemMove, StoryItemTrim, StoryItemSplit } from '@/lib/api/types'; import type {
StoryCreate,
StoryItemBatchUpdate,
StoryItemCreate,
StoryItemMove,
StoryItemReorder,
StoryItemSplit,
StoryItemTrim,
StoryItemVersionUpdate,
} from '@/lib/api/types';
import { usePlatform } from '@/platform/PlatformContext'; import { usePlatform } from '@/platform/PlatformContext';
export function useStories() { export function useStories() {
@@ -109,8 +118,15 @@ export function useMoveStoryItem() {
const queryClient = useQueryClient(); const queryClient = useQueryClient();
return useMutation({ return useMutation({
mutationFn: ({ storyId, itemId, data }: { storyId: string; itemId: string; data: StoryItemMove }) => mutationFn: ({
apiClient.moveStoryItem(storyId, itemId, data), storyId,
itemId,
data,
}: {
storyId: string;
itemId: string;
data: StoryItemMove;
}) => apiClient.moveStoryItem(storyId, itemId, data),
onSuccess: (_, variables) => { onSuccess: (_, variables) => {
queryClient.invalidateQueries({ queryKey: ['stories'] }); queryClient.invalidateQueries({ queryKey: ['stories'] });
queryClient.invalidateQueries({ queryKey: ['stories', variables.storyId] }); queryClient.invalidateQueries({ queryKey: ['stories', variables.storyId] });
@@ -122,8 +138,15 @@ export function useTrimStoryItem() {
const queryClient = useQueryClient(); const queryClient = useQueryClient();
return useMutation({ return useMutation({
mutationFn: ({ storyId, itemId, data }: { storyId: string; itemId: string; data: StoryItemTrim }) => mutationFn: ({
apiClient.trimStoryItem(storyId, itemId, data), storyId,
itemId,
data,
}: {
storyId: string;
itemId: string;
data: StoryItemTrim;
}) => apiClient.trimStoryItem(storyId, itemId, data),
onSuccess: (_, variables) => { onSuccess: (_, variables) => {
queryClient.invalidateQueries({ queryKey: ['stories'] }); queryClient.invalidateQueries({ queryKey: ['stories'] });
queryClient.invalidateQueries({ queryKey: ['stories', variables.storyId] }); queryClient.invalidateQueries({ queryKey: ['stories', variables.storyId] });
@@ -135,8 +158,15 @@ export function useSplitStoryItem() {
const queryClient = useQueryClient(); const queryClient = useQueryClient();
return useMutation({ return useMutation({
mutationFn: ({ storyId, itemId, data }: { storyId: string; itemId: string; data: StoryItemSplit }) => mutationFn: ({
apiClient.splitStoryItem(storyId, itemId, data), storyId,
itemId,
data,
}: {
storyId: string;
itemId: string;
data: StoryItemSplit;
}) => apiClient.splitStoryItem(storyId, itemId, data),
onSuccess: (_, variables) => { onSuccess: (_, variables) => {
queryClient.invalidateQueries({ queryKey: ['stories'] }); queryClient.invalidateQueries({ queryKey: ['stories'] });
queryClient.invalidateQueries({ queryKey: ['stories', variables.storyId] }); queryClient.invalidateQueries({ queryKey: ['stories', variables.storyId] });
@@ -157,6 +187,26 @@ export function useDuplicateStoryItem() {
}); });
} }
export function useSetStoryItemVersion() {
const queryClient = useQueryClient();
return useMutation({
mutationFn: ({
storyId,
itemId,
data,
}: {
storyId: string;
itemId: string;
data: StoryItemVersionUpdate;
}) => apiClient.setStoryItemVersion(storyId, itemId, data),
onSuccess: (_, variables) => {
queryClient.invalidateQueries({ queryKey: ['stories'] });
queryClient.invalidateQueries({ queryKey: ['stories', variables.storyId] });
},
});
}
export function useExportStoryAudio() { export function useExportStoryAudio() {
const platform = usePlatform(); const platform = usePlatform();
@@ -165,7 +215,10 @@ export function useExportStoryAudio() {
const blob = await apiClient.exportStoryAudio(storyId); const blob = await apiClient.exportStoryAudio(storyId);
// Create safe filename // Create safe filename
const safeName = storyName.substring(0, 50).replace(/[^a-z0-9]/gi, '-').toLowerCase(); const safeName = storyName
.substring(0, 50)
.replace(/[^a-z0-9]/gi, '-')
.toLowerCase();
const filename = `${safeName || 'story'}.wav`; const filename = `${safeName || 'story'}.wav`;
await platform.filesystem.saveFile(filename, blob, [ await platform.filesystem.saveFile(filename, blob, [
+23 -11
View File
@@ -70,6 +70,16 @@ export function useStoryPlayback(items: StoryItemDetail[] | undefined) {
} }
}, []); }, []);
// Resolve the audio buffer key and URL for an item.
// When a version_id is pinned, use that version's audio; otherwise use the generation default.
const getAudioKey = (item: StoryItemDetail) =>
item.version_id ? `v:${item.version_id}` : item.generation_id;
const getAudioUrlForItem = (item: StoryItemDetail) =>
item.version_id
? apiClient.getVersionAudioUrl(item.version_id)
: apiClient.getAudioUrl(item.generation_id);
// Preload audio files as AudioBuffers // Preload audio files as AudioBuffers
useEffect(() => { useEffect(() => {
if (!items || items.length === 0) { if (!items || items.length === 0) {
@@ -78,12 +88,12 @@ export function useStoryPlayback(items: StoryItemDetail[] | undefined) {
return; return;
} }
const currentIds = new Set(items.map((item) => item.generation_id)); const currentKeys = new Set(items.map(getAudioKey));
const audioContext = getAudioContext(); const audioContext = getAudioContext();
// Remove buffers for items that no longer exist // Remove buffers for items that no longer exist
for (const [id] of audioBuffersRef.current) { for (const [id] of audioBuffersRef.current) {
if (!currentIds.has(id)) { if (!currentKeys.has(id)) {
audioBuffersRef.current.delete(id); audioBuffersRef.current.delete(id);
} }
} }
@@ -91,24 +101,25 @@ export function useStoryPlayback(items: StoryItemDetail[] | undefined) {
// Preload audio for new items // Preload audio for new items
const preloadPromises: Promise<void>[] = []; const preloadPromises: Promise<void>[] = [];
for (const item of items) { for (const item of items) {
if (!audioBuffersRef.current.has(item.generation_id)) { const key = getAudioKey(item);
const audioUrl = apiClient.getAudioUrl(item.generation_id); if (!audioBuffersRef.current.has(key)) {
console.log('[StoryPlayback] Preloading audio buffer:', item.generation_id); const audioUrl = getAudioUrlForItem(item);
console.log('[StoryPlayback] Preloading audio buffer:', key);
const preloadPromise = fetch(audioUrl) const preloadPromise = fetch(audioUrl)
.then((response) => response.arrayBuffer()) .then((response) => response.arrayBuffer())
.then((arrayBuffer) => audioContext.decodeAudioData(arrayBuffer)) .then((arrayBuffer) => audioContext.decodeAudioData(arrayBuffer))
.then((audioBuffer) => { .then((audioBuffer) => {
audioBuffersRef.current.set(item.generation_id, audioBuffer); audioBuffersRef.current.set(key, audioBuffer);
console.log( console.log(
'[StoryPlayback] Preloaded buffer:', '[StoryPlayback] Preloaded buffer:',
item.generation_id, key,
'duration:', 'duration:',
audioBuffer.duration, audioBuffer.duration,
); );
}) })
.catch((err) => { .catch((err) => {
console.error('[StoryPlayback] Failed to preload audio:', item.generation_id, err); console.error('[StoryPlayback] Failed to preload audio:', key, err);
}); });
preloadPromises.push(preloadPromise); preloadPromises.push(preloadPromise);
@@ -216,15 +227,16 @@ export function useStoryPlayback(items: StoryItemDetail[] | undefined) {
// Schedule new sources for items that should be playing // Schedule new sources for items that should be playing
for (const item of shouldBePlaying) { for (const item of shouldBePlaying) {
if (!activeSourcesRef.current.has(item.id)) { if (!activeSourcesRef.current.has(item.id)) {
const buffer = audioBuffersRef.current.get(item.generation_id); const bufferKey = getAudioKey(item);
const buffer = audioBuffersRef.current.get(bufferKey);
if (!buffer) { if (!buffer) {
console.warn('[StoryPlayback] Buffer not loaded for:', item.generation_id); console.warn('[StoryPlayback] Buffer not loaded for:', bufferKey);
continue; continue;
} }
// Calculate when this item should start in AudioContext time // Calculate when this item should start in AudioContext time
const itemStartContextTime = storyTimeToContextTime(item.start_time_ms); const itemStartContextTime = storyTimeToContextTime(item.start_time_ms);
// Calculate effective duration and trim offsets // Calculate effective duration and trim offsets
const trimStartSec = (item.trim_start_ms || 0) / 1000; const trimStartSec = (item.trim_start_ms || 0) / 1000;
const trimEndSec = (item.trim_end_ms || 0) / 1000; const trimEndSec = (item.trim_end_ms || 0) / 1000;
+2
View File
@@ -10,6 +10,8 @@ export interface FileFilter {
export interface PlatformFilesystem { export interface PlatformFilesystem {
saveFile(filename: string, blob: Blob, filters?: FileFilter[]): Promise<void>; saveFile(filename: string, blob: Blob, filters?: FileFilter[]): Promise<void>;
openPath(path: string): Promise<void>;
pickDirectory(title: string): Promise<string | null>;
} }
export interface UpdateStatus { export interface UpdateStatus {
+9
View File
@@ -1,6 +1,7 @@
import { createRootRoute, createRoute, createRouter, Outlet } from '@tanstack/react-router'; import { createRootRoute, createRoute, createRouter, Outlet } from '@tanstack/react-router';
import { AppFrame } from '@/components/AppFrame/AppFrame'; import { AppFrame } from '@/components/AppFrame/AppFrame';
import { AudioTab } from '@/components/AudioTab/AudioTab'; import { AudioTab } from '@/components/AudioTab/AudioTab';
import { EffectsTab } from '@/components/EffectsTab/EffectsTab';
import { MainEditor } from '@/components/MainEditor/MainEditor'; import { MainEditor } from '@/components/MainEditor/MainEditor';
import { ModelsTab } from '@/components/ModelsTab/ModelsTab'; import { ModelsTab } from '@/components/ModelsTab/ModelsTab';
import { ServerTab } from '@/components/ServerTab/ServerTab'; import { ServerTab } from '@/components/ServerTab/ServerTab';
@@ -105,6 +106,13 @@ const audioRoute = createRoute({
component: AudioTab, component: AudioTab,
}); });
// Effects route
const effectsRoute = createRoute({
getParentRoute: () => rootRoute,
path: '/effects',
component: EffectsTab,
});
// Models route // Models route
const modelsRoute = createRoute({ const modelsRoute = createRoute({
getParentRoute: () => rootRoute, getParentRoute: () => rootRoute,
@@ -125,6 +133,7 @@ const routeTree = rootRoute.addChildren([
storiesRoute, storiesRoute,
voicesRoute, voicesRoute,
audioRoute, audioRoute,
effectsRoute,
modelsRoute, modelsRoute,
serverRoute, serverRoute,
]); ]);
+26
View File
@@ -0,0 +1,26 @@
import { create } from 'zustand';
import type { EffectConfig } from '@/lib/api/types';
interface EffectsStore {
selectedPresetId: string | null;
setSelectedPresetId: (id: string | null) => void;
// Working chain for the detail panel (editing a preset or building a new one)
workingChain: EffectConfig[];
setWorkingChain: (chain: EffectConfig[]) => void;
// Track if editing an existing preset vs creating new
isCreatingNew: boolean;
setIsCreatingNew: (v: boolean) => void;
}
export const useEffectsStore = create<EffectsStore>((set) => ({
selectedPresetId: null,
setSelectedPresetId: (id) => set({ selectedPresetId: id, isCreatingNew: false }),
workingChain: [],
setWorkingChain: (chain) => set({ workingChain: chain }),
isCreatingNew: false,
setIsCreatingNew: (v) => set({ isCreatingNew: v, ...(v && { selectedPresetId: null }) }),
}));
+7
View File
@@ -31,6 +31,10 @@ interface UIStore {
selectedProfileId: string | null; selectedProfileId: string | null;
setSelectedProfileId: (id: string | null) => void; setSelectedProfileId: (id: string | null) => void;
// Selected voice in Voices tab inspector
selectedVoiceId: string | null;
setSelectedVoiceId: (id: string | null) => void;
// Profile form draft (for persisting create voice modal state) // Profile form draft (for persisting create voice modal state)
profileFormDraft: ProfileFormDraft | null; profileFormDraft: ProfileFormDraft | null;
setProfileFormDraft: (draft: ProfileFormDraft | null) => void; setProfileFormDraft: (draft: ProfileFormDraft | null) => void;
@@ -55,6 +59,9 @@ export const useUIStore = create<UIStore>((set) => ({
selectedProfileId: null, selectedProfileId: null,
setSelectedProfileId: (id) => set({ selectedProfileId: id }), setSelectedProfileId: (id) => set({ selectedProfileId: id }),
selectedVoiceId: null,
setSelectedVoiceId: (id) => set({ selectedVoiceId: id }),
profileFormDraft: null, profileFormDraft: null,
setProfileFormDraft: (draft) => set({ profileFormDraft: draft }), setProfileFormDraft: (draft) => set({ profileFormDraft: draft }),
+107 -434
View File
@@ -1,462 +1,135 @@
# voicebox Backend # Voicebox Backend
Production-quality FastAPI backend for Qwen3-TTS voice cloning. FastAPI server powering voice cloning, speech generation, and audio processing. Runs locally as a Tauri sidecar or standalone via `python -m backend.main`.
## Features ## Running
- ✅ **Voice Profile Management** - Create, update, delete voice profiles with multi-sample support ```bash
- ✅ **Voice Cloning** - Generate speech using voice profiles with caching # Via justfile (recommended)
- ✅ **Generation History** - Full history tracking with search and filtering just dev:server
- ✅ **Transcription** - Whisper-based audio transcription
- ✅ **Multi-Sample Profiles** - Combine multiple reference samples for better quality # Standalone
- ✅ **Voice Prompt Caching** - Dual memory + disk caching for fast generation python -m backend.main --host 127.0.0.1 --port 17493
- ✅ **Audio Validation** - Automatic validation of reference audio quality
- ✅ **Model Management** - Lazy loading and VRAM management # With custom data directory
python -m backend.main --data-dir /path/to/data
```
The server auto-initializes the SQLite database on first startup. Models are downloaded from HuggingFace on first use.
## Architecture ## Architecture
``` ```
backend/ backend/
├── main.py # FastAPI app with all routes app.py # FastAPI app factory, CORS, lifecycle events
├── models.py # Pydantic request/response models main.py # Entry point (imports app, runs uvicorn)
├── platform_detect.py # Platform detection for backend selection config.py # Data directory paths and configuration
├── tts.py # TTS backend abstraction (delegates to MLX or PyTorch) models.py # Pydantic request/response schemas
├── transcribe.py # STT backend abstraction (delegates to MLX or PyTorch) server.py # Tauri sidecar launcher, parent-pid watchdog
├── backends/ # Backend implementations
│ ├── __init__.py # Backend factory and protocols routes/ # Thin HTTP handlers — validation, delegation, response formatting
│ ├── mlx_backend.py # MLX backend (Apple Silicon) services/ # Business logic, CRUD, orchestration
│ └── pytorch_backend.py # PyTorch backend (Windows/Linux/Intel) backends/ # TTS/STT engine implementations (MLX, PyTorch, etc.)
├── profiles.py # Voice profile CRUD database/ # ORM models, session management, migrations, seed data
├── history.py # Generation history utils/ # Shared utilities (audio, effects, caching, progress tracking)
├── studio.py # Audio editing (TODO)
├── database.py # SQLite ORM
└── utils/
├── audio.py # Audio processing utilities
├── cache.py # Voice prompt caching
└── validation.py # Input validation
``` ```
### Backend Selection ### Request flow
Voicebox automatically selects the best backend based on platform:
- **Apple Silicon (M1/M2/M3)**: Uses MLX backend with native Metal acceleration (4-5x faster)
- **Windows/Linux/Intel Mac**: Uses PyTorch backend (CUDA GPU if available, CPU fallback)
The backend is detected at runtime via `platform_detect.py`. Both backends implement the same interface, so the API remains consistent across platforms.
## API Endpoints
### Health & Info
#### `GET /`
Root endpoint with version info.
#### `GET /health`
Health check with model status.
**Response:**
```json
{
"status": "healthy",
"model_loaded": true,
"gpu_available": true,
"gpu_type": "Metal (Apple Silicon via MLX)",
"backend_type": "mlx",
"vram_used_mb": null
}
```
**Backend Types:**
- `"mlx"` - MLX backend (Apple Silicon with Metal acceleration)
- `"pytorch"` - PyTorch backend (Windows/Linux/Intel Mac)
### Voice Profiles
**Note:** The database is automatically initialized when the server starts. No manual setup required.
#### `POST /profiles`
Create a new voice profile.
**Request:**
```json
{
"name": "My Voice",
"description": "Optional description",
"language": "en"
}
```
**Response:**
```json
{
"id": "uuid",
"name": "My Voice",
"description": "Optional description",
"language": "en",
"created_at": "2024-01-01T00:00:00Z",
"updated_at": "2024-01-01T00:00:00Z"
}
```
#### `GET /profiles`
List all voice profiles.
#### `GET /profiles/{profile_id}`
Get a specific profile.
#### `PUT /profiles/{profile_id}`
Update a profile.
#### `DELETE /profiles/{profile_id}`
Delete a profile and all associated samples.
#### `POST /profiles/{profile_id}/samples`
Add a sample to a profile.
**Form Data:**
- `file`: Audio file (WAV, MP3, etc.)
- `reference_text`: Transcript of the audio
**Response:**
```json
{
"id": "sample-uuid",
"profile_id": "profile-uuid",
"audio_path": "/path/to/sample.wav",
"reference_text": "This is my voice"
}
```
#### `GET /profiles/{profile_id}/samples`
List all samples for a profile.
#### `DELETE /profiles/samples/{sample_id}`
Delete a specific sample.
### Generation
#### `POST /generate`
Generate speech from text using a voice profile.
**Request:**
```json
{
"profile_id": "uuid",
"text": "Hello, this is a test.",
"language": "en",
"seed": 42
}
```
**Response:**
```json
{
"id": "generation-uuid",
"profile_id": "profile-uuid",
"text": "Hello, this is a test.",
"language": "en",
"audio_path": "/path/to/audio.wav",
"duration": 2.5,
"seed": 42,
"created_at": "2024-01-01T00:00:00Z"
}
```
### History
#### `GET /history`
List generation history with optional filters.
**Query Parameters:**
- `profile_id` (optional): Filter by profile
- `search` (optional): Search in text content
- `limit` (default: 50): Results per page
- `offset` (default: 0): Pagination offset
#### `GET /history/{generation_id}`
Get a specific generation.
#### `DELETE /history/{generation_id}`
Delete a generation.
#### `GET /history/stats`
Get generation statistics.
**Response:**
```json
{
"total_generations": 100,
"total_duration_seconds": 250.5,
"generations_by_profile": {
"profile-uuid-1": 50,
"profile-uuid-2": 50
}
}
```
### Audio Files
#### `GET /audio/{generation_id}`
Download generated audio file.
Returns WAV file with appropriate headers.
### Transcription
#### `POST /transcribe`
Transcribe audio file to text.
**Form Data:**
- `file`: Audio file
- `language` (optional): Language hint (en or zh)
**Response:**
```json
{
"text": "Transcribed text here",
"duration": 5.5
}
```
### Model Management
#### `POST /models/load`
Manually load TTS model.
**Query Parameters:**
- `model_size`: Model size (1.7B or 0.6B)
#### `POST /models/unload`
Unload TTS model to free memory.
## Database Schema
### profiles
- `id`: UUID primary key
- `name`: Profile name (unique)
- `description`: Optional description
- `language`: Language code (en/zh)
- `created_at`: Creation timestamp
- `updated_at`: Last update timestamp
### profile_samples
- `id`: UUID primary key
- `profile_id`: Foreign key to profiles
- `audio_path`: Path to audio file
- `reference_text`: Transcript
### generations
- `id`: UUID primary key
- `profile_id`: Foreign key to profiles
- `text`: Generated text
- `language`: Language code
- `audio_path`: Path to audio file
- `duration`: Duration in seconds
- `seed`: Random seed (optional)
- `created_at`: Creation timestamp
### projects
- `id`: UUID primary key
- `name`: Project name
- `data`: JSON data
- `created_at`: Creation timestamp
- `updated_at`: Last update timestamp
## File Structure
``` ```
data/ HTTP request
├── profiles/ -> routes/ (validate input, parse params)
│ └── {profile_id}/ -> services/ (business logic, database queries, orchestration)
│ ├── {sample_id}.wav -> backends/ (TTS/STT inference)
│ └── ... -> utils/ (audio processing, effects, caching)
├── generations/
│ └── {generation_id}.wav
├── cache/
│ └── {hash}.prompt
├── projects/
│ └── {project_id}.json
└── voicebox.db
``` ```
## Setup Route handlers are intentionally thin. They validate input, delegate to a service function, and format the response. All business logic lives in `services/`.
### 1. Install Dependencies ### Key modules
```bash **services/generation.py** -- Single `run_generation()` function that handles all three generation modes (generate, retry, regenerate). Manages model loading, voice prompt creation, chunked inference, normalization, effects, and version persistence.
pip install -r requirements.txt
``` **services/task_queue.py** -- Serial generation queue. Ensures only one GPU inference runs at a time. Background tasks are tracked to prevent garbage collection.
**Note:** On Apple Silicon, also install MLX dependencies for faster inference: **backends/__init__.py** -- Protocol definitions (`TTSBackend`, `STTBackend`), model config registry, and factory functions. Adding a new engine means implementing the protocol and registering a config entry.
```bash
pip install -r requirements-mlx.txt **backends/base.py** -- Shared utilities used across all engine implementations: HuggingFace cache checks, device detection, voice prompt combination, progress tracking.
```
**database/** -- SQLAlchemy ORM models with a re-exporting `__init__.py` for backward compatibility. Migrations run automatically on startup.
### 2. Download Models (Automatic)
### Backend selection
The Qwen3-TTS models are automatically downloaded from HuggingFace Hub on first use, similar to how Whisper models work.
The server detects the best inference backend at startup:
**No manual download required!** The models will be cached locally after the first download.
| Platform | Backend | Acceleration |
Available models: |----------|---------|-------------|
- **1.7B** (recommended): `Qwen/Qwen3-TTS-12Hz-1.7B-Base` (~4GB) | macOS (Apple Silicon) | MLX | Metal / Neural Engine |
- **0.6B** (faster): `Qwen/Qwen3-TTS-12Hz-0.6B-Base` (~2GB) | Windows / Linux (NVIDIA) | PyTorch | CUDA |
| Linux (AMD) | PyTorch | ROCm |
**Note:** The first generation will take longer as the model downloads. Subsequent generations will use the cached model. | Intel Arc | PyTorch | IPEX / XPU |
| Windows (any GPU) | PyTorch | DirectML |
#### Manual Download (Optional) | Any | PyTorch | CPU fallback |
If you prefer to download models manually or have limited internet during runtime: Detection is handled by `utils/platform_detect.py`. Both backends implement the same `TTSBackend` protocol, so the API layer is engine-agnostic.
```bash ## API
# Install huggingface-cli
pip install huggingface_hub 90 endpoints organized by domain. Full interactive documentation available at `http://localhost:17493/docs` when the server is running.
# Download 1.7B model | Domain | Prefix | Description |
huggingface-cli download Qwen/Qwen3-TTS-12Hz-1.7B-Base |--------|--------|-------------|
| Health | `/`, `/health` | Server status, GPU info, filesystem checks |
# Or use Python | Profiles | `/profiles` | Voice profile CRUD, samples, avatars, import/export |
python -c "from huggingface_hub import snapshot_download; snapshot_download('Qwen/Qwen3-TTS-12Hz-1.7B-Base')" | Channels | `/channels` | Audio channel management and voice assignment |
``` | Generation | `/generate` | TTS generation, retry, regenerate, status SSE |
| History | `/history` | Generation history, search, favorites, export |
Models are cached in `~/.cache/huggingface/hub/` by default. | Transcription | `/transcribe` | Whisper-based audio-to-text |
| Stories | `/stories` | Multi-track timeline editor, audio export |
### 4. Run Server | Effects | `/effects` | Effect presets, preview, version management |
| Audio | `/audio`, `/samples` | Audio file serving |
```bash | Models | `/models` | Load, unload, download, migrate, status |
# Development (local only) | Tasks | `/tasks`, `/cache` | Active task tracking, cache management |
python -m backend.main | CUDA | `/backend/cuda-*` | CUDA binary download and management |
# Production (allow remote access) ### Quick examples
python -m backend.main --host 0.0.0.0 --port 8000
```
## Usage Examples
The desktop app, web client, and current development workflow use `http://localhost:17493` by default.
If you launch the backend manually with a different host or port, substitute that address in the examples below.
### Creating a Voice Profile
```bash
# 1. Create profile
curl -X POST http://localhost:17493/profiles \
-H "Content-Type: application/json" \
-d '{"name": "My Voice", "language": "en"}'
# Response: {"id": "abc-123", ...}
# 2. Add sample
curl -X POST http://localhost:17493/profiles/abc-123/samples \
-F "[email protected]" \
-F "reference_text=This is my voice sample"
```
### Generating Speech
```bash ```bash
# Generate speech
curl -X POST http://localhost:17493/generate \ curl -X POST http://localhost:17493/generate \
-H "Content-Type: application/json" \ -H "Content-Type: application/json" \
-d '{ -d '{"text": "Hello world", "profile_id": "...", "language": "en"}'
"profile_id": "abc-123",
"text": "Hello, this is a test.",
"language": "en",
"seed": 42
}'
# Response: {"id": "gen-456", "audio_path": "/path/to/audio.wav", ...} # List profiles
curl http://localhost:17493/profiles
# Download audio # Stream generation status (SSE)
curl http://localhost:17493/audio/gen-456 -o output.wav curl http://localhost:17493/generate/{id}/status
``` ```
### Transcribing Audio ## Data directory
```
{data_dir}/
voicebox.db # SQLite database
profiles/{id}/ # Voice samples per profile
generations/ # Generated audio files
cache/ # Voice prompt cache (memory + disk)
backends/ # Downloaded CUDA binary (if applicable)
```
Default location is the OS-specific app data directory. Override with `--data-dir` or the `VOICEBOX_DATA_DIR` environment variable.
## Code quality
Linting and formatting are enforced by [ruff](https://docs.astral.sh/ruff/), configured in `pyproject.toml`. See `STYLE_GUIDE.md` for conventions.
```bash ```bash
curl -X POST http://localhost:17493/transcribe \ just check-python # lint + format check
-F "[email protected]" \ just fix-python # auto-fix lint issues + reformat
-F "language=en" just test # run pytest
# Response: {"text": "Transcribed text", "duration": 5.5}
``` ```
## Advanced Features ## Dependencies
### Multi-Sample Profiles Runtime dependencies are in `requirements.txt`. macOS-only MLX dependencies are in `requirements-mlx.txt`. Dev tools (ruff, pytest) are installed automatically by `just setup-python`.
Add multiple samples to a profile for better quality:
```bash
# Add first sample
curl -X POST http://localhost:17493/profiles/abc-123/samples \
-F "[email protected]" \
-F "reference_text=First sample"
# Add second sample
curl -X POST http://localhost:17493/profiles/abc-123/samples \
-F "[email protected]" \
-F "reference_text=Second sample"
# Generation will automatically combine all samples
```
### Voice Prompt Caching
Voice prompts are automatically cached for faster generation:
- First generation: ~5-10 seconds (creates prompt)
- Subsequent generations: ~1-2 seconds (uses cached prompt)
Cache is stored in `data/cache/` and persists across server restarts.
### VRAM Management
Models are lazy-loaded and can be manually unloaded:
```bash
# Unload TTS model
curl -X POST http://localhost:17493/models/unload
# Load specific model size
curl -X POST "http://localhost:17493/models/load?model_size=0.6B"
```
## Error Handling
All endpoints return proper HTTP status codes:
- `200 OK`: Success
- `400 Bad Request`: Invalid input
- `404 Not Found`: Resource not found
- `500 Internal Server Error`: Server error
Error responses include details:
```json
{
"detail": "Profile not found"
}
```
## Performance Tips
1. **Use multi-sample profiles** - Better quality than single sample
2. **Let caching work** - Voice prompts are cached automatically
3. **Use 0.6B model on CPU** - Faster than 1.7B with acceptable quality
4. **Use 1.7B model on GPU** - Best quality, still fast
5. **Unload Whisper after transcription** - Frees VRAM for TTS
## TODO
- [ ] WebSocket support for generation progress
- [ ] Batch generation endpoint
- [ ] Audio effects (M3GAN, etc.)
- [ ] Voice design (text-to-voice)
- [ ] Audio studio timeline features
- [ ] Project management
- [ ] Authentication & rate limiting
- [ ] Export/import profiles
## License
See main project LICENSE.
+404
View File
@@ -0,0 +1,404 @@
# Python Style Guide
Target: **Python 3.12+** | Formatter/Linter: **Ruff** | Config: `backend/pyproject.toml`
This guide codifies the conventions used across the backend, and prescribes the target style for code written during the refactor (Phases 3-6). Existing code should be migrated incrementally -- don't reformat entire files in unrelated PRs.
---
## Formatting
Enforced by `ruff format` (Black-compatible).
- **Line length**: 120 characters.
- **Indent**: 4 spaces. No tabs.
- **Trailing commas**: Required on multi-line function signatures, arguments, collections.
- **Quotes**: Double quotes (`"`) for strings. Single quotes are acceptable in f-string expressions and dict keys inside f-strings where avoiding escapes improves readability.
Run: `ruff format backend/`
---
## Imports
Enforced by ruff's `isort` rules (rule set `I`).
**Grouping** -- three blocks separated by a blank line:
```python
import asyncio # 1. stdlib
from pathlib import Path
import numpy as np # 2. third-party
from fastapi import APIRouter, HTTPException
from sqlalchemy.orm import Session
from backend.config import get_data_dir # 3. local (absolute)
from .database import get_db # or relative
```
**Rules:**
- Within the `backend` package, use **relative imports** for sibling/child modules: `from .database import get_db`, `from ..utils.audio import load_audio`.
- Absolute imports are fine for top-level references from entry points (`main.py`, `server.py`).
- Never use wildcard imports (`from module import *`).
- One import per line for `from X import Y` when there are 4+ names; below that, comma-separated is fine.
- **Lazy imports** are acceptable for heavy dependencies (torch, transformers, mlx) inside functions to reduce startup time. Add a comment: `# lazy: heavy import`.
---
## Type Annotations
Python 3.12 means we use **built-in generics and union syntax natively**. No `from __future__ import annotations`, no `typing.List`/`typing.Dict`.
```python
# Yes
def process(items: list[str], config: dict[str, int] | None = None) -> tuple[int, str]: ...
# No
from typing import List, Dict, Optional, Tuple
def process(items: List[str], config: Optional[Dict[str, int]] = None) -> Tuple[int, str]: ...
```
**What to annotate:**
- All public function signatures (parameters + return type).
- Private functions: parameters at minimum; return type encouraged.
- Module-level variables: only when the type isn't obvious from the assignment.
- Route handlers: parameters are annotated via FastAPI's dependency injection. Add explicit `-> SomeResponse` return types when the route doesn't use `response_model`.
**Imports from `typing` that are still needed** (no built-in equivalent):
`Literal`, `TypeAlias`, `Protocol`, `runtime_checkable`, `Callable`, `Any`, `ClassVar`, `TypeVar`, `overload`, `TYPE_CHECKING`.
Use `collections.abc` for abstract types: `Sequence`, `Mapping`, `Iterable`, `Iterator`, `Generator`.
---
## Naming
| Thing | Convention | Example |
|-------|-----------|---------|
| Module | `snake_case` | `task_queue.py` |
| Class | `PascalCase` | `ProgressManager` |
| Function / method | `snake_case` | `create_profile` |
| Variable | `snake_case` | `sample_rate` |
| Constant | `UPPER_SNAKE_CASE` | `DEFAULT_SAMPLE_RATE` |
| Private | `_leading_underscore` | `_generation_queue` |
| Type alias | `PascalCase` | `EffectChain = list[dict[str, Any]]` |
**Specific conventions:**
- Database ORM models imported with `DB` prefix alias: `from .database import VoiceProfile as DBVoiceProfile`.
- Pydantic models use descriptive suffixes: `VoiceProfileCreate`, `VoiceProfileResponse`, `GenerationRequest`.
- Backend classes use engine-name prefix: `MLXTTSBackend`, `PyTorchSTTBackend`.
---
## Docstrings
**Google style**. Required on all public functions, classes, and modules.
```python
def combine_voice_prompts(
profile_dir: Path,
*,
target_sr: int = 24000,
) -> tuple[np.ndarray, int]:
"""Load and concatenate all voice prompt files for a profile.
Reads .wav/.mp3/.flac files from the profile directory, resamples to
the target sample rate, normalizes, and concatenates into a single array.
Args:
profile_dir: Path to the voice profile directory containing audio files.
target_sr: Target sample rate for the output. Defaults to 24000.
Returns:
Tuple of (concatenated audio array, sample rate).
Raises:
FileNotFoundError: If profile_dir does not exist.
ValueError: If no valid audio files are found.
"""
```
**Short form** is fine for simple functions:
```python
def get_db_path() -> Path:
"""Get the path to the SQLite database file."""
```
**When to skip**: Private helpers under ~5 lines where the name and signature make intent obvious.
**Module docstrings**: A single sentence at the top of every file describing its purpose.
```python
"""Voice profile CRUD operations."""
```
---
## Comments
Comments explain **why**, not **what**. If the code needs a comment to explain what it does, the code should be rewritten to be clearer. The exceptions are non-obvious performance choices, external constraints, and concurrency/race-condition reasoning -- those always deserve a comment.
### No section dividers
Do not use ASCII dividers to create visual sections in files:
```python
# No -- any of these:
# ============================================
# GENERATION ENDPOINTS
# ============================================
# ---------------------------------------------------------------------------
# Device detection
# ---------------------------------------------------------------------------
# --- Load model --------------------------------------------------
```
If a file needs section dividers to be navigable, the file is too long. Split it into modules. Within a function, if you need labeled sections to follow the logic, extract those sections into named functions.
### Inline comments
Inline comments (end-of-line) are fine when they add information the code can't express:
```python
# Yes -- explains a non-obvious constraint or gives context:
audio, sr = load_audio(path, sr=24000) # Qwen expects 24kHz mono
_generation_queue: asyncio.Queue = None # type: ignore # initialized at startup
"tauri://localhost", # Tauri webview (macOS)
# No -- restates the code:
# Check if profile name already exists
existing = db.query(DBVoiceProfile).filter_by(name=data.name).first()
# Delete from database
db.delete(sample)
# Update fields
profile.name = data.name
```
Delete comments that narrate what the next line of code obviously does. If the function name, variable name, or method call already communicates intent, the comment is noise.
### Block comments
Use block comments for **why** explanations -- constraints, workarounds, non-obvious decisions:
```python
# PyInstaller + multiprocessing: child processes re-execute the frozen binary
# with internal arguments. freeze_support() handles this and exits early.
multiprocessing.freeze_support()
# Mark any stale "generating" records as failed -- these are leftovers
# from a previous process that was killed mid-generation.
db.query(Generation).filter_by(status="generating").update({"status": "failed"})
```
Keep block comments tight. Two to three lines is normal. If you need a paragraph, it probably belongs in the docstring or a design doc.
### Linter/type-checker suppression
Always add a reason after `noqa` and `type: ignore`:
```python
import intel_extension_for_pytorch # noqa: F401 -- side-effect import enables XPU
_queue: asyncio.Queue = None # type: ignore[assignment] # initialized at startup
```
Bare `# noqa` or `# type: ignore` with no explanation are not allowed.
### TODO / FIXME
Use sparingly. Every `TODO` must include a brief description of what needs doing. Don't use them as a substitute for tracking work properly:
```python
# TODO: replace with async SQLAlchemy once CRUD modules are migrated (Phase 5)
result = await asyncio.to_thread(profiles.get_profile, profile_id, db)
```
Never commit `HACK`, `XXX`, or `FIXME` -- fix the problem or file an issue.
### Commented-out code
Delete it. That's what git is for. If you need to document that something was intentionally removed, a short tombstone comment is acceptable:
```python
# Removed config.json-only check -- too lenient, doesn't confirm weights exist.
```
---
## Error Handling
The refactor is standardizing on a **two-layer pattern**:
### 1. Domain layer -- raise plain exceptions
CRUD modules and services raise `ValueError`, `FileNotFoundError`, or (post-refactor) custom exceptions defined in `backend/errors.py`:
```python
# backend/errors.py (to be created in Phase 4)
class NotFoundError(Exception):
"""Raised when a requested resource does not exist."""
class ConflictError(Exception):
"""Raised on uniqueness constraint violations."""
```
```python
# In a service or CRUD module:
raise NotFoundError(f"Profile {profile_id} not found")
```
### 2. Route layer -- translate to HTTPException
Route handlers catch domain exceptions and convert:
```python
@router.post("/profiles")
async def create_profile(data: VoiceProfileCreate, db: Session = Depends(get_db)):
try:
return await profiles.create_profile(data, db)
except ConflictError as e:
raise HTTPException(status_code=409, detail=str(e))
```
**Background tasks** catch `Exception` broadly, log with `logger.exception()`, and update the task status to `"failed"`.
**Never**: silently swallow exceptions, use bare `except:`, or catch `BaseException`.
---
## Async
### Rules for the refactor
1. **Don't declare `async def` unless the function awaits something.** Several service modules still declare `async def` without awaiting -- these should be migrated to sync functions with `asyncio.to_thread()` at the route layer, or to real async SQLAlchemy.
2. **CPU-bound work** (audio processing, numpy operations) goes through `asyncio.to_thread()`:
```python
audio, sr = await asyncio.to_thread(load_audio, source_path)
```
3. **GPU-bound TTS inference** is serialized through the generation queue (`services/task_queue.py`). Never call a backend's `generate()` directly from a route handler.
4. **Fire-and-forget tasks**: use `asyncio.create_task()` and track the task reference to prevent garbage collection:
```python
task = asyncio.create_task(some_coro())
_background_tasks.add(task)
task.add_done_callback(_background_tasks.discard)
```
---
## Logging
Use the `logging` module. Not `print()`.
```python
import logging
logger = logging.getLogger(__name__)
logger.info("Loading model %s on %s", model_name, device)
logger.warning("Cache miss for %s, downloading", repo_id)
logger.exception("Generation %s failed") # logs traceback automatically
```
**Rules:**
- Use `%s`-style placeholders in log calls (not f-strings). This avoids formatting the string if the log level is filtered out.
- Use `logger.exception()` inside `except` blocks -- it captures the traceback.
- Logger name should be `__name__` (yields `backend.utils.audio`, etc.).
- Existing `print()` calls should be migrated to logging as files are touched during the refactor.
---
## Constants
- Define at **module level** in the file where they're primarily used.
- Use `UPPER_SNAKE_CASE`.
- Shared/cross-cutting constants (sample rates, file size limits, CORS origins) go in `backend/config.py` after Phase 6 consolidation.
- Magic numbers in function bodies should be extracted to named constants:
```python
# No
if len(audio) > 24000 * 60 * 10:
# Yes
MAX_AUDIO_DURATION_SAMPLES = SAMPLE_RATE * 60 * 10
if len(audio) > MAX_AUDIO_DURATION_SAMPLES:
```
---
## Function Signatures
- **Keyword-only arguments** (after `*`) for functions with 3+ parameters, especially when several share the same type:
```python
def is_model_cached(
hf_repo: str,
*,
weight_extensions: tuple[str, ...] = (".safetensors", ".bin"),
required_files: list[str] | None = None,
) -> bool:
```
- Parameters on **separate lines** when the signature exceeds ~100 characters or has 3+ params.
- **Trailing comma** after the last parameter in multi-line signatures.
- Default values inline with the parameter.
---
## String Formatting
- **f-strings** for runtime string construction.
- **`%s`-style** for `logging` calls (lazy evaluation).
- **`.format()`**: avoid; f-strings are preferred.
---
## Testing
Framework: **pytest** with `pytest-asyncio`.
- Test files: `test_<module>.py` in `backend/tests/`.
- Use `conftest.py` for shared fixtures (db sessions, test client, mock backends).
- Group related tests in classes: `class TestProfileCRUD:`.
- Use `@pytest.mark.asyncio` for async tests.
- Use `@pytest.mark.parametrize` to reduce repetition.
- Manual integration scripts stay in `tests/` but are clearly marked (filename prefix `manual_` or documented in `tests/README.md`).
---
## Project Layout
```
backend/
app.py # FastAPI app factory, CORS, lifecycle events
main.py # Entry point (imports app, runs uvicorn)
config.py # Data directory paths
models.py # Pydantic request/response schemas
server.py # Tauri sidecar launcher, parent-pid watchdog
routes/ # Thin HTTP handlers (validation, delegation, response formatting)
services/ # Business logic, CRUD, orchestration
backends/ # TTS/STT engine implementations
database/ # ORM models, session management, migrations, seeds
utils/ # Shared utilities (audio, effects, caching, progress)
tests/ # pytest suite
```
---
## Ruff Adoption
`pyproject.toml` configures ruff for linting and formatting. Run:
```bash
# Lint (check)
ruff check backend/
# Lint (auto-fix)
ruff check backend/ --fix
# Format
ruff format backend/
```
Introduce ruff fixes file-by-file as you touch them. Don't run `--fix` across the entire codebase in one shot -- that creates unreviewable diffs.
+1 -1
View File
@@ -1,3 +1,3 @@
# Backend package # Backend package
__version__ = "0.1.13" __version__ = "0.2.3"
+215
View File
@@ -0,0 +1,215 @@
"""FastAPI application factory, middleware, and lifecycle events."""
import asyncio
import logging
import os
import sys
from pathlib import Path
class ColoredFormatter(logging.Formatter):
"""Custom formatter to add colors matching uvicorn's style."""
COLORS = {
"DEBUG": "\033[36m", # Cyan
"INFO": "\033[32m", # Green
"WARNING": "\033[33m", # Yellow
"ERROR": "\033[31m", # Red
"CRITICAL": "\033[35m", # Magenta
}
RESET = "\033[0m"
def format(self, record):
log_color = self.COLORS.get(record.levelname, self.RESET)
record.levelname = f"{log_color}{record.levelname}{self.RESET}"
return super().format(record)
# Configure logging to match uvicorn's format with colors
handler = logging.StreamHandler(sys.stderr)
handler.setFormatter(ColoredFormatter("%(levelname)s: %(message)s"))
logging.basicConfig(
level=logging.INFO,
handlers=[handler],
)
logger = logging.getLogger(__name__)
# AMD GPU environment variables must be set before torch import
if not os.environ.get("HSA_OVERRIDE_GFX_VERSION"):
os.environ["HSA_OVERRIDE_GFX_VERSION"] = "10.3.0"
if not os.environ.get("MIOPEN_LOG_LEVEL"):
os.environ["MIOPEN_LOG_LEVEL"] = "4"
import torch
from fastapi import FastAPI
from fastapi.middleware.cors import CORSMiddleware
from urllib.parse import quote
from . import __version__, config, database
from .services import tts, transcribe
from .database import get_db
from .utils.platform_detect import get_backend_type
from .utils.progress import get_progress_manager
from .services.task_queue import create_background_task, init_queue
from .routes import register_routers
def safe_content_disposition(disposition_type: str, filename: str) -> str:
"""Build a Content-Disposition header safe for non-ASCII filenames.
Uses RFC 5987 ``filename*`` parameter so browsers can decode UTF-8
filenames while the ``filename`` fallback stays ASCII-only.
"""
ascii_name = "".join(c for c in filename if c.isascii() and (c.isalnum() or c in " -_.")).strip() or "download"
utf8_name = quote(filename, safe="")
return f"{disposition_type}; filename=\"{ascii_name}\"; filename*=UTF-8''{utf8_name}"
def create_app() -> FastAPI:
"""Create and configure the FastAPI application."""
application = FastAPI(
title="voicebox API",
description="Production-quality Qwen3-TTS voice cloning API",
version=__version__,
)
_configure_cors(application)
register_routers(application)
_register_lifecycle(application)
return application
def _configure_cors(application: FastAPI) -> None:
"""Set up CORS middleware with local-first defaults."""
default_origins = [
"http://localhost:5173", # Vite dev server
"http://127.0.0.1:5173",
"http://localhost:17493",
"http://127.0.0.1:17493",
"tauri://localhost", # Tauri webview (macOS)
"https://tauri.localhost", # Tauri webview (Windows/Linux)
"http://tauri.localhost", # Tauri webview (Windows, some builds)
]
env_origins = os.environ.get("VOICEBOX_CORS_ORIGINS", "")
all_origins = default_origins + [o.strip() for o in env_origins.split(",") if o.strip()]
application.add_middleware(
CORSMiddleware,
allow_origins=all_origins,
allow_credentials=True,
allow_methods=["*"],
allow_headers=["*"],
)
def _get_gpu_status() -> str:
"""Return a human-readable string describing GPU availability."""
backend_type = get_backend_type()
if torch.cuda.is_available():
device_name = torch.cuda.get_device_name(0)
is_rocm = hasattr(torch.version, "hip") and torch.version.hip is not None
if is_rocm:
return f"ROCm ({device_name})"
return f"CUDA ({device_name})"
elif hasattr(torch.backends, "mps") and torch.backends.mps.is_available():
return "MPS (Apple Silicon)"
elif backend_type == "mlx":
return "Metal (Apple Silicon via MLX)"
return "None (CPU only)"
def _register_lifecycle(application: FastAPI) -> None:
"""Attach startup and shutdown event handlers."""
@application.on_event("startup")
async def startup_event():
import platform
import sys
logger.info("Voicebox v%s starting up", __version__)
logger.info(
"Python %s on %s %s (%s)",
sys.version.split()[0],
platform.system(),
platform.release(),
platform.machine(),
)
database.init_db()
from .database.session import _db_path
logger.info("Database: %s", _db_path)
logger.info("Data directory: %s", config.get_data_dir())
init_queue()
# Mark stale "generating" records as failed -- leftovers from a killed process
from sqlalchemy import text as sa_text
db = next(get_db())
try:
result = db.execute(
sa_text(
"UPDATE generations SET status = 'failed', "
"error = 'Server was shut down during generation' "
"WHERE status IN ('generating', 'loading_model')"
)
)
if result.rowcount > 0:
logger.info("Marked %d stale generation(s) as failed", result.rowcount)
from .database import VoiceProfile as DBVoiceProfile, Generation as DBGeneration
profile_count = db.query(DBVoiceProfile).count()
generation_count = db.query(DBGeneration).count()
logger.info("Profiles: %d, Generations: %d", profile_count, generation_count)
db.commit()
except Exception as e:
db.rollback()
logger.warning("Could not clean up stale generations: %s", e)
finally:
db.close()
backend_type = get_backend_type()
logger.info("Backend: %s", backend_type.upper())
logger.info("GPU: %s", _get_gpu_status())
from .services.cuda import check_and_update_cuda_binary
create_background_task(check_and_update_cuda_binary())
try:
progress_manager = get_progress_manager()
progress_manager._set_main_loop(asyncio.get_running_loop())
except Exception as e:
logger.warning("Could not initialize progress manager event loop: %s", e)
try:
from huggingface_hub import constants as hf_constants
cache_dir = Path(hf_constants.HF_HUB_CACHE)
cache_dir.mkdir(parents=True, exist_ok=True)
logger.info("Model cache: %s", cache_dir)
except Exception as e:
logger.warning("Could not create HuggingFace cache directory: %s", e)
logger.info("Ready")
@application.on_event("shutdown")
async def shutdown_event():
logger.info("Voicebox server shutting down...")
try:
tts.unload_tts_model()
except Exception:
logger.exception("Failed to unload TTS model")
try:
transcribe.unload_whisper_model()
except Exception:
logger.exception("Failed to unload Whisper model")
app = create_app()
+347 -31
View File
@@ -1,25 +1,66 @@
""" """
Backend abstraction layer for TTS and STT. Backend abstraction layer for TTS and STT.
Provides a unified interface for MLX and PyTorch backends. Provides a unified interface for MLX and PyTorch backends,
and a model config registry that eliminates per-engine dispatch maps.
""" """
import threading import threading
from dataclasses import dataclass, field
from typing import Protocol, Optional, Tuple, List from typing import Protocol, Optional, Tuple, List
from typing_extensions import runtime_checkable from typing_extensions import runtime_checkable
import numpy as np import numpy as np
from ..platform_detect import get_backend_type from ..utils.platform_detect import get_backend_type
LANGUAGE_CODE_TO_NAME = {
"zh": "chinese",
"en": "english",
"ja": "japanese",
"ko": "korean",
"de": "german",
"fr": "french",
"ru": "russian",
"pt": "portuguese",
"es": "spanish",
"it": "italian",
}
WHISPER_HF_REPOS = {
"base": "openai/whisper-base",
"small": "openai/whisper-small",
"medium": "openai/whisper-medium",
"large": "openai/whisper-large-v3",
"turbo": "openai/whisper-large-v3-turbo",
}
@dataclass
class ModelConfig:
"""Declarative config for a downloadable model variant."""
model_name: str # e.g. "luxtts", "chatterbox-tts"
display_name: str # e.g. "LuxTTS (Fast, CPU-friendly)"
engine: str # e.g. "luxtts", "chatterbox"
hf_repo_id: str # e.g. "YatharthS/LuxTTS"
model_size: str = "default"
size_mb: int = 0
needs_trim: bool = False
supports_instruct: bool = False
languages: list[str] = field(default_factory=lambda: ["en"])
@runtime_checkable @runtime_checkable
class TTSBackend(Protocol): class TTSBackend(Protocol):
"""Protocol for TTS backend implementations.""" """Protocol for TTS backend implementations."""
# Each backend class should define MODEL_CONFIGS as a class variable:
# MODEL_CONFIGS: list[ModelConfig]
async def load_model(self, model_size: str) -> None: async def load_model(self, model_size: str) -> None:
"""Load TTS model.""" """Load TTS model."""
... ...
async def create_voice_prompt( async def create_voice_prompt(
self, self,
audio_path: str, audio_path: str,
@@ -28,12 +69,12 @@ class TTSBackend(Protocol):
) -> Tuple[dict, bool]: ) -> Tuple[dict, bool]:
""" """
Create voice prompt from reference audio. Create voice prompt from reference audio.
Returns: Returns:
Tuple of (voice_prompt_dict, was_cached) Tuple of (voice_prompt_dict, was_cached)
""" """
... ...
async def combine_voice_prompts( async def combine_voice_prompts(
self, self,
audio_paths: List[str], audio_paths: List[str],
@@ -41,12 +82,12 @@ class TTSBackend(Protocol):
) -> Tuple[np.ndarray, str]: ) -> Tuple[np.ndarray, str]:
""" """
Combine multiple voice prompts. Combine multiple voice prompts.
Returns: Returns:
Tuple of (combined_audio_array, combined_text) Tuple of (combined_audio_array, combined_text)
""" """
... ...
async def generate( async def generate(
self, self,
text: str, text: str,
@@ -57,24 +98,24 @@ class TTSBackend(Protocol):
) -> Tuple[np.ndarray, int]: ) -> Tuple[np.ndarray, int]:
""" """
Generate audio from text. Generate audio from text.
Returns: Returns:
Tuple of (audio_array, sample_rate) Tuple of (audio_array, sample_rate)
""" """
... ...
def unload_model(self) -> None: def unload_model(self) -> None:
"""Unload model to free memory.""" """Unload model to free memory."""
... ...
def is_loaded(self) -> bool: def is_loaded(self) -> bool:
"""Check if model is loaded.""" """Check if model is loaded."""
... ...
def _get_model_path(self, model_size: str) -> str: def _get_model_path(self, model_size: str) -> str:
""" """
Get model path for a given size. Get model path for a given size.
Returns: Returns:
Model path or HuggingFace Hub ID Model path or HuggingFace Hub ID
""" """
@@ -84,11 +125,11 @@ class TTSBackend(Protocol):
@runtime_checkable @runtime_checkable
class STTBackend(Protocol): class STTBackend(Protocol):
"""Protocol for STT (Speech-to-Text) backend implementations.""" """Protocol for STT (Speech-to-Text) backend implementations."""
async def load_model(self, model_size: str) -> None: async def load_model(self, model_size: str) -> None:
"""Load STT model.""" """Load STT model."""
... ...
async def transcribe( async def transcribe(
self, self,
audio_path: str, audio_path: str,
@@ -96,16 +137,16 @@ class STTBackend(Protocol):
) -> str: ) -> str:
""" """
Transcribe audio to text. Transcribe audio to text.
Returns: Returns:
Transcribed text Transcribed text
""" """
... ...
def unload_model(self) -> None: def unload_model(self) -> None:
"""Unload model to free memory.""" """Unload model to free memory."""
... ...
def is_loaded(self) -> bool: def is_loaded(self) -> bool:
"""Check if model is loaded.""" """Check if model is loaded."""
... ...
@@ -117,7 +158,8 @@ _tts_backends: dict[str, TTSBackend] = {}
_tts_backends_lock = threading.Lock() _tts_backends_lock = threading.Lock()
_stt_backend: Optional[STTBackend] = None _stt_backend: Optional[STTBackend] = None
# Supported TTS engines # Supported TTS engines — keyed by engine name, value is the backend class import path.
# The factory function uses this for the if/elif chain; the model configs live on the backend classes.
TTS_ENGINES = { TTS_ENGINES = {
"qwen": "Qwen TTS", "qwen": "Qwen TTS",
"luxtts": "LuxTTS", "luxtts": "LuxTTS",
@@ -126,10 +168,277 @@ TTS_ENGINES = {
} }
def _get_qwen_model_configs() -> list[ModelConfig]:
"""Return Qwen model configs with backend-aware HF repo IDs."""
backend_type = get_backend_type()
if backend_type == "mlx":
repo_1_7b = "mlx-community/Qwen3-TTS-12Hz-1.7B-Base-bf16"
repo_0_6b = "mlx-community/Qwen3-TTS-12Hz-1.7B-Base-bf16" # 0.6B not available in MLX, falls back
else:
repo_1_7b = "Qwen/Qwen3-TTS-12Hz-1.7B-Base"
repo_0_6b = "Qwen/Qwen3-TTS-12Hz-0.6B-Base"
return [
ModelConfig(
model_name="qwen-tts-1.7B",
display_name="Qwen TTS 1.7B",
engine="qwen",
hf_repo_id=repo_1_7b,
model_size="1.7B",
size_mb=3500,
supports_instruct=False, # Base model drops instruct silently
languages=["zh", "en", "ja", "ko", "de", "fr", "ru", "pt", "es", "it"],
),
ModelConfig(
model_name="qwen-tts-0.6B",
display_name="Qwen TTS 0.6B",
engine="qwen",
hf_repo_id=repo_0_6b,
model_size="0.6B",
size_mb=1200,
supports_instruct=False,
languages=["zh", "en", "ja", "ko", "de", "fr", "ru", "pt", "es", "it"],
),
]
def _get_non_qwen_tts_configs() -> list[ModelConfig]:
"""Return model configs for non-Qwen TTS engines.
These are static — no backend-type branching needed.
"""
return [
ModelConfig(
model_name="luxtts",
display_name="LuxTTS (Fast, CPU-friendly)",
engine="luxtts",
hf_repo_id="YatharthS/LuxTTS",
size_mb=300,
languages=["en"],
),
ModelConfig(
model_name="chatterbox-tts",
display_name="Chatterbox TTS (Multilingual)",
engine="chatterbox",
hf_repo_id="ResembleAI/chatterbox",
size_mb=3200,
needs_trim=True,
languages=[
"zh",
"en",
"ja",
"ko",
"de",
"fr",
"ru",
"pt",
"es",
"it",
"he",
"ar",
"da",
"el",
"fi",
"hi",
"ms",
"nl",
"no",
"pl",
"sv",
"sw",
"tr",
],
),
ModelConfig(
model_name="chatterbox-turbo",
display_name="Chatterbox Turbo (English, Tags)",
engine="chatterbox_turbo",
hf_repo_id="ResembleAI/chatterbox-turbo",
size_mb=1500,
needs_trim=True,
languages=["en"],
),
]
def _get_whisper_configs() -> list[ModelConfig]:
"""Return Whisper STT model configs."""
return [
ModelConfig(
model_name="whisper-base",
display_name="Whisper Base",
engine="whisper",
hf_repo_id="openai/whisper-base",
model_size="base",
),
ModelConfig(
model_name="whisper-small",
display_name="Whisper Small",
engine="whisper",
hf_repo_id="openai/whisper-small",
model_size="small",
),
ModelConfig(
model_name="whisper-medium",
display_name="Whisper Medium",
engine="whisper",
hf_repo_id="openai/whisper-medium",
model_size="medium",
),
ModelConfig(
model_name="whisper-large",
display_name="Whisper Large",
engine="whisper",
hf_repo_id="openai/whisper-large-v3",
model_size="large",
),
ModelConfig(
model_name="whisper-turbo",
display_name="Whisper Turbo",
engine="whisper",
hf_repo_id="openai/whisper-large-v3-turbo",
model_size="turbo",
),
]
def get_all_model_configs() -> list[ModelConfig]:
"""Return the full list of model configs (TTS + STT)."""
return _get_qwen_model_configs() + _get_non_qwen_tts_configs() + _get_whisper_configs()
def get_tts_model_configs() -> list[ModelConfig]:
"""Return only TTS model configs."""
return _get_qwen_model_configs() + _get_non_qwen_tts_configs()
# Lookup helpers — these replace the if/elif chains in main.py
def get_model_config(model_name: str) -> Optional[ModelConfig]:
"""Look up a model config by model_name."""
for cfg in get_all_model_configs():
if cfg.model_name == model_name:
return cfg
return None
def engine_needs_trim(engine: str) -> bool:
"""Whether this engine's output should be run through trim_tts_output."""
for cfg in get_tts_model_configs():
if cfg.engine == engine:
return cfg.needs_trim
return False
def engine_has_model_sizes(engine: str) -> bool:
"""Whether this engine supports multiple model sizes (only Qwen currently)."""
configs = [c for c in get_tts_model_configs() if c.engine == engine]
return len(configs) > 1
async def load_engine_model(engine: str, model_size: str = "default") -> None:
"""Load a model for the given engine, handling the Qwen model_size special case."""
backend = get_tts_backend_for_engine(engine)
if engine == "qwen":
await backend.load_model_async(model_size)
else:
await backend.load_model()
async def ensure_model_cached_or_raise(engine: str, model_size: str = "default") -> None:
"""Check if a model is cached, raise HTTPException if not. Used by streaming endpoint."""
from fastapi import HTTPException
backend = get_tts_backend_for_engine(engine)
cfg = None
for c in get_tts_model_configs():
if c.engine == engine and c.model_size == model_size:
cfg = c
break
if engine == "qwen":
if not backend._is_model_cached(model_size):
raise HTTPException(
status_code=400,
detail=f"Model {model_size} is not downloaded yet. Use /generate to trigger a download.",
)
else:
if not backend._is_model_cached():
display = cfg.display_name if cfg else engine
raise HTTPException(
status_code=400,
detail=f"{display} model is not downloaded yet. Use /generate to trigger a download.",
)
def unload_model_by_config(config: ModelConfig) -> bool:
"""Unload a model given its config. Returns True if it was loaded, False otherwise."""
from . import get_tts_backend_for_engine
from ..services import tts, transcribe
if config.engine == "whisper":
whisper_model = transcribe.get_whisper_model()
if whisper_model.is_loaded() and whisper_model.model_size == config.model_size:
transcribe.unload_whisper_model()
return True
return False
if config.engine == "qwen":
tts_model = tts.get_tts_model()
loaded_size = getattr(tts_model, "_current_model_size", None) or getattr(tts_model, "model_size", None)
if tts_model.is_loaded() and loaded_size == config.model_size:
tts.unload_tts_model()
return True
return False
# All other TTS engines
backend = get_tts_backend_for_engine(config.engine)
if backend.is_loaded():
backend.unload_model()
return True
return False
def check_model_loaded(config: ModelConfig) -> bool:
"""Check if a model is currently loaded."""
from . import get_tts_backend_for_engine
from ..services import tts, transcribe
try:
if config.engine == "whisper":
whisper_model = transcribe.get_whisper_model()
return whisper_model.is_loaded() and getattr(whisper_model, "model_size", None) == config.model_size
if config.engine == "qwen":
tts_model = tts.get_tts_model()
loaded_size = getattr(tts_model, "_current_model_size", None) or getattr(tts_model, "model_size", None)
return tts_model.is_loaded() and loaded_size == config.model_size
backend = get_tts_backend_for_engine(config.engine)
return backend.is_loaded()
except Exception:
return False
def get_model_load_func(config: ModelConfig):
"""Return a callable that loads/downloads the model."""
from . import get_tts_backend_for_engine
from ..services import tts, transcribe
if config.engine == "whisper":
return lambda: transcribe.get_whisper_model().load_model(config.model_size)
if config.engine == "qwen":
return lambda: tts.get_tts_model().load_model(config.model_size)
return lambda: get_tts_backend_for_engine(config.engine).load_model()
def get_tts_backend() -> TTSBackend: def get_tts_backend() -> TTSBackend:
""" """
Get or create the default (Qwen) TTS backend instance based on platform. Get or create the default (Qwen) TTS backend instance based on platform.
Returns: Returns:
TTS backend instance (MLX or PyTorch) TTS backend instance (MLX or PyTorch)
""" """
@@ -139,45 +448,50 @@ def get_tts_backend() -> TTSBackend:
def get_tts_backend_for_engine(engine: str) -> TTSBackend: def get_tts_backend_for_engine(engine: str) -> TTSBackend:
""" """
Get or create a TTS backend for the given engine. Get or create a TTS backend for the given engine.
Args: Args:
engine: Engine name ("qwen" or "luxtts") engine: Engine name (e.g. "qwen", "luxtts", "chatterbox", "chatterbox_turbo")
Returns: Returns:
TTS backend instance TTS backend instance
""" """
global _tts_backends global _tts_backends
# Fast path: check without lock # Fast path: check without lock
if engine in _tts_backends: if engine in _tts_backends:
return _tts_backends[engine] return _tts_backends[engine]
# Slow path: create with lock to avoid duplicate instantiation # Slow path: create with lock to avoid duplicate instantiation
with _tts_backends_lock: with _tts_backends_lock:
# Double-check after acquiring lock # Double-check after acquiring lock
if engine in _tts_backends: if engine in _tts_backends:
return _tts_backends[engine] return _tts_backends[engine]
if engine == "qwen": if engine == "qwen":
backend_type = get_backend_type() backend_type = get_backend_type()
if backend_type == "mlx": if backend_type == "mlx":
from .mlx_backend import MLXTTSBackend from .mlx_backend import MLXTTSBackend
backend = MLXTTSBackend() backend = MLXTTSBackend()
else: else:
from .pytorch_backend import PyTorchTTSBackend from .pytorch_backend import PyTorchTTSBackend
backend = PyTorchTTSBackend() backend = PyTorchTTSBackend()
elif engine == "luxtts": elif engine == "luxtts":
from .luxtts_backend import LuxTTSBackend from .luxtts_backend import LuxTTSBackend
backend = LuxTTSBackend() backend = LuxTTSBackend()
elif engine == "chatterbox": elif engine == "chatterbox":
from .chatterbox_backend import ChatterboxTTSBackend from .chatterbox_backend import ChatterboxTTSBackend
backend = ChatterboxTTSBackend() backend = ChatterboxTTSBackend()
elif engine == "chatterbox_turbo": elif engine == "chatterbox_turbo":
from .chatterbox_turbo_backend import ChatterboxTurboTTSBackend from .chatterbox_turbo_backend import ChatterboxTurboTTSBackend
backend = ChatterboxTurboTTSBackend() backend = ChatterboxTurboTTSBackend()
else: else:
raise ValueError(f"Unknown TTS engine: {engine}. Supported: {list(TTS_ENGINES.keys())}") raise ValueError(f"Unknown TTS engine: {engine}. Supported: {list(TTS_ENGINES.keys())}")
_tts_backends[engine] = backend _tts_backends[engine] = backend
return backend return backend
@@ -185,22 +499,24 @@ def get_tts_backend_for_engine(engine: str) -> TTSBackend:
def get_stt_backend() -> STTBackend: def get_stt_backend() -> STTBackend:
""" """
Get or create STT backend instance based on platform. Get or create STT backend instance based on platform.
Returns: Returns:
STT backend instance (MLX or PyTorch) STT backend instance (MLX or PyTorch)
""" """
global _stt_backend global _stt_backend
if _stt_backend is None: if _stt_backend is None:
backend_type = get_backend_type() backend_type = get_backend_type()
if backend_type == "mlx": if backend_type == "mlx":
from .mlx_backend import MLXSTTBackend from .mlx_backend import MLXSTTBackend
_stt_backend = MLXSTTBackend() _stt_backend = MLXSTTBackend()
else: else:
from .pytorch_backend import PyTorchSTTBackend from .pytorch_backend import PyTorchSTTBackend
_stt_backend = PyTorchSTTBackend() _stt_backend = PyTorchSTTBackend()
return _stt_backend return _stt_backend
+258
View File
@@ -0,0 +1,258 @@
"""
Shared utilities for TTS/STT backend implementations.
Eliminates duplication of cache checking, device detection,
voice prompt combination, and model loading progress tracking.
"""
import logging
import platform
from contextlib import contextmanager
from pathlib import Path
from typing import Callable, List, Optional, Tuple
import numpy as np
from ..utils.audio import normalize_audio, load_audio
from ..utils.progress import get_progress_manager
from ..utils.hf_progress import HFProgressTracker, create_hf_progress_callback
from ..utils.tasks import get_task_manager
logger = logging.getLogger(__name__)
def is_model_cached(
hf_repo: str,
*,
weight_extensions: tuple[str, ...] = (".safetensors", ".bin"),
required_files: Optional[list[str]] = None,
) -> bool:
"""
Check if a HuggingFace model is fully cached locally.
Args:
hf_repo: HuggingFace repo ID (e.g. "Qwen/Qwen3-TTS-12Hz-1.7B-Base")
weight_extensions: File extensions that count as model weights.
required_files: If set, check that these specific filenames exist
in snapshots instead of checking by extension.
Returns:
True if model is fully cached, False if missing or incomplete.
"""
try:
from huggingface_hub import constants as hf_constants
repo_cache = Path(hf_constants.HF_HUB_CACHE) / ("models--" + hf_repo.replace("/", "--"))
if not repo_cache.exists():
return False
# Incomplete blobs mean a download is still in progress
blobs_dir = repo_cache / "blobs"
if blobs_dir.exists() and any(blobs_dir.glob("*.incomplete")):
logger.debug(f"Found .incomplete files for {hf_repo}")
return False
snapshots_dir = repo_cache / "snapshots"
if not snapshots_dir.exists():
return False
if required_files:
# Check that every required filename exists somewhere in snapshots
for fname in required_files:
if not any(snapshots_dir.rglob(fname)):
return False
return True
# Check that at least one weight file exists
for ext in weight_extensions:
if any(snapshots_dir.rglob(f"*{ext}")):
return True
logger.debug(f"No model weights found for {hf_repo}")
return False
except Exception as e:
logger.warning(f"Error checking cache for {hf_repo}: {e}")
return False
def get_torch_device(
*,
allow_xpu: bool = False,
allow_directml: bool = False,
allow_mps: bool = False,
force_cpu_on_mac: bool = False,
) -> str:
"""
Detect the best available torch device.
Args:
allow_xpu: Check for Intel XPU (IPEX) support.
allow_directml: Check for DirectML (Windows) support.
allow_mps: Allow MPS (Apple Silicon). If False, MPS falls back to CPU.
force_cpu_on_mac: Force CPU on macOS regardless of GPU availability.
"""
if force_cpu_on_mac and platform.system() == "Darwin":
return "cpu"
import torch
if torch.cuda.is_available():
return "cuda"
if allow_xpu:
try:
import intel_extension_for_pytorch # noqa: F401
if hasattr(torch, "xpu") and torch.xpu.is_available():
return "xpu"
except ImportError:
pass
if allow_directml:
try:
import torch_directml
if torch_directml.device_count() > 0:
return torch_directml.device(0)
except ImportError:
pass
if allow_mps:
if hasattr(torch.backends, "mps") and torch.backends.mps.is_available():
return "mps"
return "cpu"
async def combine_voice_prompts(
audio_paths: List[str],
reference_texts: List[str],
*,
sample_rate: Optional[int] = None,
) -> Tuple[np.ndarray, str]:
"""
Combine multiple reference audio samples into one.
Loads each audio file, normalizes, concatenates, and joins texts.
Args:
audio_paths: Paths to reference audio files.
reference_texts: Corresponding transcripts.
sample_rate: If set, resample audio to this rate during loading.
"""
combined_audio = []
for path in audio_paths:
kwargs = {"sample_rate": sample_rate} if sample_rate else {}
audio, _sr = load_audio(path, **kwargs)
audio = normalize_audio(audio)
combined_audio.append(audio)
mixed = np.concatenate(combined_audio)
mixed = normalize_audio(mixed)
combined_text = " ".join(reference_texts)
return mixed, combined_text
@contextmanager
def model_load_progress(
model_name: str,
is_cached: bool,
filter_non_downloads: Optional[bool] = None,
):
"""
Context manager for model loading with HF download progress tracking.
Handles the tqdm patching, progress_manager/task_manager lifecycle,
and error reporting that every backend duplicates.
Args:
model_name: Progress tracking key (e.g. "qwen-tts-1.7B", "whisper-base").
is_cached: Whether the model is already downloaded.
filter_non_downloads: Whether to filter non-download tqdm bars.
Defaults to `is_cached`.
Yields:
The tracker context (already entered). The caller loads the model
inside the `with` block. The tqdm patch is torn down on exit.
Usage:
with model_load_progress("qwen-tts-1.7B", is_cached) as ctx:
self.model = SomeModel.from_pretrained(...)
"""
if filter_non_downloads is None:
filter_non_downloads = is_cached
progress_manager = get_progress_manager()
task_manager = get_task_manager()
progress_callback = create_hf_progress_callback(model_name, progress_manager)
tracker = HFProgressTracker(progress_callback, filter_non_downloads=filter_non_downloads)
tracker_context = tracker.patch_download()
tracker_context.__enter__()
if not is_cached:
task_manager.start_download(model_name)
progress_manager.update_progress(
model_name=model_name,
current=0,
total=0,
filename="Connecting to HuggingFace...",
status="downloading",
)
try:
yield tracker_context
except Exception as e:
# Report error to both managers
progress_manager.mark_error(model_name, str(e))
task_manager.error_download(model_name, str(e))
raise
else:
# Only mark complete if we were tracking a download
if not is_cached:
progress_manager.mark_complete(model_name)
task_manager.complete_download(model_name)
finally:
tracker_context.__exit__(None, None, None)
def patch_chatterbox_f32(model) -> None:
"""
Patch float64 -> float32 dtype mismatches in upstream chatterbox.
librosa.load returns float64 numpy arrays. Multiple upstream code paths
convert these to torch tensors via torch.from_numpy() without casting,
then matmul against float32 model weights. This patches the two known
entry points:
1. S3Tokenizer.log_mel_spectrogram — audio tensor hits _mel_filters (f32)
2. VoiceEncoder.forward — float64 mel spectrograms hit LSTM weights (f32)
"""
import types
# Patch S3Tokenizer
_tokzr = model.s3gen.tokenizer
_orig_log_mel = _tokzr.log_mel_spectrogram.__func__
def _f32_log_mel(self_tokzr, audio, padding=0):
import torch as _torch
if _torch.is_tensor(audio):
audio = audio.float()
return _orig_log_mel(self_tokzr, audio, padding)
_tokzr.log_mel_spectrogram = types.MethodType(_f32_log_mel, _tokzr)
# Patch VoiceEncoder
_ve = model.ve
_orig_ve_forward = _ve.forward.__func__
def _f32_ve_forward(self_ve, mels):
return _orig_ve_forward(self_ve, mels.float())
_ve.forward = types.MethodType(_f32_ve_forward, _ve)
+28 -158
View File
@@ -8,7 +8,6 @@ on macOS due to known MPS tensor issues.
import asyncio import asyncio
import logging import logging
import platform
import threading import threading
from pathlib import Path from pathlib import Path
from typing import ClassVar, List, Optional, Tuple from typing import ClassVar, List, Optional, Tuple
@@ -16,9 +15,13 @@ from typing import ClassVar, List, Optional, Tuple
import numpy as np import numpy as np
from . import TTSBackend from . import TTSBackend
from ..utils.audio import normalize_audio, load_audio from .base import (
from ..utils.progress import get_progress_manager is_model_cached,
from ..utils.tasks import get_task_manager get_torch_device,
combine_voice_prompts as _combine_voice_prompts,
model_load_progress,
patch_chatterbox_f32,
)
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
@@ -45,17 +48,7 @@ class ChatterboxTTSBackend:
self._model_load_lock = asyncio.Lock() self._model_load_lock = asyncio.Lock()
def _get_device(self) -> str: def _get_device(self) -> str:
"""Get the best available device. Forces CPU on macOS (MPS issue).""" return get_torch_device(force_cpu_on_mac=True)
if platform.system() == "Darwin":
return "cpu"
try:
import torch
if torch.cuda.is_available():
return "cuda"
except ImportError:
pass
return "cpu"
def is_loaded(self) -> bool: def is_loaded(self) -> bool:
return self.model is not None return self.model is not None
@@ -64,33 +57,7 @@ class ChatterboxTTSBackend:
return CHATTERBOX_HF_REPO return CHATTERBOX_HF_REPO
def _is_model_cached(self, model_size: str = "default") -> bool: def _is_model_cached(self, model_size: str = "default") -> bool:
"""Check if the Chatterbox multilingual model is cached locally.""" return is_model_cached(CHATTERBOX_HF_REPO, required_files=_MTL_WEIGHT_FILES)
try:
from huggingface_hub import constants as hf_constants
repo_cache = Path(hf_constants.HF_HUB_CACHE) / (
"models--" + CHATTERBOX_HF_REPO.replace("/", "--")
)
if not repo_cache.exists():
return False
blobs_dir = repo_cache / "blobs"
if blobs_dir.exists() and any(blobs_dir.glob("*.incomplete")):
return False
# Check for multilingual weight files
snapshots_dir = repo_cache / "snapshots"
if snapshots_dir.exists():
for fname in _MTL_WEIGHT_FILES:
if not any(snapshots_dir.rglob(fname)):
return False
return True
return False
except Exception as e:
logger.warning(f"Error checking Chatterbox cache: {e}")
return False
async def load_model(self, model_size: str = "default") -> None: async def load_model(self, model_size: str = "default") -> None:
"""Load the Chatterbox multilingual model.""" """Load the Chatterbox multilingual model."""
@@ -103,132 +70,45 @@ class ChatterboxTTSBackend:
def _load_model_sync(self): def _load_model_sync(self):
"""Synchronous model loading.""" """Synchronous model loading."""
from ..utils.hf_progress import HFProgressTracker, create_hf_progress_callback
progress_manager = get_progress_manager()
task_manager = get_task_manager()
model_name = "chatterbox-tts" model_name = "chatterbox-tts"
is_cached = self._is_model_cached() is_cached = self._is_model_cached()
# Set up HF progress tracking (intercepts tqdm for file-level progress) with model_load_progress(model_name, is_cached):
progress_callback = create_hf_progress_callback(model_name, progress_manager)
tracker = HFProgressTracker(progress_callback, filter_non_downloads=is_cached)
tracker_context = tracker.patch_download()
tracker_context.__enter__()
if not is_cached:
task_manager.start_download(model_name)
progress_manager.update_progress(
model_name=model_name,
current=0,
total=0,
filename="Connecting to HuggingFace...",
status="downloading",
)
try:
device = self._get_device() device = self._get_device()
self._device = device self._device = device
logger.info(f"Loading Chatterbox Multilingual TTS on {device}...") logger.info(f"Loading Chatterbox Multilingual TTS on {device}...")
import torch import torch
from chatterbox.mtl_tts import ChatterboxMultilingualTTS from chatterbox.mtl_tts import ChatterboxMultilingualTTS
# Load into a local variable first, apply all patches, then if device == "cpu":
# assign to self.model. This avoids leaving a half-initialised _orig_torch_load = torch.load
# model on self.model if any patch step raises an exception.
#
# Monkey-patch torch.load for CPU loading. The model's .pt files
# were saved on CUDA; from_pretrained() doesn't pass map_location
# so loading on CPU fails without this.
try:
if device == "cpu":
_orig_torch_load = torch.load
def _patched_load(*args, **kwargs): def _patched_load(*args, **kwargs):
kwargs.setdefault("map_location", "cpu") kwargs.setdefault("map_location", "cpu")
return _orig_torch_load(*args, **kwargs) return _orig_torch_load(*args, **kwargs)
with ChatterboxTTSBackend._load_lock: with ChatterboxTTSBackend._load_lock:
torch.load = _patched_load torch.load = _patched_load
try: try:
model = ChatterboxMultilingualTTS.from_pretrained( model = ChatterboxMultilingualTTS.from_pretrained(device=device)
device=device, finally:
) torch.load = _orig_torch_load
finally: else:
torch.load = _orig_torch_load model = ChatterboxMultilingualTTS.from_pretrained(device=device)
else:
model = ChatterboxMultilingualTTS.from_pretrained(
device=device,
)
finally:
tracker_context.__exit__(None, None, None)
# Fix: transformers >= 4.36 defaults LlamaModel to sdpa attention # Fix sdpa attention for output_attentions support
# which doesn't support output_attentions=True (needed by
# Chatterbox's AlignmentStreamAnalyzer). Force eager attention.
t3_tfmr = model.t3.tfmr t3_tfmr = model.t3.tfmr
if hasattr(t3_tfmr, "config") and hasattr( if hasattr(t3_tfmr, "config") and hasattr(t3_tfmr.config, "_attn_implementation"):
t3_tfmr.config, "_attn_implementation"
):
t3_tfmr.config._attn_implementation = "eager" t3_tfmr.config._attn_implementation = "eager"
for layer in getattr(t3_tfmr, "layers", []): for layer in getattr(t3_tfmr, "layers", []):
if hasattr(layer, "self_attn"): if hasattr(layer, "self_attn"):
layer.self_attn._attn_implementation = "eager" layer.self_attn._attn_implementation = "eager"
if not is_cached: patch_chatterbox_f32(model)
progress_manager.mark_complete(model_name)
task_manager.complete_download(model_name)
# Patch float64 → float32 dtype mismatches in upstream chatterbox.
# librosa.load returns float64 numpy; multiple upstream code paths
# convert it to a torch tensor via torch.from_numpy() without
# casting, then matmul it against float32 model weights.
import types
# Patch S3Tokenizer (used by s3gen.tokenizer)
_tokzr = model.s3gen.tokenizer
_orig_log_mel = _tokzr.log_mel_spectrogram.__func__
def _f32_log_mel(self_tokzr, audio, padding=0):
import torch as _torch
if _torch.is_tensor(audio):
audio = audio.float()
return _orig_log_mel(self_tokzr, audio, padding)
_tokzr.log_mel_spectrogram = types.MethodType(_f32_log_mel, _tokzr)
# Patch VoiceEncoder
_ve = model.ve
_orig_ve_forward = _ve.forward.__func__
def _f32_ve_forward(self_ve, mels):
return _orig_ve_forward(self_ve, mels.float())
_ve.forward = types.MethodType(_f32_ve_forward, _ve)
# All patches applied successfully — publish the model
self.model = model self.model = model
logger.info("Chatterbox Multilingual TTS loaded successfully") logger.info("Chatterbox Multilingual TTS loaded successfully")
except ImportError as e:
logger.error(
"chatterbox-tts package not found. "
"Install with: pip install chatterbox-tts"
)
if not is_cached:
progress_manager.mark_error(model_name, str(e))
task_manager.error_download(model_name, str(e))
raise
except Exception as e:
logger.error(f"Failed to load Chatterbox: {e}")
if not is_cached:
progress_manager.mark_error(model_name, str(e))
task_manager.error_download(model_name, str(e))
raise
def unload_model(self) -> None: def unload_model(self) -> None:
"""Unload model to free memory.""" """Unload model to free memory."""
@@ -267,17 +147,7 @@ class ChatterboxTTSBackend:
audio_paths: List[str], audio_paths: List[str],
reference_texts: List[str], reference_texts: List[str],
) -> Tuple[np.ndarray, str]: ) -> Tuple[np.ndarray, str]:
"""Combine multiple reference samples.""" return await _combine_voice_prompts(audio_paths, reference_texts)
combined_audio = []
for path in audio_paths:
audio, _sr = load_audio(path)
audio = normalize_audio(audio)
combined_audio.append(audio)
mixed = np.concatenate(combined_audio)
mixed = normalize_audio(mixed)
combined_text = " ".join(reference_texts)
return mixed, combined_text
# Per-language generation defaults. Lower temp + higher cfg = clearer speech. # Per-language generation defaults. Lower temp + higher cfg = clearer speech.
_LANG_DEFAULTS: ClassVar[dict] = { _LANG_DEFAULTS: ClassVar[dict] = {
+20 -155
View File
@@ -8,7 +8,6 @@ Forces CPU on macOS due to known MPS tensor issues.
import asyncio import asyncio
import logging import logging
import platform
import threading import threading
from pathlib import Path from pathlib import Path
from typing import ClassVar, List, Optional, Tuple from typing import ClassVar, List, Optional, Tuple
@@ -16,9 +15,13 @@ from typing import ClassVar, List, Optional, Tuple
import numpy as np import numpy as np
from . import TTSBackend from . import TTSBackend
from ..utils.audio import normalize_audio, load_audio from .base import (
from ..utils.progress import get_progress_manager is_model_cached,
from ..utils.tasks import get_task_manager get_torch_device,
combine_voice_prompts as _combine_voice_prompts,
model_load_progress,
patch_chatterbox_f32,
)
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
@@ -45,17 +48,7 @@ class ChatterboxTurboTTSBackend:
self._model_load_lock = asyncio.Lock() self._model_load_lock = asyncio.Lock()
def _get_device(self) -> str: def _get_device(self) -> str:
"""Get the best available device. Forces CPU on macOS (MPS issue).""" return get_torch_device(force_cpu_on_mac=True)
if platform.system() == "Darwin":
return "cpu"
try:
import torch
if torch.cuda.is_available():
return "cuda"
except ImportError:
pass
return "cpu"
def is_loaded(self) -> bool: def is_loaded(self) -> bool:
return self.model is not None return self.model is not None
@@ -64,33 +57,7 @@ class ChatterboxTurboTTSBackend:
return CHATTERBOX_TURBO_HF_REPO return CHATTERBOX_TURBO_HF_REPO
def _is_model_cached(self, model_size: str = "default") -> bool: def _is_model_cached(self, model_size: str = "default") -> bool:
"""Check if the Chatterbox Turbo model is cached locally.""" return is_model_cached(CHATTERBOX_TURBO_HF_REPO, required_files=_TURBO_WEIGHT_FILES)
try:
from huggingface_hub import constants as hf_constants
repo_cache = Path(hf_constants.HF_HUB_CACHE) / (
"models--" + CHATTERBOX_TURBO_HF_REPO.replace("/", "--")
)
if not repo_cache.exists():
return False
blobs_dir = repo_cache / "blobs"
if blobs_dir.exists() and any(blobs_dir.glob("*.incomplete")):
return False
# Check for turbo weight files
snapshots_dir = repo_cache / "snapshots"
if snapshots_dir.exists():
for fname in _TURBO_WEIGHT_FILES:
if not any(snapshots_dir.rglob(fname)):
return False
return True
return False
except Exception as e:
logger.warning(f"Error checking Chatterbox Turbo cache: {e}")
return False
async def load_model(self, model_size: str = "default") -> None: async def load_model(self, model_size: str = "default") -> None:
"""Load the Chatterbox Turbo model.""" """Load the Chatterbox Turbo model."""
@@ -103,59 +70,24 @@ class ChatterboxTurboTTSBackend:
def _load_model_sync(self): def _load_model_sync(self):
"""Synchronous model loading.""" """Synchronous model loading."""
from ..utils.hf_progress import HFProgressTracker, create_hf_progress_callback
progress_manager = get_progress_manager()
task_manager = get_task_manager()
model_name = "chatterbox-turbo" model_name = "chatterbox-turbo"
is_cached = self._is_model_cached() is_cached = self._is_model_cached()
# Set up HF progress tracking (intercepts tqdm for file-level progress) with model_load_progress(model_name, is_cached):
progress_callback = create_hf_progress_callback(model_name, progress_manager)
tracker = HFProgressTracker(progress_callback, filter_non_downloads=is_cached)
tracker_context = tracker.patch_download()
tracker_context.__enter__()
if not is_cached:
task_manager.start_download(model_name)
progress_manager.update_progress(
model_name=model_name,
current=0,
total=0,
filename="Connecting to HuggingFace...",
status="downloading",
)
try:
device = self._get_device() device = self._get_device()
self._device = device self._device = device
logger.info(f"Loading Chatterbox Turbo TTS on {device}...") logger.info(f"Loading Chatterbox Turbo TTS on {device}...")
import torch import torch
from huggingface_hub import snapshot_download from huggingface_hub import snapshot_download
from chatterbox.tts_turbo import ChatterboxTurboTTS from chatterbox.tts_turbo import ChatterboxTurboTTS
# Download model files ourselves so we can pass token=None local_path = snapshot_download(
# (upstream from_pretrained passes token=True which requires repo_id=CHATTERBOX_TURBO_HF_REPO,
# a stored HF token even though the repo is public). token=None,
try: allow_patterns=["*.safetensors", "*.json", "*.txt", "*.pt", "*.model"],
local_path = snapshot_download( )
repo_id=CHATTERBOX_TURBO_HF_REPO,
token=None,
allow_patterns=[
"*.safetensors", "*.json", "*.txt", "*.pt", "*.model",
],
)
finally:
tracker_context.__exit__(None, None, None)
# Monkey-patch torch.load for CPU loading. The model's .pt files
# were saved on CUDA; from_local() doesn't pass map_location
# so loading on CPU fails without this.
# Load into a local var, apply patches, then publish to
# self.model so a failed patch doesn't leave us half-initialised.
if device == "cpu": if device == "cpu":
_orig_torch_load = torch.load _orig_torch_load = torch.load
@@ -166,73 +98,16 @@ class ChatterboxTurboTTSBackend:
with ChatterboxTurboTTSBackend._load_lock: with ChatterboxTurboTTSBackend._load_lock:
torch.load = _patched_load torch.load = _patched_load
try: try:
model = ChatterboxTurboTTS.from_local( model = ChatterboxTurboTTS.from_local(local_path, device)
local_path, device,
)
finally: finally:
torch.load = _orig_torch_load torch.load = _orig_torch_load
else: else:
model = ChatterboxTurboTTS.from_local( model = ChatterboxTurboTTS.from_local(local_path, device)
local_path, device,
)
if not is_cached: patch_chatterbox_f32(model)
progress_manager.mark_complete(model_name)
task_manager.complete_download(model_name)
# Patch float64 → float32 dtype mismatches in upstream chatterbox.
# librosa.load returns float64 numpy; multiple upstream code paths
# convert it to a torch tensor via torch.from_numpy() without
# casting, then matmul it against float32 model weights.
# We patch the two known entry points:
#
# 1. S3Tokenizer.log_mel_spectrogram — the audio tensor from
# librosa hits _mel_filters (float32) in a matmul.
# 2. VoiceEncoder.forward — float64 mel spectrograms hit the
# float32 LSTM weights.
import types
# Patch S3Tokenizer (used by s3gen.tokenizer)
_tokzr = model.s3gen.tokenizer
_orig_log_mel = _tokzr.log_mel_spectrogram.__func__
def _f32_log_mel(self_tokzr, audio, padding=0):
import torch as _torch
if _torch.is_tensor(audio):
audio = audio.float()
return _orig_log_mel(self_tokzr, audio, padding)
_tokzr.log_mel_spectrogram = types.MethodType(_f32_log_mel, _tokzr)
# Patch VoiceEncoder
_ve = model.ve
_orig_ve_forward = _ve.forward.__func__
def _f32_ve_forward(self_ve, mels):
return _orig_ve_forward(self_ve, mels.float())
_ve.forward = types.MethodType(_f32_ve_forward, _ve)
# Only publish after all patches succeed
self.model = model self.model = model
logger.info("Chatterbox Turbo TTS loaded successfully") logger.info("Chatterbox Turbo TTS loaded successfully")
except ImportError as e:
logger.error(
"chatterbox-tts package not found. "
"Install with: pip install chatterbox-tts"
)
if not is_cached:
progress_manager.mark_error(model_name, str(e))
task_manager.error_download(model_name, str(e))
raise
except Exception as e:
logger.error(f"Failed to load Chatterbox Turbo: {e}")
if not is_cached:
progress_manager.mark_error(model_name, str(e))
task_manager.error_download(model_name, str(e))
raise
def unload_model(self) -> None: def unload_model(self) -> None:
"""Unload model to free memory.""" """Unload model to free memory."""
@@ -270,17 +145,7 @@ class ChatterboxTurboTTSBackend:
audio_paths: List[str], audio_paths: List[str],
reference_texts: List[str], reference_texts: List[str],
) -> Tuple[np.ndarray, str]: ) -> Tuple[np.ndarray, str]:
"""Combine multiple reference samples.""" return await _combine_voice_prompts(audio_paths, reference_texts)
combined_audio = []
for path in audio_paths:
audio, _sr = load_audio(path)
audio = normalize_audio(audio)
combined_audio.append(audio)
mixed = np.concatenate(combined_audio)
mixed = normalize_audio(mixed)
combined_text = " ".join(reference_texts)
return mixed, combined_text
async def generate( async def generate(
self, self,
+19 -116
View File
@@ -7,16 +7,13 @@ Wraps the LuxTTS (ZipVoice) model for zero-shot voice cloning.
import asyncio import asyncio
import logging import logging
from pathlib import Path from typing import Optional, Tuple
from typing import List, Optional, Tuple
import numpy as np import numpy as np
from . import TTSBackend from . import TTSBackend
from ..utils.audio import normalize_audio, load_audio from .base import is_model_cached, get_torch_device, combine_voice_prompts as _combine_voice_prompts, model_load_progress
from ..utils.cache import get_cache_key, get_cached_voice_prompt, cache_voice_prompt from ..utils.cache import get_cache_key, get_cached_voice_prompt, cache_voice_prompt
from ..utils.progress import get_progress_manager
from ..utils.tasks import get_task_manager
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
@@ -33,14 +30,7 @@ class LuxTTSBackend:
self._device = None self._device = None
def _get_device(self) -> str: def _get_device(self) -> str:
"""Get the best available device.""" return get_torch_device(allow_mps=True)
import torch
if torch.cuda.is_available():
return "cuda"
if hasattr(torch.backends, "mps") and torch.backends.mps.is_available():
return "mps"
return "cpu"
def is_loaded(self) -> bool: def is_loaded(self) -> bool:
return self.model is not None return self.model is not None
@@ -55,35 +45,10 @@ class LuxTTSBackend:
return LUXTTS_HF_REPO return LUXTTS_HF_REPO
def _is_model_cached(self, model_size: str = "default") -> bool: def _is_model_cached(self, model_size: str = "default") -> bool:
"""Check if LuxTTS model weights are cached locally.""" return is_model_cached(
try: LUXTTS_HF_REPO,
from huggingface_hub import constants as hf_constants weight_extensions=(".pt", ".safetensors", ".onnx", ".bin"),
)
repo_cache = (
Path(hf_constants.HF_HUB_CACHE)
/ ("models--" + LUXTTS_HF_REPO.replace("/", "--"))
)
if not repo_cache.exists():
return False
blobs_dir = repo_cache / "blobs"
if blobs_dir.exists() and any(blobs_dir.glob("*.incomplete")):
return False
snapshots_dir = repo_cache / "snapshots"
if snapshots_dir.exists():
has_weights = any(snapshots_dir.rglob("*.pt")) or any(
snapshots_dir.rglob("*.safetensors")
) or any(snapshots_dir.rglob("*.onnx")) or any(
snapshots_dir.rglob("*.bin")
)
return has_weights
return False
except Exception as e:
logger.warning(f"Error checking LuxTTS cache: {e}")
return False
async def load_model(self, model_size: str = "default") -> None: async def load_model(self, model_size: str = "default") -> None:
"""Load the LuxTTS model.""" """Load the LuxTTS model."""
@@ -93,67 +58,25 @@ class LuxTTSBackend:
await asyncio.to_thread(self._load_model_sync) await asyncio.to_thread(self._load_model_sync)
def _load_model_sync(self): def _load_model_sync(self):
"""Synchronous model loading."""
from ..utils.hf_progress import HFProgressTracker, create_hf_progress_callback
progress_manager = get_progress_manager()
task_manager = get_task_manager()
model_name = "luxtts" model_name = "luxtts"
is_cached = self._is_model_cached() is_cached = self._is_model_cached()
# Set up HF progress tracking (intercepts tqdm for file-level progress) with model_load_progress(model_name, is_cached):
progress_callback = create_hf_progress_callback(model_name, progress_manager)
tracker = HFProgressTracker(progress_callback, filter_non_downloads=is_cached)
tracker_context = tracker.patch_download()
tracker_context.__enter__()
if not is_cached:
task_manager.start_download(model_name)
progress_manager.update_progress(
model_name=model_name,
current=0,
total=0,
filename="Connecting to HuggingFace...",
status="downloading",
)
try:
from zipvoice.luxvoice import LuxTTS from zipvoice.luxvoice import LuxTTS
device = self.device device = self.device
logger.info(f"Loading LuxTTS on {device}...") logger.info(f"Loading LuxTTS on {device}...")
# LuxTTS constructor downloads model and loads everything if device == "cpu":
try: import os
if device == "cpu": threads = os.cpu_count() or 4
import os self.model = LuxTTS(
threads = os.cpu_count() or 4 model_path=LUXTTS_HF_REPO, device="cpu", threads=min(threads, 8),
self.model = LuxTTS( )
model_path=LUXTTS_HF_REPO, else:
device="cpu", self.model = LuxTTS(model_path=LUXTTS_HF_REPO, device=device)
threads=min(threads, 8),
)
else:
self.model = LuxTTS(
model_path=LUXTTS_HF_REPO,
device=device,
)
finally:
tracker_context.__exit__(None, None, None)
if not is_cached: logger.info("LuxTTS loaded successfully")
progress_manager.mark_complete(model_name)
task_manager.complete_download(model_name)
logger.info("LuxTTS loaded successfully")
except Exception as e:
logger.error(f"Failed to load LuxTTS: {e}")
if not is_cached:
progress_manager.mark_error(model_name, str(e))
task_manager.error_download(model_name, str(e))
raise
def unload_model(self) -> None: def unload_model(self) -> None:
"""Unload model to free memory.""" """Unload model to free memory."""
@@ -204,28 +127,8 @@ class LuxTTSBackend:
return encoded, False return encoded, False
async def combine_voice_prompts( async def combine_voice_prompts(self, audio_paths, reference_texts):
self, return await _combine_voice_prompts(audio_paths, reference_texts, sample_rate=24000)
audio_paths: List[str],
reference_texts: List[str],
) -> Tuple[np.ndarray, str]:
"""
Combine multiple reference samples.
LuxTTS doesn't have native multi-prompt support, so we concatenate
the audio and let encode_prompt handle the combined clip.
"""
combined_audio = []
for path in audio_paths:
audio, _sr = load_audio(path, sample_rate=24000)
audio = normalize_audio(audio)
combined_audio.append(audio)
mixed = np.concatenate(combined_audio)
mixed = normalize_audio(mixed)
combined_text = " ".join(reference_texts)
return mixed, combined_text
async def generate( async def generate(
self, self,
+105 -338
View File
@@ -4,49 +4,44 @@ MLX backend implementation for TTS and STT using mlx-audio.
from typing import Optional, List, Tuple from typing import Optional, List, Tuple
import asyncio import asyncio
import logging
import numpy as np import numpy as np
import os import os
from pathlib import Path from pathlib import Path
logger = logging.getLogger(__name__)
# PATCH: Import and apply offline patch BEFORE any huggingface_hub usage # PATCH: Import and apply offline patch BEFORE any huggingface_hub usage
# This prevents mlx_audio from making network requests when models are cached # This prevents mlx_audio from making network requests when models are cached
from ..utils.hf_offline_patch import patch_huggingface_hub_offline, ensure_original_qwen_config_cached from ..utils.hf_offline_patch import patch_huggingface_hub_offline, ensure_original_qwen_config_cached
patch_huggingface_hub_offline() patch_huggingface_hub_offline()
ensure_original_qwen_config_cached() ensure_original_qwen_config_cached()
from . import TTSBackend, STTBackend from . import TTSBackend, STTBackend, LANGUAGE_CODE_TO_NAME, WHISPER_HF_REPOS
from .base import is_model_cached, combine_voice_prompts as _combine_voice_prompts, model_load_progress
from ..utils.cache import get_cache_key, get_cached_voice_prompt, cache_voice_prompt from ..utils.cache import get_cache_key, get_cached_voice_prompt, cache_voice_prompt
from ..utils.audio import normalize_audio, load_audio
from ..utils.progress import get_progress_manager
from ..utils.hf_progress import HFProgressTracker, create_hf_progress_callback
from ..utils.tasks import get_task_manager
LANGUAGE_CODE_TO_NAME = {
"zh": "chinese", "en": "english", "ja": "japanese", "ko": "korean",
"de": "german", "fr": "french", "ru": "russian", "pt": "portuguese",
"es": "spanish", "it": "italian",
}
class MLXTTSBackend: class MLXTTSBackend:
"""MLX-based TTS backend using mlx-audio.""" """MLX-based TTS backend using mlx-audio."""
def __init__(self, model_size: str = "1.7B"): def __init__(self, model_size: str = "1.7B"):
self.model = None self.model = None
self.model_size = model_size self.model_size = model_size
self._current_model_size = None self._current_model_size = None
def is_loaded(self) -> bool: def is_loaded(self) -> bool:
"""Check if model is loaded.""" """Check if model is loaded."""
return self.model is not None return self.model is not None
def _get_model_path(self, model_size: str) -> str: def _get_model_path(self, model_size: str) -> str:
""" """
Get the MLX model path. Get the MLX model path.
Args: Args:
model_size: Model size (1.7B or 0.6B) model_size: Model size (1.7B or 0.6B)
Returns: Returns:
HuggingFace Hub model ID for MLX HuggingFace Hub model ID for MLX
""" """
@@ -56,187 +51,90 @@ class MLXTTSBackend:
# 0.6B not yet converted to MLX format # 0.6B not yet converted to MLX format
"0.6B": "mlx-community/Qwen3-TTS-12Hz-1.7B-Base-bf16", # Fallback to 1.7B "0.6B": "mlx-community/Qwen3-TTS-12Hz-1.7B-Base-bf16", # Fallback to 1.7B
} }
if model_size not in mlx_model_map: if model_size not in mlx_model_map:
raise ValueError(f"Unknown model size: {model_size}") raise ValueError(f"Unknown model size: {model_size}")
hf_model_id = mlx_model_map[model_size] hf_model_id = mlx_model_map[model_size]
print(f"Will download MLX model from HuggingFace Hub: {hf_model_id}") logger.info("Will download MLX model from HuggingFace Hub: %s", hf_model_id)
return hf_model_id return hf_model_id
def _is_model_cached(self, model_size: str) -> bool: def _is_model_cached(self, model_size: str) -> bool:
""" return is_model_cached(
Check if the model is already cached locally AND fully downloaded. self._get_model_path(model_size),
weight_extensions=(".safetensors", ".bin", ".npz"),
Args: )
model_size: Model size to check
Returns:
True if model is fully cached, False if missing or incomplete
"""
try:
from huggingface_hub import constants as hf_constants
model_path = self._get_model_path(model_size)
repo_cache = Path(hf_constants.HF_HUB_CACHE) / ("models--" + model_path.replace("/", "--"))
if not repo_cache.exists():
return False
# Check for .incomplete files - if any exist, download is still in progress
blobs_dir = repo_cache / "blobs"
if blobs_dir.exists() and any(blobs_dir.glob("*.incomplete")):
print(f"[_is_model_cached] Found .incomplete files for {model_size}, treating as not cached")
return False
# Check that actual model weight files exist in snapshots
snapshots_dir = repo_cache / "snapshots"
if snapshots_dir.exists():
has_weights = (
any(snapshots_dir.rglob("*.safetensors")) or
any(snapshots_dir.rglob("*.bin")) or
any(snapshots_dir.rglob("*.npz"))
)
if not has_weights:
print(f"[_is_model_cached] No model weights found for {model_size}, treating as not cached")
return False
return True
except Exception as e:
print(f"[_is_model_cached] Error checking cache for {model_size}: {e}")
return False
async def load_model_async(self, model_size: Optional[str] = None): async def load_model_async(self, model_size: Optional[str] = None):
""" """
Lazy load the MLX TTS model. Lazy load the MLX TTS model.
Args: Args:
model_size: Model size to load (1.7B or 0.6B) model_size: Model size to load (1.7B or 0.6B)
""" """
if model_size is None: if model_size is None:
model_size = self.model_size model_size = self.model_size
# If already loaded with correct size, return # If already loaded with correct size, return
if self.model is not None and self._current_model_size == model_size: if self.model is not None and self._current_model_size == model_size:
return return
# Unload existing model if different size requested # Unload existing model if different size requested
if self.model is not None and self._current_model_size != model_size: if self.model is not None and self._current_model_size != model_size:
self.unload_model() self.unload_model()
# Run blocking load in thread pool # Run blocking load in thread pool
await asyncio.to_thread(self._load_model_sync, model_size) await asyncio.to_thread(self._load_model_sync, model_size)
# Alias for compatibility # Alias for compatibility
load_model = load_model_async load_model = load_model_async
def _load_model_sync(self, model_size: str): def _load_model_sync(self, model_size: str):
"""Synchronous model loading.""" """Synchronous model loading."""
model_path = self._get_model_path(model_size)
model_name = f"qwen-tts-{model_size}"
is_cached = self._is_model_cached(model_size)
# Force offline mode when cached to avoid network requests
original_hf_hub_offline = os.environ.get("HF_HUB_OFFLINE")
if is_cached:
os.environ["HF_HUB_OFFLINE"] = "1"
logger.info("[PATCH] Model %s is cached, forcing HF_HUB_OFFLINE=1 to avoid network requests", model_size)
try: try:
# Get model path BEFORE importing mlx_audio with model_load_progress(model_name, is_cached):
model_path = self._get_model_path(model_size) from mlx_audio.tts import load
# Set up progress tracking logger.info("Loading MLX TTS model %s...", model_size)
progress_manager = get_progress_manager()
task_manager = get_task_manager() try:
model_name = f"qwen-tts-{model_size}"
# Check if model is already cached
is_cached = self._is_model_cached(model_size)
# Set up progress callback
# If cached: filter out non-download progress
# If not cached: report all progress (we're actually downloading)
progress_callback = create_hf_progress_callback(model_name, progress_manager)
tracker = HFProgressTracker(progress_callback, filter_non_downloads=is_cached)
print(f"Loading MLX TTS model {model_size}...")
# Only track download progress if model is NOT cached
if not is_cached:
# Start tracking download task
task_manager.start_download(model_name)
# Initialize progress state so SSE endpoint has initial data to send
# This provides immediate feedback while HuggingFace fetches metadata
progress_manager.update_progress(
model_name=model_name,
current=0,
total=0, # Will be updated once actual total is known
filename="Connecting to HuggingFace...",
status="downloading",
)
# IMPORTANT: Patch tqdm BEFORE importing mlx_audio
# Otherwise mlx_audio caches reference to original tqdm
tracker_context = tracker.patch_download()
tracker_context.__enter__()
# PATCH: Force offline mode when model is already cached
# This prevents crashes when HuggingFace is unreachable
original_hf_hub_offline = os.environ.get("HF_HUB_OFFLINE")
if is_cached:
os.environ["HF_HUB_OFFLINE"] = "1"
print(f"[PATCH] Model {model_size} is cached, forcing HF_HUB_OFFLINE=1 to avoid network requests")
# Import mlx_audio AFTER patching tqdm
from mlx_audio.tts import load
# Load MLX model (downloads automatically)
try:
self.model = load(model_path)
except Exception as load_error:
# If offline mode failed, try with network enabled as fallback
if is_cached and "offline" in str(load_error).lower():
print(f"[PATCH] Offline load failed, trying with network: {load_error}")
os.environ.pop("HF_HUB_OFFLINE", None)
self.model = load(model_path) self.model = load(model_path)
else: except Exception as load_error:
raise if is_cached and "offline" in str(load_error).lower():
finally: logger.warning("[PATCH] Offline load failed, trying with network: %s", load_error)
# Exit the patch context os.environ.pop("HF_HUB_OFFLINE", None)
tracker_context.__exit__(None, None, None) self.model = load(model_path)
# Restore original HF_HUB_OFFLINE setting else:
if original_hf_hub_offline is not None: raise
os.environ["HF_HUB_OFFLINE"] = original_hf_hub_offline finally:
else: if original_hf_hub_offline is not None:
os.environ.pop("HF_HUB_OFFLINE", None) os.environ["HF_HUB_OFFLINE"] = original_hf_hub_offline
else:
# Only mark download as complete if we were tracking it os.environ.pop("HF_HUB_OFFLINE", None)
if not is_cached:
progress_manager.mark_complete(model_name) self._current_model_size = model_size
task_manager.complete_download(model_name) self.model_size = model_size
logger.info("MLX TTS model %s loaded successfully", model_size)
self._current_model_size = model_size
self.model_size = model_size
print(f"MLX TTS model {model_size} loaded successfully")
except ImportError as e:
print(f"Error: mlx_audio package not found. Install with: pip install mlx-audio")
progress_manager = get_progress_manager()
task_manager = get_task_manager()
model_name = f"qwen-tts-{model_size}"
progress_manager.mark_error(model_name, str(e))
task_manager.error_download(model_name, str(e))
raise
except Exception as e:
print(f"Error loading MLX TTS model: {e}")
progress_manager = get_progress_manager()
task_manager = get_task_manager()
model_name = f"qwen-tts-{model_size}"
progress_manager.mark_error(model_name, str(e))
task_manager.error_download(model_name, str(e))
raise
def unload_model(self): def unload_model(self):
"""Unload the model to free memory.""" """Unload the model to free memory."""
if self.model is not None: if self.model is not None:
del self.model del self.model
self.model = None self.model = None
self._current_model_size = None self._current_model_size = None
print("MLX TTS model unloaded") logger.info("MLX TTS model unloaded")
async def create_voice_prompt( async def create_voice_prompt(
self, self,
audio_path: str, audio_path: str,
@@ -245,20 +143,20 @@ class MLXTTSBackend:
) -> Tuple[dict, bool]: ) -> Tuple[dict, bool]:
""" """
Create voice prompt from reference audio. Create voice prompt from reference audio.
MLX backend stores voice prompt as a dict with audio path and text. MLX backend stores voice prompt as a dict with audio path and text.
The actual voice prompt processing happens during generation. The actual voice prompt processing happens during generation.
Args: Args:
audio_path: Path to reference audio file audio_path: Path to reference audio file
reference_text: Transcript of reference audio reference_text: Transcript of reference audio
use_cache: Whether to use cached prompt if available use_cache: Whether to use cached prompt if available
Returns: Returns:
Tuple of (voice_prompt_dict, was_cached) Tuple of (voice_prompt_dict, was_cached)
""" """
await self.load_model_async(None) await self.load_model_async(None)
# Check cache if enabled # Check cache if enabled
if use_cache: if use_cache:
cache_key = get_cache_key(audio_path, reference_text) cache_key = get_cache_key(audio_path, reference_text)
@@ -272,53 +170,25 @@ class MLXTTSBackend:
return cached_prompt, True return cached_prompt, True
else: else:
# Cached file no longer exists, invalidate cache # Cached file no longer exists, invalidate cache
print(f"Cached audio file not found: {cached_audio_path}, regenerating prompt") logger.warning("Cached audio file not found: %s, regenerating prompt", cached_audio_path)
# MLX voice prompt format - store audio path and text # MLX voice prompt format - store audio path and text
# The model will process this during generation # The model will process this during generation
voice_prompt_items = { voice_prompt_items = {
"ref_audio": str(audio_path), "ref_audio": str(audio_path),
"ref_text": reference_text, "ref_text": reference_text,
} }
# Cache if enabled # Cache if enabled
if use_cache: if use_cache:
cache_key = get_cache_key(audio_path, reference_text) cache_key = get_cache_key(audio_path, reference_text)
cache_voice_prompt(cache_key, voice_prompt_items) cache_voice_prompt(cache_key, voice_prompt_items)
return voice_prompt_items, False return voice_prompt_items, False
async def combine_voice_prompts( async def combine_voice_prompts(self, audio_paths, reference_texts):
self, return await _combine_voice_prompts(audio_paths, reference_texts)
audio_paths: List[str],
reference_texts: List[str],
) -> Tuple[np.ndarray, str]:
"""
Combine multiple reference samples for better quality.
Args:
audio_paths: List of audio file paths
reference_texts: List of reference texts
Returns:
Tuple of (combined_audio, combined_text)
"""
combined_audio = []
for audio_path in audio_paths:
audio, sr = load_audio(audio_path)
audio = normalize_audio(audio)
combined_audio.append(audio)
# Concatenate audio
mixed = np.concatenate(combined_audio)
mixed = normalize_audio(mixed)
# Combine texts
combined_text = " ".join(reference_texts)
return mixed, combined_text
async def generate( async def generate(
self, self,
text: str, text: str,
@@ -342,7 +212,7 @@ class MLXTTSBackend:
""" """
await self.load_model_async(None) await self.load_model_async(None)
print(f"Generating audio for text: {text}") logger.info("Generating audio for text: %s", text)
def _generate_sync(): def _generate_sync():
"""Run synchronous generation in thread pool.""" """Run synchronous generation in thread pool."""
@@ -354,20 +224,21 @@ class MLXTTSBackend:
# Set seed if provided (MLX uses numpy random) # Set seed if provided (MLX uses numpy random)
if seed is not None: if seed is not None:
import mlx.core as mx import mlx.core as mx
np.random.seed(seed) np.random.seed(seed)
mx.random.seed(seed) mx.random.seed(seed)
# Extract voice prompt info # Extract voice prompt info
ref_audio = voice_prompt.get("ref_audio") or voice_prompt.get("ref_audio_path") ref_audio = voice_prompt.get("ref_audio") or voice_prompt.get("ref_audio_path")
ref_text = voice_prompt.get("ref_text", "") ref_text = voice_prompt.get("ref_text", "")
# Validate that the audio file exists # Validate that the audio file exists
if ref_audio and not Path(ref_audio).exists(): if ref_audio and not Path(ref_audio).exists():
print(f"Warning: Audio file not found: {ref_audio}") logger.warning("Audio file not found: %s", ref_audio)
print("This may be due to a cached voice prompt referencing a deleted temp file.") logger.warning("This may be due to a cached voice prompt referencing a deleted temp file.")
print("Regenerating without voice prompt.") logger.warning("Regenerating without voice prompt.")
ref_audio = None ref_audio = None
# Check if model supports voice cloning via generate method # Check if model supports voice cloning via generate method
# MLX API may support ref_audio parameter directly # MLX API may support ref_audio parameter directly
try: try:
@@ -375,6 +246,7 @@ class MLXTTSBackend:
if ref_audio: if ref_audio:
# Check if generate accepts ref_audio parameter # Check if generate accepts ref_audio parameter
import inspect import inspect
sig = inspect.signature(self.model.generate) sig = inspect.signature(self.model.generate)
if "ref_audio" in sig.parameters: if "ref_audio" in sig.parameters:
# Generate with voice cloning # Generate with voice cloning
@@ -393,18 +265,18 @@ class MLXTTSBackend:
sample_rate = result.sample_rate sample_rate = result.sample_rate
except Exception as e: except Exception as e:
# If voice cloning fails, try without it # If voice cloning fails, try without it
print(f"Warning: Voice cloning failed, generating without voice prompt: {e}") logger.warning("Voice cloning failed, generating without voice prompt: %s", e)
for result in self.model.generate(text, lang_code=lang): for result in self.model.generate(text, lang_code=lang):
audio_chunks.append(np.array(result.audio)) audio_chunks.append(np.array(result.audio))
sample_rate = result.sample_rate sample_rate = result.sample_rate
# Concatenate all chunks # Concatenate all chunks
if audio_chunks: if audio_chunks:
audio = np.concatenate([np.asarray(chunk, dtype=np.float32) for chunk in audio_chunks]) audio = np.concatenate([np.asarray(chunk, dtype=np.float32) for chunk in audio_chunks])
else: else:
# Fallback: empty audio # Fallback: empty audio
audio = np.array([], dtype=np.float32) audio = np.array([], dtype=np.float32)
return audio, sample_rate return audio, sample_rate
# Run blocking inference in thread pool # Run blocking inference in thread pool
@@ -413,167 +285,62 @@ class MLXTTSBackend:
return audio, sample_rate return audio, sample_rate
WHISPER_HF_REPOS = {
"base": "openai/whisper-base",
"small": "openai/whisper-small",
"medium": "openai/whisper-medium",
"large": "openai/whisper-large-v3",
}
class MLXSTTBackend: class MLXSTTBackend:
"""MLX-based STT backend using mlx-audio Whisper.""" """MLX-based STT backend using mlx-audio Whisper."""
def __init__(self, model_size: str = "base"): def __init__(self, model_size: str = "base"):
self.model = None self.model = None
self.model_size = model_size self.model_size = model_size
def is_loaded(self) -> bool: def is_loaded(self) -> bool:
"""Check if model is loaded.""" """Check if model is loaded."""
return self.model is not None return self.model is not None
def _is_model_cached(self, model_size: str) -> bool: def _is_model_cached(self, model_size: str) -> bool:
""" hf_repo = WHISPER_HF_REPOS.get(model_size, f"openai/whisper-{model_size}")
Check if the Whisper model is already cached locally AND fully downloaded. return is_model_cached(hf_repo, weight_extensions=(".safetensors", ".bin", ".npz"))
Args:
model_size: Model size to check
Returns:
True if model is fully cached, False if missing or incomplete
"""
try:
from huggingface_hub import constants as hf_constants
hf_repo = WHISPER_HF_REPOS.get(model_size, f"openai/whisper-{model_size}")
repo_cache = Path(hf_constants.HF_HUB_CACHE) / ("models--" + hf_repo.replace("/", "--"))
if not repo_cache.exists():
return False
# Check for .incomplete files - if any exist, download is still in progress
blobs_dir = repo_cache / "blobs"
if blobs_dir.exists() and any(blobs_dir.glob("*.incomplete")):
print(f"[_is_model_cached] Found .incomplete files for whisper-{model_size}, treating as not cached")
return False
# Check that actual model weight files exist in snapshots
snapshots_dir = repo_cache / "snapshots"
if snapshots_dir.exists():
has_weights = (
any(snapshots_dir.rglob("*.safetensors")) or
any(snapshots_dir.rglob("*.bin")) or
any(snapshots_dir.rglob("*.npz"))
)
if not has_weights:
print(f"[_is_model_cached] No model weights found for whisper-{model_size}, treating as not cached")
return False
return True
except Exception as e:
print(f"[_is_model_cached] Error checking cache for whisper-{model_size}: {e}")
return False
async def load_model_async(self, model_size: Optional[str] = None): async def load_model_async(self, model_size: Optional[str] = None):
""" """
Lazy load the MLX Whisper model. Lazy load the MLX Whisper model.
Args: Args:
model_size: Model size (tiny, base, small, medium, large) model_size: Model size (tiny, base, small, medium, large)
""" """
if model_size is None: if model_size is None:
model_size = self.model_size model_size = self.model_size
if self.model is not None and self.model_size == model_size: if self.model is not None and self.model_size == model_size:
return return
# Run blocking load in thread pool # Run blocking load in thread pool
await asyncio.to_thread(self._load_model_sync, model_size) await asyncio.to_thread(self._load_model_sync, model_size)
# Alias for compatibility # Alias for compatibility
load_model = load_model_async load_model = load_model_async
def _load_model_sync(self, model_size: str): def _load_model_sync(self, model_size: str):
"""Synchronous model loading.""" """Synchronous model loading."""
try: progress_model_name = f"whisper-{model_size}"
progress_manager = get_progress_manager() is_cached = self._is_model_cached(model_size)
task_manager = get_task_manager()
progress_model_name = f"whisper-{model_size}"
# Check if model is already cached with model_load_progress(progress_model_name, is_cached):
is_cached = self._is_model_cached(model_size)
# Set up progress callback and tracker
# If cached: filter out non-download progress
# If not cached: report all progress (we're actually downloading)
progress_callback = create_hf_progress_callback(progress_model_name, progress_manager)
tracker = HFProgressTracker(progress_callback, filter_non_downloads=is_cached)
# Patch tqdm BEFORE importing mlx_audio
tracker_context = tracker.patch_download()
tracker_context.__enter__()
# Import mlx_audio
from mlx_audio.stt import load from mlx_audio.stt import load
# MLX Whisper uses the standard OpenAI models
model_name = WHISPER_HF_REPOS.get(model_size, f"openai/whisper-{model_size}") model_name = WHISPER_HF_REPOS.get(model_size, f"openai/whisper-{model_size}")
logger.info("Loading MLX Whisper model %s...", model_size)
self.model = load(model_name)
print(f"Loading MLX Whisper model {model_size}...") self.model_size = model_size
logger.info("MLX Whisper model %s loaded successfully", model_size)
# Only track download progress if model is NOT cached
if not is_cached:
# Start tracking download task
task_manager.start_download(progress_model_name)
# Initialize progress state so SSE endpoint has initial data to send
progress_manager.update_progress(
model_name=progress_model_name,
current=0,
total=0,
filename="Connecting to HuggingFace...",
status="downloading",
)
# Load the model (tqdm is patched, but filters out non-download progress)
try:
self.model = load(model_name)
finally:
# Exit the patch context
tracker_context.__exit__(None, None, None)
# Only mark download as complete if we were tracking it
if not is_cached:
progress_manager.mark_complete(progress_model_name)
task_manager.complete_download(progress_model_name)
self.model_size = model_size
print(f"MLX Whisper model {model_size} loaded successfully")
except ImportError as e:
print(f"Error: mlx_audio package not found. Install with: pip install mlx-audio")
progress_manager = get_progress_manager()
task_manager = get_task_manager()
progress_model_name = f"whisper-{model_size}"
progress_manager.mark_error(progress_model_name, str(e))
task_manager.error_download(progress_model_name, str(e))
raise
except Exception as e:
print(f"Error loading MLX Whisper model: {e}")
progress_manager = get_progress_manager()
task_manager = get_task_manager()
progress_model_name = f"whisper-{model_size}"
progress_manager.mark_error(progress_model_name, str(e))
task_manager.error_download(progress_model_name, str(e))
raise
def unload_model(self): def unload_model(self):
"""Unload the model to free memory.""" """Unload the model to free memory."""
if self.model is not None: if self.model is not None:
del self.model del self.model
self.model = None self.model = None
print("MLX Whisper model unloaded") logger.info("MLX Whisper model unloaded")
async def transcribe( async def transcribe(
self, self,
audio_path: str, audio_path: str,
+92 -359
View File
@@ -4,67 +4,47 @@ PyTorch backend implementation for TTS and STT.
from typing import Optional, List, Tuple from typing import Optional, List, Tuple
import asyncio import asyncio
import logging
import torch import torch
import numpy as np import numpy as np
from pathlib import Path
from . import TTSBackend, STTBackend logger = logging.getLogger(__name__)
from . import TTSBackend, STTBackend, LANGUAGE_CODE_TO_NAME, WHISPER_HF_REPOS
from .base import (
is_model_cached,
get_torch_device,
combine_voice_prompts as _combine_voice_prompts,
model_load_progress,
)
from ..utils.cache import get_cache_key, get_cached_voice_prompt, cache_voice_prompt from ..utils.cache import get_cache_key, get_cached_voice_prompt, cache_voice_prompt
from ..utils.audio import normalize_audio, load_audio from ..utils.audio import load_audio
from ..utils.progress import get_progress_manager
from ..utils.hf_progress import HFProgressTracker, create_hf_progress_callback
from ..utils.tasks import get_task_manager
LANGUAGE_CODE_TO_NAME = {
"zh": "chinese", "en": "english", "ja": "japanese", "ko": "korean",
"de": "german", "fr": "french", "ru": "russian", "pt": "portuguese",
"es": "spanish", "it": "italian",
}
class PyTorchTTSBackend: class PyTorchTTSBackend:
"""PyTorch-based TTS backend using Qwen3-TTS.""" """PyTorch-based TTS backend using Qwen3-TTS."""
def __init__(self, model_size: str = "1.7B"): def __init__(self, model_size: str = "1.7B"):
self.model = None self.model = None
self.model_size = model_size self.model_size = model_size
self.device = self._get_device() self.device = self._get_device()
self._current_model_size = None self._current_model_size = None
def _get_device(self) -> str: def _get_device(self) -> str:
"""Get the best available device.""" """Get the best available device."""
if torch.cuda.is_available(): return get_torch_device(allow_xpu=True, allow_directml=True)
return "cuda"
# Intel Arc / Intel Xe GPU via intel-extension-for-pytorch (IPEX)
try:
import intel_extension_for_pytorch # noqa: F401
if hasattr(torch, 'xpu') and torch.xpu.is_available():
return "xpu"
except ImportError:
pass
# Any GPU on Windows via DirectML (torch-directml)
try:
import torch_directml
if torch_directml.device_count() > 0:
return torch_directml.device(0)
except ImportError:
pass
# MPS (Apple Silicon) — kept for completeness but MLX backend is preferred
if hasattr(torch.backends, 'mps') and torch.backends.mps.is_available():
return "cpu" # MPS disabled for stability; MLX backend handles Apple Silicon
return "cpu"
def is_loaded(self) -> bool: def is_loaded(self) -> bool:
"""Check if model is loaded.""" """Check if model is loaded."""
return self.model is not None return self.model is not None
def _get_model_path(self, model_size: str) -> str: def _get_model_path(self, model_size: str) -> str:
""" """
Get the HuggingFace Hub model ID. Get the HuggingFace Hub model ID.
Args: Args:
model_size: Model size (1.7B or 0.6B) model_size: Model size (1.7B or 0.6B)
Returns: Returns:
HuggingFace Hub model ID HuggingFace Hub model ID
""" """
@@ -72,179 +52,79 @@ class PyTorchTTSBackend:
"1.7B": "Qwen/Qwen3-TTS-12Hz-1.7B-Base", "1.7B": "Qwen/Qwen3-TTS-12Hz-1.7B-Base",
"0.6B": "Qwen/Qwen3-TTS-12Hz-0.6B-Base", "0.6B": "Qwen/Qwen3-TTS-12Hz-0.6B-Base",
} }
if model_size not in hf_model_map: if model_size not in hf_model_map:
raise ValueError(f"Unknown model size: {model_size}") raise ValueError(f"Unknown model size: {model_size}")
return hf_model_map[model_size] return hf_model_map[model_size]
def _is_model_cached(self, model_size: str) -> bool: def _is_model_cached(self, model_size: str) -> bool:
""" return is_model_cached(self._get_model_path(model_size))
Check if the model is already cached locally AND fully downloaded.
Args:
model_size: Model size to check
Returns:
True if model is fully cached, False if missing or incomplete
"""
try:
from huggingface_hub import constants as hf_constants
model_path = self._get_model_path(model_size)
repo_cache = Path(hf_constants.HF_HUB_CACHE) / ("models--" + model_path.replace("/", "--"))
if not repo_cache.exists():
return False
# Check for .incomplete files - if any exist, download is still in progress
blobs_dir = repo_cache / "blobs"
if blobs_dir.exists() and any(blobs_dir.glob("*.incomplete")):
print(f"[_is_model_cached] Found .incomplete files for {model_size}, treating as not cached")
return False
# Check that actual model weight files exist in snapshots
snapshots_dir = repo_cache / "snapshots"
if snapshots_dir.exists():
has_weights = (
any(snapshots_dir.rglob("*.safetensors")) or
any(snapshots_dir.rglob("*.bin"))
)
if not has_weights:
print(f"[_is_model_cached] No model weights found for {model_size}, treating as not cached")
return False
return True
except Exception as e:
print(f"[_is_model_cached] Error checking cache for {model_size}: {e}")
return False
async def load_model_async(self, model_size: Optional[str] = None): async def load_model_async(self, model_size: Optional[str] = None):
""" """
Lazy load the TTS model with automatic downloading from HuggingFace Hub. Lazy load the TTS model with automatic downloading from HuggingFace Hub.
Args: Args:
model_size: Model size to load (1.7B or 0.6B) model_size: Model size to load (1.7B or 0.6B)
""" """
if model_size is None: if model_size is None:
model_size = self.model_size model_size = self.model_size
# If already loaded with correct size, return # If already loaded with correct size, return
if self.model is not None and self._current_model_size == model_size: if self.model is not None and self._current_model_size == model_size:
return return
# Unload existing model if different size requested # Unload existing model if different size requested
if self.model is not None and self._current_model_size != model_size: if self.model is not None and self._current_model_size != model_size:
self.unload_model() self.unload_model()
# Run blocking load in thread pool # Run blocking load in thread pool
await asyncio.to_thread(self._load_model_sync, model_size) await asyncio.to_thread(self._load_model_sync, model_size)
# Alias for compatibility # Alias for compatibility
load_model = load_model_async load_model = load_model_async
def _load_model_sync(self, model_size: str): def _load_model_sync(self, model_size: str):
"""Synchronous model loading.""" """Synchronous model loading."""
try: model_name = f"qwen-tts-{model_size}"
progress_manager = get_progress_manager() is_cached = self._is_model_cached(model_size)
task_manager = get_task_manager()
model_name = f"qwen-tts-{model_size}"
# Check if model is already cached with model_load_progress(model_name, is_cached):
is_cached = self._is_model_cached(model_size)
# Set up progress callback and tracker
# If cached: filter out non-download progress (like "Segment 1/1" during generation)
# If not cached: report all progress (we're actually downloading)
progress_callback = create_hf_progress_callback(model_name, progress_manager)
tracker = HFProgressTracker(progress_callback, filter_non_downloads=is_cached)
# Patch tqdm BEFORE importing qwen_tts
tracker_context = tracker.patch_download()
tracker_context.__enter__()
# Import qwen_tts
from qwen_tts import Qwen3TTSModel from qwen_tts import Qwen3TTSModel
# Get model path (local or HuggingFace Hub ID)
model_path = self._get_model_path(model_size) model_path = self._get_model_path(model_size)
logger.info("Loading TTS model %s on %s...", model_size, self.device)
print(f"Loading TTS model {model_size} on {self.device}...") if self.device == "cpu":
self.model = Qwen3TTSModel.from_pretrained(
# Only track download progress if model is NOT cached model_path,
if not is_cached: torch_dtype=torch.float32,
# Start tracking download task low_cpu_mem_usage=False,
task_manager.start_download(model_name) )
else:
# Initialize progress state so SSE endpoint has initial data to send self.model = Qwen3TTSModel.from_pretrained(
progress_manager.update_progress( model_path,
model_name=model_name, device_map=self.device,
current=0, torch_dtype=torch.bfloat16,
total=0, # Will be updated once actual total is known
filename="Connecting to HuggingFace...",
status="downloading",
) )
# Load the model (tqdm is patched, but filters out non-download progress) self._current_model_size = model_size
try: self.model_size = model_size
# Don't pass device_map on CPU: accelerate's meta-tensor mechanism logger.info("TTS model %s loaded successfully", model_size)
# causes "Cannot copy out of meta tensor" when moving to CPU.
# Instead load directly then call .to(device) if needed.
if self.device == "cpu":
self.model = Qwen3TTSModel.from_pretrained(
model_path,
torch_dtype=torch.float32,
low_cpu_mem_usage=False,
)
else:
self.model = Qwen3TTSModel.from_pretrained(
model_path,
device_map=self.device,
torch_dtype=torch.bfloat16,
)
finally:
# Exit the patch context
tracker_context.__exit__(None, None, None)
# Only mark download as complete if we were tracking it
if not is_cached:
progress_manager.mark_complete(model_name)
task_manager.complete_download(model_name)
self._current_model_size = model_size
self.model_size = model_size
print(f"TTS model {model_size} loaded successfully")
except ImportError as e:
print(f"Error: qwen_tts package not found. Install with: pip install git+https://github.com/QwenLM/Qwen3-TTS.git")
progress_manager = get_progress_manager()
task_manager = get_task_manager()
model_name = f"qwen-tts-{model_size}"
progress_manager.mark_error(model_name, str(e))
task_manager.error_download(model_name, str(e))
raise
except Exception as e:
print(f"Error loading TTS model: {e}")
print(f"Tip: The model will be automatically downloaded from HuggingFace Hub on first use.")
progress_manager = get_progress_manager()
task_manager = get_task_manager()
model_name = f"qwen-tts-{model_size}"
progress_manager.mark_error(model_name, str(e))
task_manager.error_download(model_name, str(e))
raise
def unload_model(self): def unload_model(self):
"""Unload the model to free memory.""" """Unload the model to free memory."""
if self.model is not None: if self.model is not None:
del self.model del self.model
self.model = None self.model = None
self._current_model_size = None self._current_model_size = None
if torch.cuda.is_available(): if torch.cuda.is_available():
torch.cuda.empty_cache() torch.cuda.empty_cache()
print("TTS model unloaded") logger.info("TTS model unloaded")
async def create_voice_prompt( async def create_voice_prompt(
self, self,
audio_path: str, audio_path: str,
@@ -253,17 +133,17 @@ class PyTorchTTSBackend:
) -> Tuple[dict, bool]: ) -> Tuple[dict, bool]:
""" """
Create voice prompt from reference audio. Create voice prompt from reference audio.
Args: Args:
audio_path: Path to reference audio file audio_path: Path to reference audio file
reference_text: Transcript of reference audio reference_text: Transcript of reference audio
use_cache: Whether to use cached prompt if available use_cache: Whether to use cached prompt if available
Returns: Returns:
Tuple of (voice_prompt_dict, was_cached) Tuple of (voice_prompt_dict, was_cached)
""" """
await self.load_model_async(None) await self.load_model_async(None)
# Check cache if enabled # Check cache if enabled
if use_cache: if use_cache:
cache_key = get_cache_key(audio_path, reference_text) cache_key = get_cache_key(audio_path, reference_text)
@@ -279,7 +159,7 @@ class PyTorchTTSBackend:
# Legacy cache format - convert to dict # Legacy cache format - convert to dict
# This shouldn't happen in practice, but handle it # This shouldn't happen in practice, but handle it
return {"prompt": cached_prompt}, True return {"prompt": cached_prompt}, True
def _create_prompt_sync(): def _create_prompt_sync():
"""Run synchronous voice prompt creation in thread pool.""" """Run synchronous voice prompt creation in thread pool."""
return self.model.create_voice_clone_prompt( return self.model.create_voice_clone_prompt(
@@ -287,48 +167,24 @@ class PyTorchTTSBackend:
ref_text=reference_text, ref_text=reference_text,
x_vector_only_mode=False, x_vector_only_mode=False,
) )
# Run blocking operation in thread pool # Run blocking operation in thread pool
voice_prompt_items = await asyncio.to_thread(_create_prompt_sync) voice_prompt_items = await asyncio.to_thread(_create_prompt_sync)
# Cache if enabled # Cache if enabled
if use_cache: if use_cache:
cache_key = get_cache_key(audio_path, reference_text) cache_key = get_cache_key(audio_path, reference_text)
cache_voice_prompt(cache_key, voice_prompt_items) cache_voice_prompt(cache_key, voice_prompt_items)
return voice_prompt_items, False return voice_prompt_items, False
async def combine_voice_prompts( async def combine_voice_prompts(
self, self,
audio_paths: List[str], audio_paths: List[str],
reference_texts: List[str], reference_texts: List[str],
) -> Tuple[np.ndarray, str]: ) -> Tuple[np.ndarray, str]:
""" return await _combine_voice_prompts(audio_paths, reference_texts)
Combine multiple reference samples for better quality.
Args:
audio_paths: List of audio file paths
reference_texts: List of reference texts
Returns:
Tuple of (combined_audio, combined_text)
"""
combined_audio = []
for audio_path in audio_paths:
audio, sr = load_audio(audio_path)
audio = normalize_audio(audio)
combined_audio.append(audio)
# Concatenate audio
mixed = np.concatenate(combined_audio)
mixed = normalize_audio(mixed)
# Combine texts
combined_text = " ".join(reference_texts)
return mixed, combined_text
async def generate( async def generate(
self, self,
text: str, text: str,
@@ -376,15 +232,6 @@ class PyTorchTTSBackend:
return audio, sample_rate return audio, sample_rate
WHISPER_HF_REPOS = {
"base": "openai/whisper-base",
"small": "openai/whisper-small",
"medium": "openai/whisper-medium",
"large": "openai/whisper-large-v3",
"turbo": "openai/whisper-large-v3-turbo",
}
class PyTorchSTTBackend: class PyTorchSTTBackend:
"""PyTorch-based STT backend using Whisper.""" """PyTorch-based STT backend using Whisper."""
@@ -393,72 +240,18 @@ class PyTorchSTTBackend:
self.processor = None self.processor = None
self.model_size = model_size self.model_size = model_size
self.device = self._get_device() self.device = self._get_device()
def _get_device(self) -> str: def _get_device(self) -> str:
"""Get the best available device.""" """Get the best available device."""
if torch.cuda.is_available(): return get_torch_device(allow_xpu=True, allow_directml=True)
return "cuda"
# Intel Arc / Intel Xe GPU via intel-extension-for-pytorch (IPEX)
try:
import intel_extension_for_pytorch # noqa: F401
if hasattr(torch, 'xpu') and torch.xpu.is_available():
return "xpu"
except ImportError:
pass
# Any GPU on Windows via DirectML (torch-directml)
try:
import torch_directml
if torch_directml.device_count() > 0:
return torch_directml.device(0)
except ImportError:
pass
if hasattr(torch.backends, 'mps') and torch.backends.mps.is_available():
return "cpu" # MPS disabled for stability
return "cpu"
def is_loaded(self) -> bool: def is_loaded(self) -> bool:
"""Check if model is loaded.""" """Check if model is loaded."""
return self.model is not None return self.model is not None
def _is_model_cached(self, model_size: str) -> bool: def _is_model_cached(self, model_size: str) -> bool:
""" hf_repo = WHISPER_HF_REPOS.get(model_size, f"openai/whisper-{model_size}")
Check if the Whisper model is already cached locally AND fully downloaded. return is_model_cached(hf_repo)
Args:
model_size: Model size to check
Returns:
True if model is fully cached, False if missing or incomplete
"""
try:
from huggingface_hub import constants as hf_constants
hf_repo = WHISPER_HF_REPOS.get(model_size, f"openai/whisper-{model_size}")
repo_cache = Path(hf_constants.HF_HUB_CACHE) / ("models--" + hf_repo.replace("/", "--"))
if not repo_cache.exists():
return False
# Check for .incomplete files - if any exist, download is still in progress
blobs_dir = repo_cache / "blobs"
if blobs_dir.exists() and any(blobs_dir.glob("*.incomplete")):
print(f"[_is_model_cached] Found .incomplete files for whisper-{model_size}, treating as not cached")
return False
# Check that actual model weight files exist in snapshots
snapshots_dir = repo_cache / "snapshots"
if snapshots_dir.exists():
has_weights = (
any(snapshots_dir.rglob("*.safetensors")) or
any(snapshots_dir.rglob("*.bin"))
)
if not has_weights:
print(f"[_is_model_cached] No model weights found for whisper-{model_size}, treating as not cached")
return False
return True
except Exception as e:
print(f"[_is_model_cached] Error checking cache for whisper-{model_size}: {e}")
return False
async def load_model_async(self, model_size: Optional[str] = None): async def load_model_async(self, model_size: Optional[str] = None):
""" """
@@ -467,95 +260,35 @@ class PyTorchSTTBackend:
Args: Args:
model_size: Model size (tiny, base, small, medium, large) model_size: Model size (tiny, base, small, medium, large)
""" """
print(f"[DEBUG] load_model_async called with size: {model_size}")
if model_size is None: if model_size is None:
model_size = self.model_size model_size = self.model_size
print(f"[DEBUG] Model already loaded? {self.model is not None}, current size: {self.model_size}, requested: {model_size}")
if self.model is not None and self.model_size == model_size: if self.model is not None and self.model_size == model_size:
print(f"[DEBUG] Early return - model already loaded")
return return
print(f"[DEBUG] Calling asyncio.to_thread for _load_model_sync")
# Run blocking load in thread pool
await asyncio.to_thread(self._load_model_sync, model_size) await asyncio.to_thread(self._load_model_sync, model_size)
print(f"[DEBUG] asyncio.to_thread completed")
# Alias for compatibility # Alias for compatibility
load_model = load_model_async load_model = load_model_async
def _load_model_sync(self, model_size: str): def _load_model_sync(self, model_size: str):
"""Synchronous model loading.""" """Synchronous model loading."""
print(f"[DEBUG] _load_model_sync called for Whisper {model_size}") progress_model_name = f"whisper-{model_size}"
try: is_cached = self._is_model_cached(model_size)
progress_manager = get_progress_manager()
task_manager = get_task_manager()
progress_model_name = f"whisper-{model_size}"
# Check if model is already cached with model_load_progress(progress_model_name, is_cached):
is_cached = self._is_model_cached(model_size)
# Set up progress callback and tracker
# If cached: filter out non-download progress
# If not cached: report all progress (we're actually downloading)
progress_callback = create_hf_progress_callback(progress_model_name, progress_manager)
tracker = HFProgressTracker(progress_callback, filter_non_downloads=is_cached)
# Patch tqdm BEFORE importing transformers
print("[DEBUG] Starting tqdm patch BEFORE transformers import")
tracker_context = tracker.patch_download()
tracker_context.__enter__()
print("[DEBUG] tqdm patched, now importing transformers")
# Import transformers
from transformers import WhisperProcessor, WhisperForConditionalGeneration from transformers import WhisperProcessor, WhisperForConditionalGeneration
model_name = WHISPER_HF_REPOS.get(model_size, f"openai/whisper-{model_size}") model_name = WHISPER_HF_REPOS.get(model_size, f"openai/whisper-{model_size}")
print(f"[DEBUG] Model name: {model_name}") logger.info("Loading Whisper model %s on %s...", model_size, self.device)
print(f"Loading Whisper model {model_size} on {self.device}...") self.processor = WhisperProcessor.from_pretrained(model_name)
self.model = WhisperForConditionalGeneration.from_pretrained(model_name)
# Only track download progress if model is NOT cached self.model.to(self.device)
if not is_cached: self.model_size = model_size
# Start tracking download task logger.info("Whisper model %s loaded successfully", model_size)
task_manager.start_download(progress_model_name)
# Initialize progress state so SSE endpoint has initial data to send
progress_manager.update_progress(
model_name=progress_model_name,
current=0,
total=0, # Will be updated once actual total is known
filename="Connecting to HuggingFace...",
status="downloading",
)
# Load models (tqdm is patched, but filters out non-download progress)
try:
self.processor = WhisperProcessor.from_pretrained(model_name)
self.model = WhisperForConditionalGeneration.from_pretrained(model_name)
finally:
# Exit the patch context
tracker_context.__exit__(None, None, None)
# Only mark download as complete if we were tracking it
if not is_cached:
progress_manager.mark_complete(progress_model_name)
task_manager.complete_download(progress_model_name)
self.model.to(self.device)
self.model_size = model_size
print(f"Whisper model {model_size} loaded successfully")
except Exception as e:
print(f"Error loading Whisper model: {e}")
progress_manager = get_progress_manager()
task_manager = get_task_manager()
progress_model_name = f"whisper-{model_size}"
progress_manager.mark_error(progress_model_name, str(e))
task_manager.error_download(progress_model_name, str(e))
raise
def unload_model(self): def unload_model(self):
"""Unload the model to free memory.""" """Unload the model to free memory."""
if self.model is not None: if self.model is not None:
@@ -563,12 +296,12 @@ class PyTorchSTTBackend:
del self.processor del self.processor
self.model = None self.model = None
self.processor = None self.processor = None
if torch.cuda.is_available(): if torch.cuda.is_available():
torch.cuda.empty_cache() torch.cuda.empty_cache()
print("Whisper model unloaded") logger.info("Whisper model unloaded")
async def transcribe( async def transcribe(
self, self,
audio_path: str, audio_path: str,
@@ -576,21 +309,21 @@ class PyTorchSTTBackend:
) -> str: ) -> str:
""" """
Transcribe audio to text. Transcribe audio to text.
Args: Args:
audio_path: Path to audio file audio_path: Path to audio file
language: Optional language hint (en or zh) language: Optional language hint (en or zh)
Returns: Returns:
Transcribed text Transcribed text
""" """
await self.load_model_async(None) await self.load_model_async(None)
def _transcribe_sync(): def _transcribe_sync():
"""Run synchronous transcription in thread pool.""" """Run synchronous transcription in thread pool."""
# Load audio # Load audio
audio, sr = load_audio(audio_path, sample_rate=16000) audio, sr = load_audio(audio_path, sample_rate=16000)
# Process audio # Process audio
inputs = self.processor( inputs = self.processor(
audio, audio,
@@ -598,7 +331,7 @@ class PyTorchSTTBackend:
return_tensors="pt", return_tensors="pt",
) )
inputs = inputs.to(self.device) inputs = inputs.to(self.device)
# Generate transcription # Generate transcription
# If language is provided, force it; otherwise let Whisper auto-detect # If language is provided, force it; otherwise let Whisper auto-detect
generate_kwargs = {} generate_kwargs = {}
@@ -608,20 +341,20 @@ class PyTorchSTTBackend:
task="transcribe", task="transcribe",
) )
generate_kwargs["forced_decoder_ids"] = forced_decoder_ids generate_kwargs["forced_decoder_ids"] = forced_decoder_ids
with torch.no_grad(): with torch.no_grad():
predicted_ids = self.model.generate( predicted_ids = self.model.generate(
inputs["input_features"], inputs["input_features"],
**generate_kwargs, **generate_kwargs,
) )
# Decode # Decode
transcription = self.processor.batch_decode( transcription = self.processor.batch_decode(
predicted_ids, predicted_ids,
skip_special_tokens=True, skip_special_tokens=True,
)[0] )[0]
return transcription.strip() return transcription.strip()
# Run blocking transcription in thread pool # Run blocking transcription in thread pool
return await asyncio.to_thread(_transcribe_sync) return await asyncio.to_thread(_transcribe_sync)
+285 -82
View File
@@ -8,10 +8,14 @@ Usage:
import PyInstaller.__main__ import PyInstaller.__main__
import argparse import argparse
import logging
import os import os
import platform import platform
import sys
from pathlib import Path from pathlib import Path
logger = logging.getLogger(__name__)
def is_apple_silicon(): def is_apple_silicon():
"""Check if running on Apple Silicon.""" """Check if running on Apple Silicon."""
@@ -27,113 +31,312 @@ def build_server(cuda=False):
""" """
backend_dir = Path(__file__).parent backend_dir = Path(__file__).parent
binary_name = 'voicebox-server-cuda' if cuda else 'voicebox-server' binary_name = "voicebox-server-cuda" if cuda else "voicebox-server"
# PyInstaller arguments # PyInstaller arguments
args = [ args = [
'server.py', # Use server.py as entry point instead of main.py "server.py", # Use server.py as entry point instead of main.py
'--onefile', "--onefile",
'--name', binary_name, "--name",
binary_name,
] ]
# Hide console window on Windows only. On macOS/Linux the sidecar needs
# stdout/stderr for Tauri to capture logs.
if platform.system() == "Windows":
args.append("--noconsole")
# Add local qwen_tts path if specified (for editable installs) # Add local qwen_tts path if specified (for editable installs)
qwen_tts_path = os.getenv('QWEN_TTS_PATH') qwen_tts_path = os.getenv("QWEN_TTS_PATH")
if qwen_tts_path and Path(qwen_tts_path).exists(): if qwen_tts_path and Path(qwen_tts_path).exists():
args.extend(['--paths', str(qwen_tts_path)]) args.extend(["--paths", str(qwen_tts_path)])
print(f"Using local qwen_tts source from: {qwen_tts_path}") logger.info("Using local qwen_tts source from: %s", qwen_tts_path)
# Add common hidden imports # Add common hidden imports
args.extend([ args.extend(
'--hidden-import', 'backend', [
'--hidden-import', 'backend.main', "--hidden-import",
'--hidden-import', 'backend.config', "backend",
'--hidden-import', 'backend.database', "--hidden-import",
'--hidden-import', 'backend.models', "backend.main",
'--hidden-import', 'backend.profiles', "--hidden-import",
'--hidden-import', 'backend.history', "backend.config",
'--hidden-import', 'backend.tts', "--hidden-import",
'--hidden-import', 'backend.transcribe', "backend.database",
'--hidden-import', 'backend.platform_detect', "--hidden-import",
'--hidden-import', 'backend.backends', "backend.models",
'--hidden-import', 'backend.backends.pytorch_backend', "--hidden-import",
'--hidden-import', 'backend.utils.audio', "backend.services.profiles",
'--hidden-import', 'backend.utils.cache', "--hidden-import",
'--hidden-import', 'backend.utils.progress', "backend.services.history",
'--hidden-import', 'backend.utils.hf_progress', "--hidden-import",
'--hidden-import', 'backend.utils.validation', "backend.services.tts",
'--hidden-import', 'backend.cuda_download', "--hidden-import",
'--hidden-import', 'torch', "backend.services.transcribe",
'--hidden-import', 'transformers', "--hidden-import",
'--hidden-import', 'fastapi', "backend.utils.platform_detect",
'--hidden-import', 'uvicorn', "--hidden-import",
'--hidden-import', 'sqlalchemy', "backend.backends",
'--hidden-import', 'librosa', "--hidden-import",
'--hidden-import', 'soundfile', "backend.backends.pytorch_backend",
'--hidden-import', 'qwen_tts', "--hidden-import",
'--hidden-import', 'qwen_tts.inference', "backend.utils.audio",
'--hidden-import', 'qwen_tts.inference.qwen3_tts_model', "--hidden-import",
'--hidden-import', 'qwen_tts.inference.qwen3_tts_tokenizer', "backend.utils.cache",
'--hidden-import', 'qwen_tts.core', "--hidden-import",
'--hidden-import', 'qwen_tts.cli', "backend.utils.progress",
'--copy-metadata', 'qwen-tts', "--hidden-import",
'--collect-submodules', 'qwen_tts', "backend.utils.hf_progress",
'--collect-data', 'qwen_tts', "--hidden-import",
# Fix for pkg_resources and jaraco namespace packages "backend.services.cuda",
'--hidden-import', 'pkg_resources.extern', "--hidden-import",
'--collect-submodules', 'jaraco', "backend.services.effects",
]) "--hidden-import",
"backend.utils.effects",
"--hidden-import",
"backend.services.versions",
"--hidden-import",
"pedalboard",
"--hidden-import",
"chatterbox",
"--hidden-import",
"chatterbox.tts_turbo",
"--hidden-import",
"chatterbox.mtl_tts",
"--hidden-import",
"backend.backends.chatterbox_backend",
"--hidden-import",
"backend.backends.chatterbox_turbo_backend",
"--hidden-import",
"backend.backends.luxtts_backend",
"--hidden-import",
"zipvoice",
"--hidden-import",
"zipvoice.luxvoice",
"--collect-all",
"zipvoice",
"--collect-all",
"linacodec",
"--hidden-import",
"torch",
"--hidden-import",
"transformers",
"--hidden-import",
"fastapi",
"--hidden-import",
"uvicorn",
"--hidden-import",
"sqlalchemy",
"--hidden-import",
"librosa",
"--hidden-import",
"soundfile",
"--hidden-import",
"qwen_tts",
"--hidden-import",
"qwen_tts.inference",
"--hidden-import",
"qwen_tts.inference.qwen3_tts_model",
"--hidden-import",
"qwen_tts.inference.qwen3_tts_tokenizer",
"--hidden-import",
"qwen_tts.core",
"--hidden-import",
"qwen_tts.cli",
"--copy-metadata",
"qwen-tts",
"--copy-metadata",
"requests",
"--copy-metadata",
"transformers",
"--copy-metadata",
"huggingface-hub",
"--copy-metadata",
"tokenizers",
"--copy-metadata",
"safetensors",
"--copy-metadata",
"tqdm",
"--hidden-import",
"requests",
"--collect-submodules",
"qwen_tts",
"--collect-data",
"qwen_tts",
# Fix for pkg_resources and jaraco namespace packages
"--hidden-import",
"pkg_resources.extern",
"--collect-submodules",
"jaraco",
# inflect uses typeguard @typechecked which calls inspect.getsource()
# at import time — needs .py source files, not just .pyc bytecode
"--collect-all",
"inflect",
# perth ships pretrained watermark model files (hparams.yaml, .pth.tar)
# in perth/perth_net/pretrained/ — needed by chatterbox at runtime
"--collect-all",
"perth",
# piper_phonemize ships espeak-ng-data/ (phoneme tables, language dicts)
# needed by LuxTTS for text-to-phoneme conversion
"--collect-all",
"piper_phonemize",
]
)
# Add CUDA-specific hidden imports # Add CUDA-specific hidden imports
if cuda: if cuda:
print("Building with CUDA support") logger.info("Building with CUDA support")
args.extend([ args.extend(
'--hidden-import', 'torch.cuda', [
'--hidden-import', 'torch.backends.cudnn', "--hidden-import",
]) "torch.cuda",
"--hidden-import",
"torch.backends.cudnn",
]
)
else:
# Exclude NVIDIA CUDA packages from CPU-only builds to keep binary small.
# When building from a venv with CUDA torch installed, PyInstaller would
# bundle ~3GB of NVIDIA shared libraries. We exclude both the Python
# modules and the binary DLLs.
nvidia_packages = [
"nvidia",
"nvidia.cublas",
"nvidia.cuda_cupti",
"nvidia.cuda_nvrtc",
"nvidia.cuda_runtime",
"nvidia.cudnn",
"nvidia.cufft",
"nvidia.curand",
"nvidia.cusolver",
"nvidia.cusparse",
"nvidia.nccl",
"nvidia.nvjitlink",
"nvidia.nvtx",
]
for pkg in nvidia_packages:
args.extend(["--exclude-module", pkg])
# Add MLX-specific imports if building on Apple Silicon (never for CUDA builds) # Add MLX-specific imports if building on Apple Silicon (never for CUDA builds)
if is_apple_silicon() and not cuda: if is_apple_silicon() and not cuda:
print("Building for Apple Silicon - including MLX dependencies") logger.info("Building for Apple Silicon - including MLX dependencies")
args.extend([ args.extend(
'--hidden-import', 'backend.backends.mlx_backend', [
'--hidden-import', 'mlx', "--hidden-import",
'--hidden-import', 'mlx.core', "backend.backends.mlx_backend",
'--hidden-import', 'mlx.nn', "--hidden-import",
'--hidden-import', 'mlx_audio', "mlx",
'--hidden-import', 'mlx_audio.tts', "--hidden-import",
'--hidden-import', 'mlx_audio.stt', "mlx.core",
'--collect-submodules', 'mlx', "--hidden-import",
'--collect-submodules', 'mlx_audio', "mlx.nn",
# Use --collect-all so PyInstaller bundles both data files AND "--hidden-import",
# native shared libraries (.dylib, .metallib) for MLX. "mlx_audio",
# Previously only --collect-data was used, which caused MLX to "--hidden-import",
# raise OSError at runtime inside the bundled binary because "mlx_audio.tts",
# the Metal shader libraries were missing. "--hidden-import",
'--collect-all', 'mlx', "mlx_audio.stt",
'--collect-all', 'mlx_audio', "--collect-submodules",
]) "mlx",
"--collect-submodules",
"mlx_audio",
# Use --collect-all so PyInstaller bundles both data files AND
# native shared libraries (.dylib, .metallib) for MLX.
# Previously only --collect-data was used, which caused MLX to
# raise OSError at runtime inside the bundled binary because
# the Metal shader libraries were missing.
"--collect-all",
"mlx",
"--collect-all",
"mlx_audio",
]
)
elif not cuda: elif not cuda:
print("Building for non-Apple Silicon platform - PyTorch only") logger.info("Building for non-Apple Silicon platform - PyTorch only")
args.extend([ dist_dir = str(backend_dir / "dist")
'--noconfirm', build_dir = str(backend_dir / "build")
'--clean',
]) args.extend(
[
"--distpath",
dist_dir,
"--workpath",
build_dir,
"--noconfirm",
"--clean",
]
)
# Change to backend directory # Change to backend directory
os.chdir(backend_dir) os.chdir(backend_dir)
# For CPU builds on Windows, ensure we're using CPU-only torch.
# If CUDA torch is installed (local dev), swap to CPU torch before building,
# then restore CUDA torch after. This prevents PyInstaller from bundling
# ~3GB of CUDA DLLs into the CPU binary.
restore_cuda = False
if not cuda and platform.system() == "Windows":
import subprocess
result = subprocess.run(
[sys.executable, "-c", "import torch; print(torch.version.cuda or '')"], capture_output=True, text=True
)
has_cuda_torch = bool(result.stdout.strip())
if has_cuda_torch:
logger.info("CUDA torch detected — installing CPU torch for CPU build...")
subprocess.run(
[
sys.executable,
"-m",
"pip",
"install",
"torch",
"torchvision",
"torchaudio",
"--index-url",
"https://download.pytorch.org/whl/cpu",
"--force-reinstall",
"-q",
],
check=True,
)
restore_cuda = True
# Run PyInstaller # Run PyInstaller
PyInstaller.__main__.run(args) try:
PyInstaller.__main__.run(args)
print(f"Binary built in {backend_dir / 'dist' / binary_name}") finally:
# Restore CUDA torch if we swapped it out (even on build failure)
if restore_cuda:
logger.info("Restoring CUDA torch...")
import subprocess
subprocess.run(
[
sys.executable,
"-m",
"pip",
"install",
"torch",
"torchvision",
"torchaudio",
"--index-url",
"https://download.pytorch.org/whl/cu126",
"--force-reinstall",
"-q",
],
check=True,
)
logger.info("Binary built in %s", backend_dir / "dist" / binary_name)
if __name__ == '__main__': if __name__ == "__main__":
parser = argparse.ArgumentParser(description="Build voicebox-server binary") parser = argparse.ArgumentParser(description="Build voicebox-server binary")
parser.add_argument( parser.add_argument(
'--cuda', "--cuda",
action='store_true', action="store_true",
help="Build CUDA-enabled binary (voicebox-server-cuda)", help="Build CUDA-enabled binary (voicebox-server-cuda)",
) )
cli_args = parser.parse_args() cli_args = parser.parse_args()
+12 -2
View File
@@ -4,20 +4,24 @@ Configuration module for voicebox backend.
Handles data directory configuration for production bundling. Handles data directory configuration for production bundling.
""" """
import logging
import os import os
from pathlib import Path from pathlib import Path
logger = logging.getLogger(__name__)
# Allow users to override the HuggingFace model download directory. # Allow users to override the HuggingFace model download directory.
# Set VOICEBOX_MODELS_DIR to an absolute path before starting the server. # Set VOICEBOX_MODELS_DIR to an absolute path before starting the server.
# This sets HF_HUB_CACHE so all huggingface_hub downloads go to that path. # This sets HF_HUB_CACHE so all huggingface_hub downloads go to that path.
_custom_models_dir = os.environ.get("VOICEBOX_MODELS_DIR") _custom_models_dir = os.environ.get("VOICEBOX_MODELS_DIR")
if _custom_models_dir: if _custom_models_dir:
os.environ["HF_HUB_CACHE"] = _custom_models_dir os.environ["HF_HUB_CACHE"] = _custom_models_dir
print(f"[config] Model download path set to: {_custom_models_dir}") logger.info("Model download path set to: %s", _custom_models_dir)
# Default data directory (used in development) # Default data directory (used in development)
_data_dir = Path("data") _data_dir = Path("data")
def set_data_dir(path: str | Path): def set_data_dir(path: str | Path):
""" """
Set the data directory path. Set the data directory path.
@@ -28,7 +32,8 @@ def set_data_dir(path: str | Path):
global _data_dir global _data_dir
_data_dir = Path(path) _data_dir = Path(path)
_data_dir.mkdir(parents=True, exist_ok=True) _data_dir.mkdir(parents=True, exist_ok=True)
print(f"Data directory set to: {_data_dir.absolute()}") logger.info("Data directory set to: %s", _data_dir.absolute())
def get_data_dir() -> Path: def get_data_dir() -> Path:
""" """
@@ -39,28 +44,33 @@ def get_data_dir() -> Path:
""" """
return _data_dir return _data_dir
def get_db_path() -> Path: def get_db_path() -> Path:
"""Get database file path.""" """Get database file path."""
return _data_dir / "voicebox.db" return _data_dir / "voicebox.db"
def get_profiles_dir() -> Path: def get_profiles_dir() -> Path:
"""Get profiles directory path.""" """Get profiles directory path."""
path = _data_dir / "profiles" path = _data_dir / "profiles"
path.mkdir(parents=True, exist_ok=True) path.mkdir(parents=True, exist_ok=True)
return path return path
def get_generations_dir() -> Path: def get_generations_dir() -> Path:
"""Get generations directory path.""" """Get generations directory path."""
path = _data_dir / "generations" path = _data_dir / "generations"
path.mkdir(parents=True, exist_ok=True) path.mkdir(parents=True, exist_ok=True)
return path return path
def get_cache_dir() -> Path: def get_cache_dir() -> Path:
"""Get cache directory path.""" """Get cache directory path."""
path = _data_dir / "cache" path = _data_dir / "cache"
path.mkdir(parents=True, exist_ok=True) path.mkdir(parents=True, exist_ok=True)
return path return path
def get_models_dir() -> Path: def get_models_dir() -> Path:
"""Get models directory path.""" """Get models directory path."""
path = _data_dir / "models" path = _data_dir / "models"
-332
View File
@@ -1,332 +0,0 @@
"""
SQLite database ORM using SQLAlchemy.
"""
from sqlalchemy import create_engine, Column, String, Integer, Float, DateTime, Text, ForeignKey, Boolean
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import sessionmaker, Session
from datetime import datetime
import uuid
from pathlib import Path
from . import config
Base = declarative_base()
class VoiceProfile(Base):
"""Voice profile database model."""
__tablename__ = "profiles"
id = Column(String, primary_key=True, default=lambda: str(uuid.uuid4()))
name = Column(String, unique=True, nullable=False)
description = Column(Text)
language = Column(String, default="en")
avatar_path = Column(String, nullable=True)
created_at = Column(DateTime, default=datetime.utcnow)
updated_at = Column(DateTime, default=datetime.utcnow, onupdate=datetime.utcnow)
class ProfileSample(Base):
"""Voice profile sample database model."""
__tablename__ = "profile_samples"
id = Column(String, primary_key=True, default=lambda: str(uuid.uuid4()))
profile_id = Column(String, ForeignKey("profiles.id"), nullable=False)
audio_path = Column(String, nullable=False)
reference_text = Column(Text, nullable=False)
class Generation(Base):
"""Generation history database model."""
__tablename__ = "generations"
id = Column(String, primary_key=True, default=lambda: str(uuid.uuid4()))
profile_id = Column(String, ForeignKey("profiles.id"), nullable=False)
text = Column(Text, nullable=False)
language = Column(String, default="en")
audio_path = Column(String, nullable=True)
duration = Column(Float, nullable=True)
seed = Column(Integer)
instruct = Column(Text)
engine = Column(String, default="qwen")
model_size = Column(String, nullable=True)
status = Column(String, default="completed") # generating, completed, failed
error = Column(Text, nullable=True)
created_at = Column(DateTime, default=datetime.utcnow)
class Story(Base):
"""Story database model."""
__tablename__ = "stories"
id = Column(String, primary_key=True, default=lambda: str(uuid.uuid4()))
name = Column(String, nullable=False)
description = Column(Text)
created_at = Column(DateTime, default=datetime.utcnow)
updated_at = Column(DateTime, default=datetime.utcnow, onupdate=datetime.utcnow)
class StoryItem(Base):
"""Story item database model (links generations to stories)."""
__tablename__ = "story_items"
id = Column(String, primary_key=True, default=lambda: str(uuid.uuid4()))
story_id = Column(String, ForeignKey("stories.id"), nullable=False)
generation_id = Column(String, ForeignKey("generations.id"), nullable=False)
start_time_ms = Column(Integer, nullable=False, default=0) # Milliseconds from story start
track = Column(Integer, nullable=False, default=0) # Track number (0 = main track)
trim_start_ms = Column(Integer, nullable=False, default=0) # Milliseconds trimmed from start
trim_end_ms = Column(Integer, nullable=False, default=0) # Milliseconds trimmed from end
created_at = Column(DateTime, default=datetime.utcnow)
class Project(Base):
"""Audio studio project database model."""
__tablename__ = "projects"
id = Column(String, primary_key=True, default=lambda: str(uuid.uuid4()))
name = Column(String, nullable=False)
data = Column(Text) # JSON string
created_at = Column(DateTime, default=datetime.utcnow)
updated_at = Column(DateTime, default=datetime.utcnow, onupdate=datetime.utcnow)
class AudioChannel(Base):
"""Audio channel (bus) database model."""
__tablename__ = "audio_channels"
id = Column(String, primary_key=True, default=lambda: str(uuid.uuid4()))
name = Column(String, nullable=False)
is_default = Column(Boolean, default=False)
created_at = Column(DateTime, default=datetime.utcnow)
class ChannelDeviceMapping(Base):
"""Mapping between channels and OS audio devices."""
__tablename__ = "channel_device_mappings"
id = Column(String, primary_key=True, default=lambda: str(uuid.uuid4()))
channel_id = Column(String, ForeignKey("audio_channels.id"), nullable=False)
device_id = Column(String, nullable=False) # OS device identifier
class ProfileChannelMapping(Base):
"""Mapping between voice profiles and audio channels (many-to-many)."""
__tablename__ = "profile_channel_mappings"
profile_id = Column(String, ForeignKey("profiles.id"), primary_key=True)
channel_id = Column(String, ForeignKey("audio_channels.id"), primary_key=True)
# Database setup will be initialized in init_db()
engine = None
SessionLocal = None
_db_path = None
def init_db():
"""Initialize database tables."""
global engine, SessionLocal, _db_path
_db_path = config.get_db_path()
_db_path.parent.mkdir(parents=True, exist_ok=True)
engine = create_engine(
f"sqlite:///{_db_path}",
connect_args={"check_same_thread": False},
)
SessionLocal = sessionmaker(autocommit=False, autoflush=False, bind=engine)
# Run migrations before creating tables
_run_migrations(engine)
Base.metadata.create_all(bind=engine)
# Create default channel if it doesn't exist
db = SessionLocal()
try:
default_channel = db.query(AudioChannel).filter(AudioChannel.is_default == True).first()
if not default_channel:
default_channel = AudioChannel(
id=str(uuid.uuid4()),
name="Default",
is_default=True
)
db.add(default_channel)
# Assign all existing profiles to default channel
profiles = db.query(VoiceProfile).all()
for profile in profiles:
mapping = ProfileChannelMapping(
profile_id=profile.id,
channel_id=default_channel.id
)
db.add(mapping)
db.commit()
finally:
db.close()
def _run_migrations(engine):
"""Run database migrations."""
from sqlalchemy import inspect, text
inspector = inspect(engine)
# Check if story_items table exists
if 'story_items' not in inspector.get_table_names():
return # Table doesn't exist yet, will be created fresh
# Get columns in story_items table
columns = {col['name'] for col in inspector.get_columns('story_items')}
# Migration: Remove position column and ensure start_time_ms exists
# SQLite doesn't support DROP COLUMN easily, so we recreate the table
if 'position' in columns:
print("Migrating story_items: removing position column, using start_time_ms")
with engine.connect() as conn:
# Check if start_time_ms already exists
has_start_time = 'start_time_ms' in columns
if not has_start_time:
# First, add the new column temporarily
conn.execute(text("ALTER TABLE story_items ADD COLUMN start_time_ms INTEGER DEFAULT 0"))
# Calculate timecodes from position ordering
result = conn.execute(text("""
SELECT si.id, si.story_id, si.position, g.duration
FROM story_items si
JOIN generations g ON si.generation_id = g.id
ORDER BY si.story_id, si.position
"""))
rows = result.fetchall()
current_story_id = None
current_time_ms = 0
for row in rows:
item_id, story_id, position, duration = row
if story_id != current_story_id:
current_story_id = story_id
current_time_ms = 0
conn.execute(
text("UPDATE story_items SET start_time_ms = :time WHERE id = :id"),
{"time": current_time_ms, "id": item_id}
)
current_time_ms += int(duration * 1000) + 200
conn.commit()
# Now recreate the table without the position column
# 1. Create new table
conn.execute(text("""
CREATE TABLE story_items_new (
id VARCHAR PRIMARY KEY,
story_id VARCHAR NOT NULL,
generation_id VARCHAR NOT NULL,
start_time_ms INTEGER NOT NULL DEFAULT 0,
created_at DATETIME,
FOREIGN KEY (story_id) REFERENCES stories(id),
FOREIGN KEY (generation_id) REFERENCES generations(id)
)
"""))
# 2. Copy data
conn.execute(text("""
INSERT INTO story_items_new (id, story_id, generation_id, start_time_ms, created_at)
SELECT id, story_id, generation_id, start_time_ms, created_at FROM story_items
"""))
# 3. Drop old table
conn.execute(text("DROP TABLE story_items"))
# 4. Rename new table
conn.execute(text("ALTER TABLE story_items_new RENAME TO story_items"))
conn.commit()
print("Migrated story_items table to use start_time_ms (removed position column)")
# Migration: Add track column if it doesn't exist
# Re-check columns after potential position migration
columns = {col['name'] for col in inspector.get_columns('story_items')}
if 'track' not in columns:
print("Migrating story_items: adding track column")
with engine.connect() as conn:
conn.execute(text("ALTER TABLE story_items ADD COLUMN track INTEGER NOT NULL DEFAULT 0"))
conn.commit()
print("Added track column to story_items")
# Migration: Add trim columns if they don't exist
# Re-check columns after potential track migration
columns = {col['name'] for col in inspector.get_columns('story_items')}
if 'trim_start_ms' not in columns:
print("Migrating story_items: adding trim_start_ms column")
with engine.connect() as conn:
conn.execute(text("ALTER TABLE story_items ADD COLUMN trim_start_ms INTEGER NOT NULL DEFAULT 0"))
conn.commit()
print("Added trim_start_ms column to story_items")
columns = {col['name'] for col in inspector.get_columns('story_items')}
if 'trim_end_ms' not in columns:
print("Migrating story_items: adding trim_end_ms column")
with engine.connect() as conn:
conn.execute(text("ALTER TABLE story_items ADD COLUMN trim_end_ms INTEGER NOT NULL DEFAULT 0"))
conn.commit()
print("Added trim_end_ms column to story_items")
# Migration: Add avatar_path to profiles table
if 'profiles' in inspector.get_table_names():
columns = {col['name'] for col in inspector.get_columns('profiles')}
if 'avatar_path' not in columns:
print("Migrating profiles: adding avatar_path column")
with engine.connect() as conn:
conn.execute(text("ALTER TABLE profiles ADD COLUMN avatar_path VARCHAR"))
conn.commit()
print("Added avatar_path column to profiles")
# Migration: Add status and error columns to generations table
if 'generations' in inspector.get_table_names():
columns = {col['name'] for col in inspector.get_columns('generations')}
if 'status' not in columns:
print("Migrating generations: adding status column")
with engine.connect() as conn:
conn.execute(text("ALTER TABLE generations ADD COLUMN status VARCHAR DEFAULT 'completed'"))
conn.commit()
print("Added status column to generations")
if 'error' not in columns:
print("Migrating generations: adding error column")
with engine.connect() as conn:
conn.execute(text("ALTER TABLE generations ADD COLUMN error TEXT"))
conn.commit()
print("Added error column to generations")
if 'engine' not in columns:
print("Migrating generations: adding engine column")
with engine.connect() as conn:
conn.execute(text("ALTER TABLE generations ADD COLUMN engine VARCHAR DEFAULT 'qwen'"))
conn.commit()
print("Added engine column to generations")
# Re-read columns after engine migration (variable name shadows outer `engine`)
columns = {col['name'] for col in inspector.get_columns('generations')}
if 'model_size' not in columns:
print("Migrating generations: adding model_size column")
with engine.connect() as conn:
conn.execute(text("ALTER TABLE generations ADD COLUMN model_size VARCHAR"))
conn.commit()
print("Added model_size column to generations")
def get_db():
"""Get database session (generator for dependency injection)."""
db = SessionLocal()
try:
yield db
finally:
db.close()
+44
View File
@@ -0,0 +1,44 @@
"""Database package — ORM models, session management, and migrations.
Re-exports all public symbols so that ``from .database import get_db``
and ``from .database import Generation as DBGeneration`` continue to work
without changing any importers.
"""
from .models import (
Base,
AudioChannel,
ChannelDeviceMapping,
EffectPreset,
Generation,
GenerationVersion,
ProfileChannelMapping,
ProfileSample,
Project,
Story,
StoryItem,
VoiceProfile,
)
from .session import engine, SessionLocal, _db_path, init_db, get_db
__all__ = [
# Models
"Base",
"AudioChannel",
"ChannelDeviceMapping",
"EffectPreset",
"Generation",
"GenerationVersion",
"ProfileChannelMapping",
"ProfileSample",
"Project",
"Story",
"StoryItem",
"VoiceProfile",
# Session
"engine",
"SessionLocal",
"_db_path",
"init_db",
"get_db",
]
+170
View File
@@ -0,0 +1,170 @@
"""Column-level migrations for the voicebox SQLite database.
Why not Alembic? voicebox is a single-user desktop app shipping as a
PyInstaller binary. Every user has exactly one SQLite file. Alembic's
strengths -- migration tracking across environments, rollback, team
coordination -- don't apply here and would add bundling complexity
(alembic.ini, env.py, versions/ directory all need to survive
PyInstaller). The column-existence checks below are idempotent, run in
<50 ms on startup, and have worked reliably across 12 schema changes.
If the project ever moves to a server-based deployment or Postgres, this
decision should be revisited.
Adding a new migration:
1. Append a new ``_migrate_*`` helper at the bottom of this file.
2. Call it from ``run_migrations()`` in the appropriate spot.
3. The helper should check column/table existence before acting
(idempotent) and print a short message when it does real work.
"""
import logging
from sqlalchemy import inspect, text
logger = logging.getLogger(__name__)
def run_migrations(engine) -> None:
"""Run all schema migrations. Safe to call on every startup."""
inspector = inspect(engine)
tables = set(inspector.get_table_names())
_migrate_story_items(engine, inspector, tables)
_migrate_profiles(engine, inspector, tables)
_migrate_generations(engine, inspector, tables)
_migrate_effect_presets(engine, inspector, tables)
_migrate_generation_versions(engine, inspector, tables)
# -- helpers ---------------------------------------------------------------
def _get_columns(inspector, table: str) -> set[str]:
return {col["name"] for col in inspector.get_columns(table)}
def _add_column(engine, table: str, column_sql: str, label: str) -> None:
"""Add a column if it doesn't already exist."""
with engine.connect() as conn:
conn.execute(text(f"ALTER TABLE {table} ADD COLUMN {column_sql}"))
conn.commit()
logger.info("Added %s column to %s", label, table)
# -- per-table migrations --------------------------------------------------
def _migrate_story_items(engine, inspector, tables: set[str]) -> None:
if "story_items" not in tables:
return
columns = _get_columns(inspector, "story_items")
# Replace position-based ordering with absolute timecodes
if "position" in columns:
logger.info("Migrating story_items: removing position column, using start_time_ms")
with engine.connect() as conn:
if "start_time_ms" not in columns:
conn.execute(text(
"ALTER TABLE story_items ADD COLUMN start_time_ms INTEGER DEFAULT 0"
))
result = conn.execute(text("""
SELECT si.id, si.story_id, si.position, g.duration
FROM story_items si
JOIN generations g ON si.generation_id = g.id
ORDER BY si.story_id, si.position
"""))
current_story_id = None
current_time_ms = 0
for item_id, story_id, _position, duration in result.fetchall():
if story_id != current_story_id:
current_story_id = story_id
current_time_ms = 0
conn.execute(
text("UPDATE story_items SET start_time_ms = :time WHERE id = :id"),
{"time": current_time_ms, "id": item_id},
)
current_time_ms += int((duration or 0) * 1000) + 200
conn.commit()
# Recreate table without the position column (SQLite lacks DROP COLUMN)
conn.execute(text("""
CREATE TABLE story_items_new (
id VARCHAR PRIMARY KEY,
story_id VARCHAR NOT NULL,
generation_id VARCHAR NOT NULL,
start_time_ms INTEGER NOT NULL DEFAULT 0,
track INTEGER NOT NULL DEFAULT 0,
trim_start_ms INTEGER NOT NULL DEFAULT 0,
trim_end_ms INTEGER NOT NULL DEFAULT 0,
version_id VARCHAR,
created_at DATETIME,
FOREIGN KEY (story_id) REFERENCES stories(id),
FOREIGN KEY (generation_id) REFERENCES generations(id)
)
"""))
conn.execute(text("""
INSERT INTO story_items_new (id, story_id, generation_id, start_time_ms, track, trim_start_ms, trim_end_ms, version_id, created_at)
SELECT id, story_id, generation_id, start_time_ms,
COALESCE(track, 0), COALESCE(trim_start_ms, 0), COALESCE(trim_end_ms, 0), version_id, created_at
FROM story_items
"""))
conn.execute(text("DROP TABLE story_items"))
conn.execute(text("ALTER TABLE story_items_new RENAME TO story_items"))
conn.commit()
# Re-read after table recreation
columns = _get_columns(inspector, "story_items")
if "track" not in columns:
_add_column(engine, "story_items", "track INTEGER NOT NULL DEFAULT 0", "track")
# Re-read so subsequent checks see new columns
columns = _get_columns(inspector, "story_items")
if "trim_start_ms" not in columns:
_add_column(engine, "story_items", "trim_start_ms INTEGER NOT NULL DEFAULT 0", "trim_start_ms")
if "trim_end_ms" not in columns:
_add_column(engine, "story_items", "trim_end_ms INTEGER NOT NULL DEFAULT 0", "trim_end_ms")
if "version_id" not in columns:
_add_column(engine, "story_items", "version_id VARCHAR", "version_id")
def _migrate_profiles(engine, inspector, tables: set[str]) -> None:
if "profiles" not in tables:
return
columns = _get_columns(inspector, "profiles")
if "avatar_path" not in columns:
_add_column(engine, "profiles", "avatar_path VARCHAR", "avatar_path")
if "effects_chain" not in columns:
_add_column(engine, "profiles", "effects_chain TEXT", "effects_chain")
def _migrate_generations(engine, inspector, tables: set[str]) -> None:
if "generations" not in tables:
return
columns = _get_columns(inspector, "generations")
if "status" not in columns:
_add_column(engine, "generations", "status VARCHAR DEFAULT 'completed'", "status")
if "error" not in columns:
_add_column(engine, "generations", "error TEXT", "error")
if "engine" not in columns:
_add_column(engine, "generations", "engine VARCHAR DEFAULT 'qwen'", "engine")
# Re-read after engine column (variable name shadows outer scope in old code)
columns = _get_columns(inspector, "generations")
if "model_size" not in columns:
_add_column(engine, "generations", "model_size VARCHAR", "model_size")
if "is_favorited" not in columns:
_add_column(engine, "generations", "is_favorited BOOLEAN DEFAULT 0", "is_favorited")
def _migrate_effect_presets(engine, inspector, tables: set[str]) -> None:
if "effect_presets" not in tables:
return
columns = _get_columns(inspector, "effect_presets")
if "sort_order" not in columns:
_add_column(engine, "effect_presets", "sort_order INTEGER DEFAULT 100", "sort_order")
def _migrate_generation_versions(engine, inspector, tables: set[str]) -> None:
if "generation_versions" not in tables:
return
columns = _get_columns(inspector, "generation_versions")
if "source_version_id" not in columns:
_add_column(engine, "generation_versions", "source_version_id VARCHAR", "source_version_id")
+155
View File
@@ -0,0 +1,155 @@
"""ORM model definitions for the voicebox SQLite database."""
from datetime import datetime
import uuid
from sqlalchemy import Column, String, Integer, Float, DateTime, Text, ForeignKey, Boolean
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
class VoiceProfile(Base):
"""Voice profile."""
__tablename__ = "profiles"
id = Column(String, primary_key=True, default=lambda: str(uuid.uuid4()))
name = Column(String, unique=True, nullable=False)
description = Column(Text)
language = Column(String, default="en")
avatar_path = Column(String, nullable=True)
effects_chain = Column(Text, nullable=True)
created_at = Column(DateTime, default=datetime.utcnow)
updated_at = Column(DateTime, default=datetime.utcnow, onupdate=datetime.utcnow)
class ProfileSample(Base):
"""Audio sample attached to a voice profile."""
__tablename__ = "profile_samples"
id = Column(String, primary_key=True, default=lambda: str(uuid.uuid4()))
profile_id = Column(String, ForeignKey("profiles.id"), nullable=False)
audio_path = Column(String, nullable=False)
reference_text = Column(Text, nullable=False)
class Generation(Base):
"""A single TTS generation."""
__tablename__ = "generations"
id = Column(String, primary_key=True, default=lambda: str(uuid.uuid4()))
profile_id = Column(String, ForeignKey("profiles.id"), nullable=False)
text = Column(Text, nullable=False)
language = Column(String, default="en")
audio_path = Column(String, nullable=True)
duration = Column(Float, nullable=True)
seed = Column(Integer)
instruct = Column(Text)
engine = Column(String, default="qwen")
model_size = Column(String, nullable=True)
status = Column(String, default="completed")
error = Column(Text, nullable=True)
is_favorited = Column(Boolean, default=False)
created_at = Column(DateTime, default=datetime.utcnow)
class Story(Base):
"""A story that sequences multiple generations."""
__tablename__ = "stories"
id = Column(String, primary_key=True, default=lambda: str(uuid.uuid4()))
name = Column(String, nullable=False)
description = Column(Text)
created_at = Column(DateTime, default=datetime.utcnow)
updated_at = Column(DateTime, default=datetime.utcnow, onupdate=datetime.utcnow)
class StoryItem(Base):
"""Links a generation to a story at a specific timecode."""
__tablename__ = "story_items"
id = Column(String, primary_key=True, default=lambda: str(uuid.uuid4()))
story_id = Column(String, ForeignKey("stories.id"), nullable=False)
generation_id = Column(String, ForeignKey("generations.id"), nullable=False)
version_id = Column(String, ForeignKey("generation_versions.id"), nullable=True)
start_time_ms = Column(Integer, nullable=False, default=0)
track = Column(Integer, nullable=False, default=0)
trim_start_ms = Column(Integer, nullable=False, default=0)
trim_end_ms = Column(Integer, nullable=False, default=0)
created_at = Column(DateTime, default=datetime.utcnow)
class Project(Base):
"""Audio studio project (JSON blob)."""
__tablename__ = "projects"
id = Column(String, primary_key=True, default=lambda: str(uuid.uuid4()))
name = Column(String, nullable=False)
data = Column(Text)
created_at = Column(DateTime, default=datetime.utcnow)
updated_at = Column(DateTime, default=datetime.utcnow, onupdate=datetime.utcnow)
class GenerationVersion(Base):
"""A version of a generation's audio (original, processed, alternate takes)."""
__tablename__ = "generation_versions"
id = Column(String, primary_key=True, default=lambda: str(uuid.uuid4()))
generation_id = Column(String, ForeignKey("generations.id"), nullable=False)
label = Column(String, nullable=False)
audio_path = Column(String, nullable=False)
effects_chain = Column(Text, nullable=True)
source_version_id = Column(String, ForeignKey("generation_versions.id"), nullable=True)
is_default = Column(Boolean, default=False)
created_at = Column(DateTime, default=datetime.utcnow)
class EffectPreset(Base):
"""Saved effect chain preset."""
__tablename__ = "effect_presets"
id = Column(String, primary_key=True, default=lambda: str(uuid.uuid4()))
name = Column(String, unique=True, nullable=False)
description = Column(Text, nullable=True)
effects_chain = Column(Text, nullable=False)
is_builtin = Column(Boolean, default=False)
sort_order = Column(Integer, default=100)
created_at = Column(DateTime, default=datetime.utcnow)
class AudioChannel(Base):
"""Audio output channel (bus)."""
__tablename__ = "audio_channels"
id = Column(String, primary_key=True, default=lambda: str(uuid.uuid4()))
name = Column(String, nullable=False)
is_default = Column(Boolean, default=False)
created_at = Column(DateTime, default=datetime.utcnow)
class ChannelDeviceMapping(Base):
"""Mapping between a channel and an OS audio device."""
__tablename__ = "channel_device_mappings"
id = Column(String, primary_key=True, default=lambda: str(uuid.uuid4()))
channel_id = Column(String, ForeignKey("audio_channels.id"), nullable=False)
device_id = Column(String, nullable=False)
class ProfileChannelMapping(Base):
"""Many-to-many mapping between voice profiles and audio channels."""
__tablename__ = "profile_channel_mappings"
profile_id = Column(String, ForeignKey("profiles.id"), primary_key=True)
channel_id = Column(String, ForeignKey("audio_channels.id"), primary_key=True)
+71
View File
@@ -0,0 +1,71 @@
"""Post-migration data seeding and backfills."""
import json
import logging
import uuid
from pathlib import Path
logger = logging.getLogger(__name__)
def backfill_generation_versions(SessionLocal, Generation, GenerationVersion) -> None:
"""Create 'clean' version entries for generations that predate the versions feature."""
db = SessionLocal()
try:
existing_version_gen_ids = {
row[0] for row in db.query(GenerationVersion.generation_id).all()
}
generations = db.query(Generation).filter(
Generation.status == "completed",
Generation.audio_path.isnot(None),
Generation.audio_path != "",
).all()
count = 0
for gen in generations:
if gen.id in existing_version_gen_ids:
continue
if not Path(gen.audio_path).exists():
continue
version = GenerationVersion(
id=str(uuid.uuid4()),
generation_id=gen.id,
label="clean",
audio_path=gen.audio_path,
effects_chain=None,
is_default=True,
)
db.add(version)
count += 1
if count > 0:
db.commit()
logger.info("Backfilled %d generation version entries", count)
finally:
db.close()
def seed_builtin_presets(SessionLocal, EffectPreset) -> None:
"""Ensure built-in effect presets exist in the database."""
from ..utils.effects import BUILTIN_PRESETS
db = SessionLocal()
try:
for idx, (_key, preset_data) in enumerate(BUILTIN_PRESETS.items()):
sort_order = preset_data.get("sort_order", idx)
existing = db.query(EffectPreset).filter_by(name=preset_data["name"]).first()
if not existing:
preset = EffectPreset(
id=str(uuid.uuid4()),
name=preset_data["name"],
description=preset_data.get("description"),
effects_chain=json.dumps(preset_data["effects_chain"]),
is_builtin=True,
sort_order=sort_order,
)
db.add(preset)
elif existing.sort_order != sort_order:
existing.sort_order = sort_order
db.commit()
finally:
db.close()
+78
View File
@@ -0,0 +1,78 @@
"""Engine creation, initialization, and session management."""
import logging
import uuid
from sqlalchemy import create_engine
from sqlalchemy.orm import sessionmaker
from .. import config
from .models import (
Base,
AudioChannel,
EffectPreset,
Generation,
GenerationVersion,
ProfileChannelMapping,
VoiceProfile,
)
from .migrations import run_migrations
from .seed import backfill_generation_versions, seed_builtin_presets
logger = logging.getLogger(__name__)
# Initialized by init_db()
engine = None
SessionLocal = None
_db_path = None
def init_db() -> None:
"""Initialize the database engine, run migrations, create tables, and seed data."""
global engine, SessionLocal, _db_path
_db_path = config.get_db_path()
_db_path.parent.mkdir(parents=True, exist_ok=True)
engine = create_engine(
f"sqlite:///{_db_path}",
connect_args={"check_same_thread": False},
)
SessionLocal = sessionmaker(autocommit=False, autoflush=False, bind=engine)
run_migrations(engine)
Base.metadata.create_all(bind=engine)
# Create default audio channel if it doesn't exist
db = SessionLocal()
try:
default_channel = db.query(AudioChannel).filter(AudioChannel.is_default == True).first()
if not default_channel:
default_channel = AudioChannel(
id=str(uuid.uuid4()),
name="Default",
is_default=True,
)
db.add(default_channel)
for profile in db.query(VoiceProfile).all():
db.add(ProfileChannelMapping(
profile_id=profile.id,
channel_id=default_channel.id,
))
db.commit()
finally:
db.close()
backfill_generation_versions(SessionLocal, Generation, GenerationVersion)
seed_builtin_presets(SessionLocal, EffectPreset)
def get_db():
"""Yield a database session (FastAPI dependency)."""
db = SessionLocal()
try:
yield db
finally:
db.close()
-221
View File
@@ -1,221 +0,0 @@
"""
Example usage of the voicebox backend API.
This script demonstrates how to:
1. Create a voice profile
2. Add samples to the profile
3. Generate speech
4. List history
"""
import requests
import time
from pathlib import Path
# API base URL
BASE_URL = "http://localhost:8000"
def check_health():
"""Check if the server is running."""
response = requests.get(f"{BASE_URL}/health")
data = response.json()
print(f"Server status: {data['status']}")
print(f"Model loaded: {data['model_loaded']}")
print(f"GPU available: {data['gpu_available']}")
print()
return data
def create_profile(name: str, description: str = None, language: str = "en"):
"""Create a new voice profile."""
response = requests.post(
f"{BASE_URL}/profiles",
json={
"name": name,
"description": description,
"language": language,
},
)
response.raise_for_status()
profile = response.json()
print(f"Created profile: {profile['name']} (ID: {profile['id']})")
return profile
def add_sample(profile_id: str, audio_file: str, reference_text: str):
"""Add a sample to a voice profile."""
with open(audio_file, "rb") as f:
files = {"file": f}
data = {"reference_text": reference_text}
response = requests.post(
f"{BASE_URL}/profiles/{profile_id}/samples",
files=files,
data=data,
)
response.raise_for_status()
sample = response.json()
print(f"Added sample: {sample['id']}")
return sample
def generate_speech(profile_id: str, text: str, language: str = "en", seed: int = None):
"""Generate speech using a voice profile."""
print(f"Generating speech: '{text[:50]}...'")
start_time = time.time()
response = requests.post(
f"{BASE_URL}/generate",
json={
"profile_id": profile_id,
"text": text,
"language": language,
"seed": seed,
},
)
response.raise_for_status()
generation = response.json()
elapsed = time.time() - start_time
print(f"Generated in {elapsed:.2f}s (duration: {generation['duration']:.2f}s)")
print(f"Generation ID: {generation['id']}")
return generation
def download_audio(generation_id: str, output_file: str):
"""Download generated audio."""
response = requests.get(f"{BASE_URL}/audio/{generation_id}")
response.raise_for_status()
with open(output_file, "wb") as f:
f.write(response.content)
print(f"Saved audio to: {output_file}")
def list_profiles():
"""List all voice profiles."""
response = requests.get(f"{BASE_URL}/profiles")
response.raise_for_status()
profiles = response.json()
print(f"Found {len(profiles)} profiles:")
for profile in profiles:
print(f" - {profile['name']} (ID: {profile['id']})")
return profiles
def list_history(profile_id: str = None, limit: int = 10):
"""List generation history."""
params = {"limit": limit}
if profile_id:
params["profile_id"] = profile_id
response = requests.get(f"{BASE_URL}/history", params=params)
response.raise_for_status()
history = response.json()
print(f"Found {len(history)} generations:")
for gen in history:
print(f" - {gen['text'][:50]}... ({gen['duration']:.2f}s)")
return history
def transcribe_audio(audio_file: str, language: str = None):
"""Transcribe audio file."""
print(f"Transcribing: {audio_file}")
with open(audio_file, "rb") as f:
files = {"file": f}
data = {}
if language:
data["language"] = language
response = requests.post(
f"{BASE_URL}/transcribe",
files=files,
data=data,
)
response.raise_for_status()
result = response.json()
print(f"Transcription: {result['text']}")
print(f"Duration: {result['duration']:.2f}s")
return result
def main():
"""Run example workflow."""
print("=" * 60)
print("voicebox Backend API Example")
print("=" * 60)
print()
# 1. Check health
print("1. Checking server health...")
check_health()
# 2. Create a profile
print("2. Creating voice profile...")
profile = create_profile(
name="Example Voice",
description="A test voice profile",
language="en",
)
profile_id = profile["id"]
print()
# 3. Add samples (you'll need actual audio files)
print("3. Adding samples...")
print(" (Skipping - add your own audio files here)")
# Uncomment and add your audio file:
# sample = add_sample(
# profile_id,
# "path/to/your/sample.wav",
# "This is the transcript of the audio",
# )
print()
# 4. Generate speech (requires samples to be added first)
print("4. Generating speech...")
print(" (Skipping - add samples first)")
# Uncomment after adding samples:
# generation = generate_speech(
# profile_id,
# "Hello, this is a test of the voice cloning system.",
# language="en",
# seed=42,
# )
#
# # 5. Download audio
# print("\n5. Downloading audio...")
# download_audio(generation["id"], "output.wav")
print()
# 6. List profiles
print("6. Listing all profiles...")
list_profiles()
print()
# 7. List history
print("7. Listing generation history...")
list_history(limit=5)
print()
# 8. Transcribe audio (you'll need an audio file)
print("8. Transcribing audio...")
print(" (Skipping - add your own audio file here)")
# Uncomment and add your audio file:
# transcribe_audio("path/to/audio.wav", language="en")
print()
print("=" * 60)
print("Example complete!")
print("=" * 60)
if __name__ == "__main__":
main()
+7 -2600
View File
File diff suppressed because it is too large Load Diff
-48
View File
@@ -1,48 +0,0 @@
"""
Database migration script to add instruct column to generations table.
Run this once to update existing databases:
python -m backend.migrate_add_instruct
"""
import sqlite3
import os
from pathlib import Path
def migrate():
"""Add instruct column to generations table if it doesn't exist."""
# Get data directory
data_dir = os.environ.get("VOICEBOX_DATA_DIR")
if data_dir:
db_path = Path(data_dir) / "voicebox.db"
else:
db_path = Path.cwd() / "data" / "voicebox.db"
if not db_path.exists():
print(f"Database not found at {db_path}, skipping migration")
return
conn = sqlite3.connect(db_path)
cursor = conn.cursor()
# Check if instruct column already exists
cursor.execute("PRAGMA table_info(generations)")
columns = [row[1] for row in cursor.fetchall()]
if 'instruct' in columns:
print("instruct column already exists, skipping migration")
conn.close()
return
# Add instruct column
print("Adding instruct column to generations table...")
cursor.execute("ALTER TABLE generations ADD COLUMN instruct TEXT")
conn.commit()
conn.close()
print("Migration complete!")
if __name__ == "__main__":
migrate()
+175 -7
View File
@@ -9,18 +9,25 @@ from datetime import datetime
class VoiceProfileCreate(BaseModel): class VoiceProfileCreate(BaseModel):
"""Request model for creating a voice profile.""" """Request model for creating a voice profile."""
name: str = Field(..., min_length=1, max_length=100) name: str = Field(..., min_length=1, max_length=100)
description: Optional[str] = Field(None, max_length=500) description: Optional[str] = Field(None, max_length=500)
language: str = Field(default="en", pattern="^(zh|en|ja|ko|de|fr|ru|pt|es|it|he|ar|da|el|fi|hi|ms|nl|no|pl|sv|sw|tr)$") language: str = Field(
default="en", pattern="^(zh|en|ja|ko|de|fr|ru|pt|es|it|he|ar|da|el|fi|hi|ms|nl|no|pl|sv|sw|tr)$"
)
class VoiceProfileResponse(BaseModel): class VoiceProfileResponse(BaseModel):
"""Response model for voice profile.""" """Response model for voice profile."""
id: str id: str
name: str name: str
description: Optional[str] description: Optional[str]
language: str language: str
avatar_path: Optional[str] = None avatar_path: Optional[str] = None
effects_chain: Optional[List["EffectConfig"]] = None
generation_count: int = 0
sample_count: int = 0
created_at: datetime created_at: datetime
updated_at: datetime updated_at: datetime
@@ -30,16 +37,19 @@ class VoiceProfileResponse(BaseModel):
class ProfileSampleCreate(BaseModel): class ProfileSampleCreate(BaseModel):
"""Request model for adding a sample to a profile.""" """Request model for adding a sample to a profile."""
reference_text: str = Field(..., min_length=1, max_length=1000) reference_text: str = Field(..., min_length=1, max_length=1000)
class ProfileSampleUpdate(BaseModel): class ProfileSampleUpdate(BaseModel):
"""Request model for updating a profile sample.""" """Request model for updating a profile sample."""
reference_text: str = Field(..., min_length=1, max_length=1000) reference_text: str = Field(..., min_length=1, max_length=1000)
class ProfileSampleResponse(BaseModel): class ProfileSampleResponse(BaseModel):
"""Response model for profile sample.""" """Response model for profile sample."""
id: str id: str
profile_id: str profile_id: str
audio_path: str audio_path: str
@@ -51,20 +61,29 @@ class ProfileSampleResponse(BaseModel):
class GenerationRequest(BaseModel): class GenerationRequest(BaseModel):
"""Request model for voice generation.""" """Request model for voice generation."""
profile_id: str profile_id: str
text: str = Field(..., min_length=1, max_length=50000) text: str = Field(..., min_length=1, max_length=50000)
language: str = Field(default="en", pattern="^(zh|en|ja|ko|de|fr|ru|pt|es|it|he)$") language: str = Field(default="en", pattern="^(zh|en|ja|ko|de|fr|ru|pt|es|it|he|ar|da|el|fi|hi|ms|nl|no|pl|sv|sw|tr)$")
seed: Optional[int] = Field(None, ge=0) seed: Optional[int] = Field(None, ge=0)
model_size: Optional[str] = Field(default="1.7B", pattern="^(1\\.7B|0\\.6B)$") model_size: Optional[str] = Field(default="1.7B", pattern="^(1\\.7B|0\\.6B)$")
instruct: Optional[str] = Field(None, max_length=500) instruct: Optional[str] = Field(None, max_length=500)
engine: Optional[str] = Field(default="qwen", pattern="^(qwen|luxtts|chatterbox|chatterbox_turbo)$") engine: Optional[str] = Field(default="qwen", pattern="^(qwen|luxtts|chatterbox|chatterbox_turbo)$")
max_chunk_chars: int = Field(default=800, ge=100, le=5000, description="Max characters per chunk for long text splitting") max_chunk_chars: int = Field(
crossfade_ms: int = Field(default=50, ge=0, le=500, description="Crossfade duration in ms between chunks (0 for hard cut)") default=800, ge=100, le=5000, description="Max characters per chunk for long text splitting"
)
crossfade_ms: int = Field(
default=50, ge=0, le=500, description="Crossfade duration in ms between chunks (0 for hard cut)"
)
normalize: bool = Field(default=True, description="Normalize output audio volume") normalize: bool = Field(default=True, description="Normalize output audio volume")
effects_chain: Optional[List["EffectConfig"]] = Field(
None, description="Effects chain to apply after generation (overrides profile default)"
)
class GenerationResponse(BaseModel): class GenerationResponse(BaseModel):
"""Response model for voice generation.""" """Response model for voice generation."""
id: str id: str
profile_id: str profile_id: str
text: str text: str
@@ -77,7 +96,10 @@ class GenerationResponse(BaseModel):
model_size: Optional[str] = None model_size: Optional[str] = None
status: str = "completed" status: str = "completed"
error: Optional[str] = None error: Optional[str] = None
is_favorited: bool = False
created_at: datetime created_at: datetime
versions: Optional[List["GenerationVersionResponse"]] = None
active_version_id: Optional[str] = None
class Config: class Config:
from_attributes = True from_attributes = True
@@ -85,6 +107,7 @@ class GenerationResponse(BaseModel):
class HistoryQuery(BaseModel): class HistoryQuery(BaseModel):
"""Query model for generation history.""" """Query model for generation history."""
profile_id: Optional[str] = None profile_id: Optional[str] = None
search: Optional[str] = None search: Optional[str] = None
limit: int = Field(default=50, ge=1, le=100) limit: int = Field(default=50, ge=1, le=100)
@@ -93,6 +116,7 @@ class HistoryQuery(BaseModel):
class HistoryResponse(BaseModel): class HistoryResponse(BaseModel):
"""Response model for history entry (includes profile name).""" """Response model for history entry (includes profile name)."""
id: str id: str
profile_id: str profile_id: str
profile_name: str profile_name: str
@@ -106,7 +130,10 @@ class HistoryResponse(BaseModel):
model_size: Optional[str] = None model_size: Optional[str] = None
status: str = "completed" status: str = "completed"
error: Optional[str] = None error: Optional[str] = None
is_favorited: bool = False
created_at: datetime created_at: datetime
versions: Optional[List["GenerationVersionResponse"]] = None
active_version_id: Optional[str] = None
class Config: class Config:
from_attributes = True from_attributes = True
@@ -114,23 +141,27 @@ class HistoryResponse(BaseModel):
class HistoryListResponse(BaseModel): class HistoryListResponse(BaseModel):
"""Response model for history list.""" """Response model for history list."""
items: List[HistoryResponse] items: List[HistoryResponse]
total: int total: int
class TranscriptionRequest(BaseModel): class TranscriptionRequest(BaseModel):
"""Request model for audio transcription.""" """Request model for audio transcription."""
language: Optional[str] = Field(None, pattern="^(en|zh)$") language: Optional[str] = Field(None, pattern="^(en|zh)$")
class TranscriptionResponse(BaseModel): class TranscriptionResponse(BaseModel):
"""Response model for transcription.""" """Response model for transcription."""
text: str text: str
duration: float duration: float
class HealthResponse(BaseModel): class HealthResponse(BaseModel):
"""Response model for health check.""" """Response model for health check."""
status: str status: str
model_loaded: bool model_loaded: bool
model_downloaded: Optional[bool] = None # Whether model is cached/downloaded model_downloaded: Optional[bool] = None # Whether model is cached/downloaded
@@ -144,6 +175,7 @@ class HealthResponse(BaseModel):
class DirectoryCheck(BaseModel): class DirectoryCheck(BaseModel):
"""Health status for a single directory.""" """Health status for a single directory."""
path: str path: str
exists: bool exists: bool
writable: bool writable: bool
@@ -152,6 +184,7 @@ class DirectoryCheck(BaseModel):
class FilesystemHealthResponse(BaseModel): class FilesystemHealthResponse(BaseModel):
"""Response model for filesystem health check.""" """Response model for filesystem health check."""
healthy: bool healthy: bool
disk_free_mb: Optional[float] = None disk_free_mb: Optional[float] = None
disk_total_mb: Optional[float] = None disk_total_mb: Optional[float] = None
@@ -160,6 +193,7 @@ class FilesystemHealthResponse(BaseModel):
class ModelStatus(BaseModel): class ModelStatus(BaseModel):
"""Response model for model status.""" """Response model for model status."""
model_name: str model_name: str
display_name: str display_name: str
hf_repo_id: Optional[str] = None # HuggingFace repository ID hf_repo_id: Optional[str] = None # HuggingFace repository ID
@@ -171,33 +205,38 @@ class ModelStatus(BaseModel):
class ModelStatusListResponse(BaseModel): class ModelStatusListResponse(BaseModel):
"""Response model for model status list.""" """Response model for model status list."""
models: List[ModelStatus] models: List[ModelStatus]
class ModelDownloadRequest(BaseModel): class ModelDownloadRequest(BaseModel):
"""Request model for triggering model download.""" """Request model for triggering model download."""
model_name: str model_name: str
class ModelMigrateRequest(BaseModel): class ModelMigrateRequest(BaseModel):
"""Request model for migrating models to a new directory.""" """Request model for migrating models to a new directory."""
destination: str destination: str
class ActiveDownloadTask(BaseModel): class ActiveDownloadTask(BaseModel):
"""Response model for active download task.""" """Response model for active download task."""
model_name: str model_name: str
status: str status: str
started_at: datetime started_at: datetime
error: Optional[str] = None error: Optional[str] = None
progress: Optional[float] = None # 0-100 percentage progress: Optional[float] = None # 0-100 percentage
current: Optional[int] = None # bytes downloaded current: Optional[int] = None # bytes downloaded
total: Optional[int] = None # total bytes total: Optional[int] = None # total bytes
filename: Optional[str] = None # current file being downloaded filename: Optional[str] = None # current file being downloaded
class ActiveGenerationTask(BaseModel): class ActiveGenerationTask(BaseModel):
"""Response model for active generation task.""" """Response model for active generation task."""
task_id: str task_id: str
profile_id: str profile_id: str
text_preview: str text_preview: str
@@ -206,24 +245,28 @@ class ActiveGenerationTask(BaseModel):
class ActiveTasksResponse(BaseModel): class ActiveTasksResponse(BaseModel):
"""Response model for active tasks.""" """Response model for active tasks."""
downloads: List[ActiveDownloadTask] downloads: List[ActiveDownloadTask]
generations: List[ActiveGenerationTask] generations: List[ActiveGenerationTask]
class AudioChannelCreate(BaseModel): class AudioChannelCreate(BaseModel):
"""Request model for creating an audio channel.""" """Request model for creating an audio channel."""
name: str = Field(..., min_length=1, max_length=100) name: str = Field(..., min_length=1, max_length=100)
device_ids: List[str] = Field(default_factory=list) device_ids: List[str] = Field(default_factory=list)
class AudioChannelUpdate(BaseModel): class AudioChannelUpdate(BaseModel):
"""Request model for updating an audio channel.""" """Request model for updating an audio channel."""
name: Optional[str] = Field(None, min_length=1, max_length=100) name: Optional[str] = Field(None, min_length=1, max_length=100)
device_ids: Optional[List[str]] = None device_ids: Optional[List[str]] = None
class AudioChannelResponse(BaseModel): class AudioChannelResponse(BaseModel):
"""Response model for audio channel.""" """Response model for audio channel."""
id: str id: str
name: str name: str
is_default: bool is_default: bool
@@ -236,22 +279,26 @@ class AudioChannelResponse(BaseModel):
class ChannelVoiceAssignment(BaseModel): class ChannelVoiceAssignment(BaseModel):
"""Request model for assigning voices to a channel.""" """Request model for assigning voices to a channel."""
profile_ids: List[str] profile_ids: List[str]
class ProfileChannelAssignment(BaseModel): class ProfileChannelAssignment(BaseModel):
"""Request model for assigning channels to a profile.""" """Request model for assigning channels to a profile."""
channel_ids: List[str] channel_ids: List[str]
class StoryCreate(BaseModel): class StoryCreate(BaseModel):
"""Request model for creating a story.""" """Request model for creating a story."""
name: str = Field(..., min_length=1, max_length=100) name: str = Field(..., min_length=1, max_length=100)
description: Optional[str] = Field(None, max_length=500) description: Optional[str] = Field(None, max_length=500)
class StoryResponse(BaseModel): class StoryResponse(BaseModel):
"""Response model for story (list view).""" """Response model for story (list view)."""
id: str id: str
name: str name: str
description: Optional[str] description: Optional[str]
@@ -265,9 +312,11 @@ class StoryResponse(BaseModel):
class StoryItemDetail(BaseModel): class StoryItemDetail(BaseModel):
"""Detail model for story item with generation info.""" """Detail model for story item with generation info."""
id: str id: str
story_id: str story_id: str
generation_id: str generation_id: str
version_id: Optional[str] = None
start_time_ms: int start_time_ms: int
track: int = 0 track: int = 0
trim_start_ms: int = 0 trim_start_ms: int = 0
@@ -283,6 +332,9 @@ class StoryItemDetail(BaseModel):
seed: Optional[int] seed: Optional[int]
instruct: Optional[str] instruct: Optional[str]
generation_created_at: datetime generation_created_at: datetime
# Versions available for this generation
versions: Optional[List["GenerationVersionResponse"]] = None
active_version_id: Optional[str] = None
class Config: class Config:
from_attributes = True from_attributes = True
@@ -290,6 +342,7 @@ class StoryItemDetail(BaseModel):
class StoryDetailResponse(BaseModel): class StoryDetailResponse(BaseModel):
"""Response model for story with items.""" """Response model for story with items."""
id: str id: str
name: str name: str
description: Optional[str] description: Optional[str]
@@ -303,6 +356,7 @@ class StoryDetailResponse(BaseModel):
class StoryItemCreate(BaseModel): class StoryItemCreate(BaseModel):
"""Request model for adding a generation to a story.""" """Request model for adding a generation to a story."""
generation_id: str generation_id: str
start_time_ms: Optional[int] = None # If not provided, will be calculated automatically start_time_ms: Optional[int] = None # If not provided, will be calculated automatically
track: Optional[int] = 0 # Track number (0 = main track) track: Optional[int] = 0 # Track number (0 = main track)
@@ -310,32 +364,146 @@ class StoryItemCreate(BaseModel):
class StoryItemUpdateTime(BaseModel): class StoryItemUpdateTime(BaseModel):
"""Request model for updating a story item's timecode.""" """Request model for updating a story item's timecode."""
generation_id: str generation_id: str
start_time_ms: int = Field(..., ge=0) start_time_ms: int = Field(..., ge=0)
class StoryItemBatchUpdate(BaseModel): class StoryItemBatchUpdate(BaseModel):
"""Request model for batch updating story item timecodes.""" """Request model for batch updating story item timecodes."""
updates: List[StoryItemUpdateTime] updates: List[StoryItemUpdateTime]
class StoryItemReorder(BaseModel): class StoryItemReorder(BaseModel):
"""Request model for reordering story items.""" """Request model for reordering story items."""
generation_ids: List[str] = Field(..., min_length=1) generation_ids: List[str] = Field(..., min_length=1)
class StoryItemMove(BaseModel): class StoryItemMove(BaseModel):
"""Request model for moving a story item (position and/or track).""" """Request model for moving a story item (position and/or track)."""
start_time_ms: int = Field(..., ge=0) start_time_ms: int = Field(..., ge=0)
track: int = 0 track: int = 0
class StoryItemTrim(BaseModel): class StoryItemTrim(BaseModel):
"""Request model for trimming a story item.""" """Request model for trimming a story item."""
trim_start_ms: int = Field(..., ge=0) trim_start_ms: int = Field(..., ge=0)
trim_end_ms: int = Field(..., ge=0) trim_end_ms: int = Field(..., ge=0)
class StoryItemSplit(BaseModel): class StoryItemSplit(BaseModel):
"""Request model for splitting a story item.""" """Request model for splitting a story item."""
split_time_ms: int = Field(..., ge=0) # Time within the clip to split at (relative to clip start) split_time_ms: int = Field(..., ge=0) # Time within the clip to split at (relative to clip start)
class StoryItemVersionUpdate(BaseModel):
"""Request model for setting a story item's pinned version."""
version_id: Optional[str] = None # null = use generation default
class EffectConfig(BaseModel):
"""A single effect in an effects chain."""
type: str
enabled: bool = True
params: dict = Field(default_factory=dict)
class EffectsChain(BaseModel):
"""An ordered list of effects to apply."""
effects: List[EffectConfig] = Field(default_factory=list)
class EffectPresetCreate(BaseModel):
"""Request model for creating an effect preset."""
name: str = Field(..., min_length=1, max_length=100)
description: Optional[str] = Field(None, max_length=500)
effects_chain: List[EffectConfig]
class EffectPresetUpdate(BaseModel):
"""Request model for updating an effect preset."""
name: Optional[str] = Field(None, min_length=1, max_length=100)
description: Optional[str] = None
effects_chain: Optional[List[EffectConfig]] = None
class EffectPresetResponse(BaseModel):
"""Response model for effect preset."""
id: str
name: str
description: Optional[str] = None
effects_chain: List[EffectConfig]
is_builtin: bool = False
created_at: datetime
class Config:
from_attributes = True
class GenerationVersionResponse(BaseModel):
"""Response model for a generation version."""
id: str
generation_id: str
label: str
audio_path: str
effects_chain: Optional[List[EffectConfig]] = None
source_version_id: Optional[str] = None
is_default: bool
created_at: datetime
class Config:
from_attributes = True
class ApplyEffectsRequest(BaseModel):
"""Request to apply effects to an existing generation."""
effects_chain: List[EffectConfig]
source_version_id: Optional[str] = Field(
None, description="Version to use as source audio (defaults to clean/original)"
)
label: Optional[str] = Field(None, max_length=100, description="Label for this version (auto-generated if omitted)")
set_as_default: bool = Field(default=True, description="Set this version as the default")
class ProfileEffectsUpdate(BaseModel):
"""Request to update the default effects chain on a profile."""
effects_chain: Optional[List[EffectConfig]] = Field(None, description="Effects chain (null to remove)")
class AvailableEffectParam(BaseModel):
"""Description of a single effect parameter."""
default: float
min: float
max: float
step: float
description: str
class AvailableEffect(BaseModel):
"""Description of an available effect type."""
type: str
label: str
description: str
params: dict # param_name -> AvailableEffectParam
class AvailableEffectsResponse(BaseModel):
"""Response listing all available effect types."""
effects: List[AvailableEffect]
+83
View File
@@ -0,0 +1,83 @@
[project]
name = "voicebox-backend"
version = "0.2.3"
requires-python = ">=3.12"
# ---------------------------------------------------------------------------
# Ruff – linter + formatter
# ---------------------------------------------------------------------------
[tool.ruff]
target-version = "py312"
line-length = 120
src = ["."]
# Files/dirs to skip entirely.
extend-exclude = [
"voicebox-server.spec",
"build_binary.py",
]
[tool.ruff.lint]
select = [
"F", # pyflakes
"E", # pycodestyle errors
"W", # pycodestyle warnings
"I", # isort
"N", # pep8-naming
"UP", # pyupgrade (modernize syntax for 3.12)
"B", # flake8-bugbear
"A", # flake8-builtins (shadowing built-in names)
"SIM", # flake8-simplify
"T20", # flake8-print (flag print() calls)
"RET", # flake8-return
"PIE", # misc lints
"PT", # flake8-pytest-style
"RUF", # ruff-specific rules
"ERA", # commented-out code detection
"FIX", # flag TODO/FIXME/HACK/XXX for review
]
ignore = [
# Allow print() in existing code -- remove items from this list as files
# are migrated to logging during the refactor.
"T201", # print() found
# These conflict with the formatter or are too noisy during migration:
"E501", # line too long (formatter handles this)
"RET504", # unnecessary assignment before return
"SIM108", # use ternary operator (sometimes less readable)
"B008", # function call in default argument (FastAPI Depends() pattern)
"UP007", # use X | Y for union (auto-fixed by UP, but noisy on big diffs)
]
# Per-file rule overrides.
[tool.ruff.lint.per-file-ignores]
# Tests can use assert, print, and magic values freely.
"tests/**" = ["S101", "T201", "PLR2004", "ERA001"]
# __init__.py re-exports are expected to have unused imports.
"**/__init__.py" = ["F401"]
# Entry points and scripts legitimately use print.
"server.py" = ["T201"]
"main.py" = ["T201"]
# AMD GPU env vars must be set before torch import.
"app.py" = ["E402"]
[tool.ruff.lint.isort]
known-first-party = ["backend"]
# Group "from backend.*" imports into the first-party section.
force-single-line = false
combine-as-imports = true
[tool.ruff.format]
quote-style = "double"
indent-style = "space"
docstring-code-format = true
# ---------------------------------------------------------------------------
# pytest
# ---------------------------------------------------------------------------
[tool.pytest.ini_options]
testpaths = ["tests"]
asyncio_mode = "auto"
+1
View File
@@ -38,6 +38,7 @@ librosa>=0.10.0
soundfile>=0.12.0 soundfile>=0.12.0
numpy>=1.24.0 numpy>=1.24.0
numba>=0.60.0,<0.61.0 numba>=0.60.0,<0.61.0
pedalboard>=0.9.0
# HTTP client (for CUDA backend download) # HTTP client (for CUDA backend download)
httpx>=0.27.0 httpx>=0.27.0
+32
View File
@@ -0,0 +1,32 @@
"""Route registration for the voicebox API."""
from fastapi import FastAPI
def register_routers(app: FastAPI) -> None:
"""Include all domain routers on the application."""
from .health import router as health_router
from .profiles import router as profiles_router
from .channels import router as channels_router
from .generations import router as generations_router
from .history import router as history_router
from .transcription import router as transcription_router
from .stories import router as stories_router
from .effects import router as effects_router
from .audio import router as audio_router
from .models import router as models_router
from .tasks import router as tasks_router
from .cuda import router as cuda_router
app.include_router(health_router)
app.include_router(profiles_router)
app.include_router(channels_router)
app.include_router(generations_router)
app.include_router(history_router)
app.include_router(transcription_router)
app.include_router(stories_router)
app.include_router(effects_router)
app.include_router(audio_router)
app.include_router(models_router)
app.include_router(tasks_router)
app.include_router(cuda_router)
+71
View File
@@ -0,0 +1,71 @@
"""Audio file serving endpoints."""
from pathlib import Path
from fastapi import APIRouter, Depends, HTTPException
from fastapi.responses import FileResponse
from sqlalchemy.orm import Session
from .. import models
from ..services import history
from ..database import get_db
router = APIRouter()
@router.get("/audio/version/{version_id}")
async def get_version_audio(version_id: str, db: Session = Depends(get_db)):
"""Serve audio for a specific version."""
from ..services import versions as versions_mod
version = versions_mod.get_version(version_id, db)
if not version:
raise HTTPException(status_code=404, detail="Version not found")
audio_path = Path(version.audio_path)
if not audio_path.exists():
raise HTTPException(status_code=404, detail="Audio file not found")
return FileResponse(
audio_path,
media_type="audio/wav",
filename=f"generation_{version.generation_id}_{version.label}.wav",
)
@router.get("/audio/{generation_id}")
async def get_audio(generation_id: str, db: Session = Depends(get_db)):
"""Serve generated audio file (serves the default version)."""
generation = await history.get_generation(generation_id, db)
if not generation:
raise HTTPException(status_code=404, detail="Generation not found")
audio_path = Path(generation.audio_path)
if not audio_path.exists():
raise HTTPException(status_code=404, detail="Audio file not found")
return FileResponse(
audio_path,
media_type="audio/wav",
filename=f"generation_{generation_id}.wav",
)
@router.get("/samples/{sample_id}")
async def get_sample_audio(sample_id: str, db: Session = Depends(get_db)):
"""Serve profile sample audio file."""
from ..database import ProfileSample as DBProfileSample
sample = db.query(DBProfileSample).filter_by(id=sample_id).first()
if not sample:
raise HTTPException(status_code=404, detail="Sample not found")
audio_path = Path(sample.audio_path)
if not audio_path.exists():
raise HTTPException(status_code=404, detail="Audio file not found")
return FileResponse(
audio_path,
media_type="audio/wav",
filename=f"sample_{sample_id}.wav",
)
+98
View File
@@ -0,0 +1,98 @@
"""Audio channel endpoints."""
from fastapi import APIRouter, Depends, HTTPException
from sqlalchemy.orm import Session
from .. import models
from ..services import channels
from ..database import get_db
router = APIRouter()
@router.get("/channels", response_model=list[models.AudioChannelResponse])
async def list_channels(db: Session = Depends(get_db)):
"""List all audio channels."""
return await channels.list_channels(db)
@router.post("/channels", response_model=models.AudioChannelResponse)
async def create_channel(
data: models.AudioChannelCreate,
db: Session = Depends(get_db),
):
"""Create a new audio channel."""
try:
return await channels.create_channel(data, db)
except ValueError as e:
raise HTTPException(status_code=400, detail=str(e))
@router.get("/channels/{channel_id}", response_model=models.AudioChannelResponse)
async def get_channel(
channel_id: str,
db: Session = Depends(get_db),
):
"""Get an audio channel by ID."""
channel = await channels.get_channel(channel_id, db)
if not channel:
raise HTTPException(status_code=404, detail="Channel not found")
return channel
@router.put("/channels/{channel_id}", response_model=models.AudioChannelResponse)
async def update_channel(
channel_id: str,
data: models.AudioChannelUpdate,
db: Session = Depends(get_db),
):
"""Update an audio channel."""
try:
channel = await channels.update_channel(channel_id, data, db)
if not channel:
raise HTTPException(status_code=404, detail="Channel not found")
return channel
except ValueError as e:
raise HTTPException(status_code=400, detail=str(e))
@router.delete("/channels/{channel_id}")
async def delete_channel(
channel_id: str,
db: Session = Depends(get_db),
):
"""Delete an audio channel."""
try:
success = await channels.delete_channel(channel_id, db)
if not success:
raise HTTPException(status_code=404, detail="Channel not found")
return {"message": "Channel deleted successfully"}
except ValueError as e:
raise HTTPException(status_code=400, detail=str(e))
@router.get("/channels/{channel_id}/voices")
async def get_channel_voices(
channel_id: str,
db: Session = Depends(get_db),
):
"""Get list of profile IDs assigned to a channel."""
try:
profile_ids = await channels.get_channel_voices(channel_id, db)
return {"profile_ids": profile_ids}
except ValueError as e:
raise HTTPException(status_code=400, detail=str(e))
@router.put("/channels/{channel_id}/voices")
async def set_channel_voices(
channel_id: str,
data: models.ChannelVoiceAssignment,
db: Session = Depends(get_db),
):
"""Set which voices are assigned to a channel."""
try:
await channels.set_channel_voices(channel_id, data, db)
return {"message": "Channel voices updated successfully"}
except ValueError as e:
raise HTTPException(status_code=400, detail=str(e))
+82
View File
@@ -0,0 +1,82 @@
"""CUDA backend management endpoints."""
import logging
from fastapi import APIRouter, HTTPException
from fastapi.responses import StreamingResponse
from ..services.task_queue import create_background_task
from ..utils.progress import get_progress_manager
router = APIRouter()
logger = logging.getLogger(__name__)
@router.get("/backend/cuda-status")
async def get_cuda_status():
"""Get CUDA backend download/availability status."""
from ..services import cuda
return cuda.get_cuda_status()
@router.post("/backend/download-cuda")
async def download_cuda_backend():
"""Download the CUDA backend binary."""
from ..services import cuda
if cuda.get_cuda_binary_path() is not None:
raise HTTPException(status_code=409, detail="CUDA backend already downloaded")
progress_manager = get_progress_manager()
existing = progress_manager.get_progress(cuda.PROGRESS_KEY)
if existing and existing.get("status") == "downloading":
raise HTTPException(status_code=409, detail="CUDA backend download already in progress")
async def _download():
try:
await cuda.download_cuda_binary()
except Exception as e:
logger.error("CUDA download failed: %s", e)
create_background_task(_download())
return {"message": "CUDA backend download started", "progress_key": "cuda-backend"}
@router.delete("/backend/cuda")
async def delete_cuda_backend():
"""Delete the downloaded CUDA backend binary."""
from ..services import cuda
if cuda.is_cuda_active():
raise HTTPException(
status_code=409,
detail="Cannot delete CUDA backend while it is active. Switch to CPU first.",
)
deleted = await cuda.delete_cuda_binary()
if not deleted:
raise HTTPException(status_code=404, detail="No CUDA backend found to delete")
return {"message": "CUDA backend deleted"}
@router.get("/backend/cuda-progress")
async def get_cuda_download_progress():
"""Get CUDA backend download progress via Server-Sent Events."""
progress_manager = get_progress_manager()
async def event_generator():
async for event in progress_manager.subscribe("cuda-backend"):
yield event
return StreamingResponse(
event_generator(),
media_type="text/event-stream",
headers={
"Cache-Control": "no-cache",
"Connection": "keep-alive",
"X-Accel-Buffering": "no",
},
)
+261
View File
@@ -0,0 +1,261 @@
"""Effects presets and generation version endpoints."""
import asyncio
import io
import uuid
from pathlib import Path
from fastapi import APIRouter, Depends, HTTPException
from fastapi.responses import StreamingResponse
from sqlalchemy.orm import Session
from .. import config, models
from ..services import history
from ..database import Generation as DBGeneration, get_db
router = APIRouter()
@router.post("/effects/preview/{generation_id}")
async def preview_effects(
generation_id: str,
data: models.ApplyEffectsRequest,
db: Session = Depends(get_db),
):
"""Apply effects to a generation's clean audio and stream back without saving."""
gen = db.query(DBGeneration).filter_by(id=generation_id).first()
if not gen:
raise HTTPException(status_code=404, detail="Generation not found")
if (gen.status or "completed") != "completed":
raise HTTPException(status_code=400, detail="Generation is not completed")
from ..services import versions as versions_mod
from ..utils.effects import apply_effects, validate_effects_chain
from ..utils.audio import load_audio
chain_dicts = [e.model_dump() for e in data.effects_chain]
error = validate_effects_chain(chain_dicts)
if error:
raise HTTPException(status_code=400, detail=error)
all_versions = versions_mod.list_versions(generation_id, db)
clean_version = next((v for v in all_versions if v.effects_chain is None), None)
source_path = clean_version.audio_path if clean_version else gen.audio_path
if not source_path or not Path(source_path).exists():
raise HTTPException(status_code=404, detail="Source audio file not found")
audio, sample_rate = await asyncio.to_thread(load_audio, source_path)
processed = await asyncio.to_thread(apply_effects, audio, sample_rate, chain_dicts)
import soundfile as sf
buf = io.BytesIO()
await asyncio.to_thread(lambda: sf.write(buf, processed, sample_rate, format="WAV"))
buf.seek(0)
return StreamingResponse(
buf,
media_type="audio/wav",
headers={
"Content-Disposition": f'inline; filename="preview_{generation_id}.wav"',
"Cache-Control": "no-cache, no-store",
},
)
@router.get("/effects/available", response_model=models.AvailableEffectsResponse)
async def get_available_effects():
"""List all available effect types with parameter definitions."""
from ..utils.effects import get_available_effects as _get_effects
return models.AvailableEffectsResponse(effects=[models.AvailableEffect(**e) for e in _get_effects()])
@router.get("/effects/presets", response_model=list[models.EffectPresetResponse])
async def list_effect_presets(db: Session = Depends(get_db)):
"""List all effect presets (built-in + user-created)."""
from ..services import effects as effects_mod
return effects_mod.list_presets(db)
@router.get("/effects/presets/{preset_id}", response_model=models.EffectPresetResponse)
async def get_effect_preset(preset_id: str, db: Session = Depends(get_db)):
"""Get a specific effect preset."""
from ..services import effects as effects_mod
preset = effects_mod.get_preset(preset_id, db)
if not preset:
raise HTTPException(status_code=404, detail="Preset not found")
return preset
@router.post("/effects/presets", response_model=models.EffectPresetResponse)
async def create_effect_preset(
data: models.EffectPresetCreate,
db: Session = Depends(get_db),
):
"""Create a new effect preset."""
from ..services import effects as effects_mod
try:
return effects_mod.create_preset(data, db)
except ValueError as e:
raise HTTPException(status_code=400, detail=str(e))
@router.put("/effects/presets/{preset_id}", response_model=models.EffectPresetResponse)
async def update_effect_preset(
preset_id: str,
data: models.EffectPresetUpdate,
db: Session = Depends(get_db),
):
"""Update an effect preset."""
from ..services import effects as effects_mod
try:
result = effects_mod.update_preset(preset_id, data, db)
if not result:
raise HTTPException(status_code=404, detail="Preset not found")
return result
except ValueError as e:
raise HTTPException(status_code=400, detail=str(e))
@router.delete("/effects/presets/{preset_id}")
async def delete_effect_preset(preset_id: str, db: Session = Depends(get_db)):
"""Delete a user effect preset."""
from ..services import effects as effects_mod
try:
if not effects_mod.delete_preset(preset_id, db):
raise HTTPException(status_code=404, detail="Preset not found")
return {"status": "deleted"}
except ValueError as e:
raise HTTPException(status_code=400, detail=str(e))
@router.get(
"/generations/{generation_id}/versions",
response_model=list[models.GenerationVersionResponse],
)
async def list_generation_versions(
generation_id: str,
db: Session = Depends(get_db),
):
"""List all versions for a generation."""
gen = await history.get_generation(generation_id, db)
if not gen:
raise HTTPException(status_code=404, detail="Generation not found")
from ..services import versions as versions_mod
return versions_mod.list_versions(generation_id, db)
@router.post(
"/generations/{generation_id}/versions/apply-effects",
response_model=models.GenerationVersionResponse,
)
async def apply_effects_to_generation(
generation_id: str,
data: models.ApplyEffectsRequest,
db: Session = Depends(get_db),
):
"""Apply an effects chain to an existing generation, creating a new version."""
gen = db.query(DBGeneration).filter_by(id=generation_id).first()
if not gen:
raise HTTPException(status_code=404, detail="Generation not found")
if (gen.status or "completed") != "completed":
raise HTTPException(status_code=400, detail="Generation is not completed")
from ..services import versions as versions_mod
from ..utils.effects import apply_effects, validate_effects_chain
from ..utils.audio import load_audio, save_audio
chain_dicts = [e.model_dump() for e in data.effects_chain]
error = validate_effects_chain(chain_dicts)
if error:
raise HTTPException(status_code=400, detail=error)
all_versions = versions_mod.list_versions(generation_id, db)
source_version_id = data.source_version_id
if source_version_id:
source_version = next((v for v in all_versions if v.id == source_version_id), None)
if not source_version:
raise HTTPException(status_code=404, detail="Source version not found")
source_path = source_version.audio_path
else:
clean_version = next((v for v in all_versions if v.effects_chain is None), None)
if not clean_version:
source_path = gen.audio_path
else:
source_path = clean_version.audio_path
source_version_id = clean_version.id
if not source_path or not Path(source_path).exists():
raise HTTPException(status_code=404, detail="Source audio file not found")
audio, sample_rate = await asyncio.to_thread(load_audio, source_path)
processed_audio = await asyncio.to_thread(apply_effects, audio, sample_rate, chain_dicts)
version_id = str(uuid.uuid4())
processed_path = config.get_generations_dir() / f"{generation_id}_{version_id[:8]}.wav"
await asyncio.to_thread(save_audio, processed_audio, str(processed_path), sample_rate)
label = data.label or f"version-{len(all_versions) + 1}"
version = versions_mod.create_version(
generation_id=generation_id,
label=label,
audio_path=str(processed_path),
db=db,
effects_chain=chain_dicts,
is_default=data.set_as_default,
source_version_id=source_version_id,
)
return version
@router.put(
"/generations/{generation_id}/versions/{version_id}/set-default",
response_model=models.GenerationVersionResponse,
)
async def set_default_version(
generation_id: str,
version_id: str,
db: Session = Depends(get_db),
):
"""Set a specific version as the default for a generation."""
from ..services import versions as versions_mod
version = versions_mod.get_version(version_id, db)
if not version or version.generation_id != generation_id:
raise HTTPException(status_code=404, detail="Version not found")
result = versions_mod.set_default_version(version_id, db)
if not result:
raise HTTPException(status_code=404, detail="Version not found")
return result
@router.delete("/generations/{generation_id}/versions/{version_id}")
async def delete_generation_version(
generation_id: str,
version_id: str,
db: Session = Depends(get_db),
):
"""Delete a version. Cannot delete the last remaining version."""
from ..services import versions as versions_mod
version = versions_mod.get_version(version_id, db)
if not version or version.generation_id != generation_id:
raise HTTPException(status_code=404, detail="Version not found")
if not versions_mod.delete_version(version_id, db):
raise HTTPException(
status_code=400,
detail="Cannot delete the last remaining version",
)
return {"status": "deleted"}
+276
View File
@@ -0,0 +1,276 @@
"""TTS generation endpoints."""
import asyncio
import uuid
from fastapi import APIRouter, Depends, HTTPException
from fastapi.responses import StreamingResponse
from sqlalchemy.orm import Session
from .. import models
from ..services import history, profiles, tts
from ..database import Generation as DBGeneration, VoiceProfile as DBVoiceProfile, get_db
from ..services.generation import run_generation
from ..services.task_queue import enqueue_generation
from ..utils.tasks import get_task_manager
router = APIRouter()
@router.post("/generate", response_model=models.GenerationResponse)
async def generate_speech(
data: models.GenerationRequest,
db: Session = Depends(get_db),
):
"""Generate speech from text using a voice profile."""
task_manager = get_task_manager()
generation_id = str(uuid.uuid4())
profile = await profiles.get_profile(data.profile_id, db)
if not profile:
raise HTTPException(status_code=404, detail="Profile not found")
from ..backends import engine_has_model_sizes
engine = data.engine or "qwen"
model_size = (data.model_size or "1.7B") if engine_has_model_sizes(engine) else None
generation = await history.create_generation(
profile_id=data.profile_id,
text=data.text,
language=data.language,
audio_path="",
duration=0,
seed=data.seed,
db=db,
instruct=data.instruct,
generation_id=generation_id,
status="generating",
engine=engine,
model_size=model_size if engine_has_model_sizes(engine) else None,
)
task_manager.start_generation(
task_id=generation_id,
profile_id=data.profile_id,
text=data.text,
)
effects_chain_config = None
if data.effects_chain is not None:
effects_chain_config = [e.model_dump() for e in data.effects_chain]
else:
import json as _json
profile_obj = db.query(DBVoiceProfile).filter_by(id=data.profile_id).first()
if profile_obj and profile_obj.effects_chain:
try:
effects_chain_config = _json.loads(profile_obj.effects_chain)
except Exception:
pass
enqueue_generation(
run_generation(
generation_id=generation_id,
profile_id=data.profile_id,
text=data.text,
language=data.language,
engine=engine,
model_size=model_size,
seed=data.seed,
normalize=data.normalize,
effects_chain=effects_chain_config,
instruct=data.instruct,
mode="generate",
max_chunk_chars=data.max_chunk_chars,
crossfade_ms=data.crossfade_ms,
)
)
return generation
@router.post("/generate/{generation_id}/retry", response_model=models.GenerationResponse)
async def retry_generation(generation_id: str, db: Session = Depends(get_db)):
"""Retry a failed generation using the same parameters."""
gen = db.query(DBGeneration).filter_by(id=generation_id).first()
if not gen:
raise HTTPException(status_code=404, detail="Generation not found")
if (gen.status or "completed") != "failed":
raise HTTPException(status_code=400, detail="Only failed generations can be retried")
gen.status = "generating"
gen.error = None
gen.audio_path = ""
gen.duration = 0
db.commit()
db.refresh(gen)
task_manager = get_task_manager()
task_manager.start_generation(
task_id=generation_id,
profile_id=gen.profile_id,
text=gen.text,
)
enqueue_generation(
run_generation(
generation_id=generation_id,
profile_id=gen.profile_id,
text=gen.text,
language=gen.language,
engine=gen.engine or "qwen",
model_size=gen.model_size or "1.7B",
seed=gen.seed,
instruct=gen.instruct,
mode="retry",
)
)
return models.GenerationResponse.model_validate(gen)
@router.post(
"/generate/{generation_id}/regenerate",
response_model=models.GenerationResponse,
)
async def regenerate_generation(generation_id: str, db: Session = Depends(get_db)):
"""Re-run TTS with the same parameters and save the result as a new version."""
gen = db.query(DBGeneration).filter_by(id=generation_id).first()
if not gen:
raise HTTPException(status_code=404, detail="Generation not found")
if (gen.status or "completed") != "completed":
raise HTTPException(status_code=400, detail="Generation must be completed to regenerate")
gen.status = "generating"
gen.error = None
db.commit()
db.refresh(gen)
task_manager = get_task_manager()
task_manager.start_generation(
task_id=generation_id,
profile_id=gen.profile_id,
text=gen.text,
)
version_id = str(uuid.uuid4())
enqueue_generation(
run_generation(
generation_id=generation_id,
profile_id=gen.profile_id,
text=gen.text,
language=gen.language,
engine=gen.engine or "qwen",
model_size=gen.model_size or "1.7B",
seed=gen.seed,
instruct=gen.instruct,
mode="regenerate",
version_id=version_id,
)
)
return models.GenerationResponse.model_validate(gen)
@router.get("/generate/{generation_id}/status")
async def get_generation_status(generation_id: str, db: Session = Depends(get_db)):
"""SSE endpoint that streams generation status updates."""
import json
async def event_stream():
while True:
db.expire_all()
gen = db.query(DBGeneration).filter_by(id=generation_id).first()
if not gen:
yield f"data: {json.dumps({'status': 'not_found', 'id': generation_id})}\n\n"
return
payload = {
"id": gen.id,
"status": gen.status or "completed",
"duration": gen.duration,
"error": gen.error,
}
yield f"data: {json.dumps(payload)}\n\n"
if (gen.status or "completed") in ("completed", "failed"):
return
await asyncio.sleep(1)
return StreamingResponse(
event_stream(),
media_type="text/event-stream",
headers={
"Cache-Control": "no-cache",
"Connection": "keep-alive",
"X-Accel-Buffering": "no",
},
)
@router.post("/generate/stream")
async def stream_speech(
data: models.GenerationRequest,
db: Session = Depends(get_db),
):
"""Generate speech and stream the WAV audio directly without saving to disk."""
from ..backends import get_tts_backend_for_engine, ensure_model_cached_or_raise, load_engine_model, engine_needs_trim
profile = await profiles.get_profile(data.profile_id, db)
if not profile:
raise HTTPException(status_code=404, detail="Profile not found")
engine = data.engine or "qwen"
tts_model = get_tts_backend_for_engine(engine)
model_size = data.model_size or "1.7B"
await ensure_model_cached_or_raise(engine, model_size)
await load_engine_model(engine, model_size)
voice_prompt = await profiles.create_voice_prompt_for_profile(
data.profile_id,
db,
engine=engine,
)
from ..utils.chunked_tts import generate_chunked
trim_fn = None
if engine_needs_trim(engine):
from ..utils.audio import trim_tts_output
trim_fn = trim_tts_output
audio, sample_rate = await generate_chunked(
tts_model,
data.text,
voice_prompt,
language=data.language,
seed=data.seed,
instruct=data.instruct,
max_chunk_chars=data.max_chunk_chars,
crossfade_ms=data.crossfade_ms,
trim_fn=trim_fn,
)
if data.normalize:
from ..utils.audio import normalize_audio
audio = normalize_audio(audio)
wav_bytes = tts.audio_to_wav_bytes(audio, sample_rate)
async def _wav_stream():
chunk_size = 64 * 1024
for i in range(0, len(wav_bytes), chunk_size):
yield wav_bytes[i : i + chunk_size]
return StreamingResponse(
_wav_stream(),
media_type="audio/wav",
headers={"Content-Disposition": 'attachment; filename="speech.wav"'},
)
+225
View File
@@ -0,0 +1,225 @@
"""Health and infrastructure endpoints."""
import asyncio
import os
import signal
import torch
from fastapi import APIRouter, Depends
from sqlalchemy.orm import Session
from .. import config, models
from ..services import tts
from ..database import get_db
from ..utils.platform_detect import get_backend_type
router = APIRouter()
@router.get("/")
async def root():
"""Root endpoint."""
from .. import __version__
return {"message": "voicebox API", "version": __version__}
@router.post("/shutdown")
async def shutdown():
"""Gracefully shutdown the server."""
async def shutdown_async():
await asyncio.sleep(0.1)
os.kill(os.getpid(), signal.SIGTERM)
asyncio.create_task(shutdown_async())
return {"message": "Shutting down..."}
@router.post("/watchdog/disable")
async def watchdog_disable():
"""Disable the parent process watchdog so the server keeps running."""
from backend.server import disable_watchdog
disable_watchdog()
return {"message": "Watchdog disabled"}
@router.get("/health", response_model=models.HealthResponse)
async def health():
"""Health check endpoint."""
from huggingface_hub import constants as hf_constants
from pathlib import Path
tts_model = tts.get_tts_model()
backend_type = get_backend_type()
has_cuda = torch.cuda.is_available()
has_mps = hasattr(torch.backends, "mps") and torch.backends.mps.is_available()
has_xpu = False
xpu_name = None
try:
import intel_extension_for_pytorch as ipex # noqa: F401 -- side-effect import enables XPU
if hasattr(torch, "xpu") and torch.xpu.is_available():
has_xpu = True
try:
xpu_name = torch.xpu.get_device_name(0)
except Exception:
xpu_name = "Intel GPU"
except ImportError:
pass
has_directml = False
directml_name = None
try:
import torch_directml
if torch_directml.device_count() > 0:
has_directml = True
try:
directml_name = torch_directml.device_name(0)
except Exception:
directml_name = "DirectML GPU"
except ImportError:
pass
gpu_available = has_cuda or has_mps or has_xpu or has_directml or backend_type == "mlx"
gpu_type = None
if has_cuda:
gpu_type = f"CUDA ({torch.cuda.get_device_name(0)})"
elif has_mps:
gpu_type = "MPS (Apple Silicon)"
elif backend_type == "mlx":
gpu_type = "Metal (Apple Silicon via MLX)"
elif has_xpu:
gpu_type = f"XPU ({xpu_name})"
elif has_directml:
gpu_type = f"DirectML ({directml_name})"
vram_used = None
if has_cuda:
vram_used = torch.cuda.memory_allocated() / 1024 / 1024
model_loaded = False
model_size = None
try:
if tts_model.is_loaded():
model_loaded = True
model_size = getattr(tts_model, "_current_model_size", None)
if not model_size:
model_size = getattr(tts_model, "model_size", None)
except Exception:
model_loaded = False
model_size = None
model_downloaded = None
try:
from ..backends import get_model_config
default_config = get_model_config("qwen-tts-1.7B")
default_model_id = default_config.hf_repo_id if default_config else "Qwen/Qwen3-TTS-12Hz-1.7B-Base"
try:
from huggingface_hub import scan_cache_dir
cache_info = scan_cache_dir()
for repo in cache_info.repos:
if repo.repo_id == default_model_id:
model_downloaded = True
break
except (ImportError, Exception):
cache_dir = hf_constants.HF_HUB_CACHE
repo_cache = Path(cache_dir) / ("models--" + default_model_id.replace("/", "--"))
if repo_cache.exists():
has_model_files = (
any(repo_cache.rglob("*.bin"))
or any(repo_cache.rglob("*.safetensors"))
or any(repo_cache.rglob("*.pt"))
or any(repo_cache.rglob("*.pth"))
or any(repo_cache.rglob("*.npz"))
)
model_downloaded = has_model_files
except Exception:
pass
return models.HealthResponse(
status="healthy",
model_loaded=model_loaded,
model_downloaded=model_downloaded,
model_size=model_size,
gpu_available=gpu_available,
gpu_type=gpu_type,
vram_used_mb=vram_used,
backend_type=backend_type,
backend_variant=os.environ.get("VOICEBOX_BACKEND_VARIANT", "cuda" if torch.cuda.is_available() else "cpu"),
)
@router.get("/health/filesystem", response_model=models.FilesystemHealthResponse)
async def filesystem_health():
"""Check filesystem health: directory existence, write permissions, and disk space."""
import shutil
dirs_to_check = {
"generations": config.get_generations_dir(),
"profiles": config.get_profiles_dir(),
"data": config.get_data_dir(),
}
checks: list[models.DirectoryCheck] = []
all_ok = True
for _label, dir_path in dirs_to_check.items():
exists = dir_path.exists()
writable = False
error = None
if exists:
probe = dir_path / ".voicebox_probe"
try:
probe.write_text("ok")
probe.unlink()
writable = True
except PermissionError:
error = "Permission denied"
except OSError as e:
error = str(e)
finally:
try:
probe.unlink(missing_ok=True)
except Exception:
pass
else:
error = "Directory does not exist"
if not exists or not writable:
all_ok = False
checks.append(
models.DirectoryCheck(
path=str(dir_path),
exists=exists,
writable=writable,
error=error,
)
)
disk_free_mb = None
disk_total_mb = None
try:
usage = shutil.disk_usage(str(config.get_data_dir()))
disk_free_mb = round(usage.free / (1024 * 1024), 1)
disk_total_mb = round(usage.total / (1024 * 1024), 1)
if disk_free_mb < 500:
all_ok = False
except OSError:
all_ok = False
return models.FilesystemHealthResponse(
healthy=all_ok,
disk_free_mb=disk_free_mb,
disk_total_mb=disk_total_mb,
directories=checks,
)
+178
View File
@@ -0,0 +1,178 @@
"""Generation history endpoints."""
import io
from pathlib import Path
from fastapi import APIRouter, Depends, File, HTTPException, UploadFile
from fastapi.responses import FileResponse, StreamingResponse
from sqlalchemy.orm import Session
from .. import models
from ..services import export_import, history
from ..app import safe_content_disposition
from ..database import Generation as DBGeneration, VoiceProfile as DBVoiceProfile, get_db
router = APIRouter()
@router.get("/history", response_model=models.HistoryListResponse)
async def list_history(
profile_id: str | None = None,
search: str | None = None,
limit: int = 50,
offset: int = 0,
db: Session = Depends(get_db),
):
"""List generation history with optional filters."""
query = models.HistoryQuery(
profile_id=profile_id,
search=search,
limit=limit,
offset=offset,
)
return await history.list_generations(query, db)
@router.get("/history/stats")
async def get_stats(db: Session = Depends(get_db)):
"""Get generation statistics."""
return await history.get_generation_stats(db)
@router.post("/history/import")
async def import_generation(
file: UploadFile = File(...),
db: Session = Depends(get_db),
):
"""Import a generation from a ZIP archive."""
MAX_FILE_SIZE = 50 * 1024 * 1024
content = await file.read()
if len(content) > MAX_FILE_SIZE:
raise HTTPException(
status_code=400, detail=f"File too large. Maximum size is {MAX_FILE_SIZE / (1024 * 1024)}MB"
)
try:
result = await export_import.import_generation_from_zip(content, db)
return result
except ValueError as e:
raise HTTPException(status_code=400, detail=str(e))
except Exception as e:
raise HTTPException(status_code=500, detail=str(e))
@router.get("/history/{generation_id}", response_model=models.HistoryResponse)
async def get_generation(
generation_id: str,
db: Session = Depends(get_db),
):
"""Get a generation by ID."""
result = (
db.query(DBGeneration, DBVoiceProfile.name.label("profile_name"))
.join(DBVoiceProfile, DBGeneration.profile_id == DBVoiceProfile.id)
.filter(DBGeneration.id == generation_id)
.first()
)
if not result:
raise HTTPException(status_code=404, detail="Generation not found")
gen, profile_name = result
return models.HistoryResponse(
id=gen.id,
profile_id=gen.profile_id,
profile_name=profile_name,
text=gen.text,
language=gen.language,
audio_path=gen.audio_path,
duration=gen.duration,
seed=gen.seed,
instruct=gen.instruct,
created_at=gen.created_at,
)
@router.post("/history/{generation_id}/favorite")
async def toggle_favorite(
generation_id: str,
db: Session = Depends(get_db),
):
"""Toggle the favorite status of a generation."""
gen = db.query(DBGeneration).filter_by(id=generation_id).first()
if not gen:
raise HTTPException(status_code=404, detail="Generation not found")
gen.is_favorited = not gen.is_favorited
db.commit()
return {"is_favorited": gen.is_favorited}
@router.delete("/history/{generation_id}")
async def delete_generation(
generation_id: str,
db: Session = Depends(get_db),
):
"""Delete a generation."""
success = await history.delete_generation(generation_id, db)
if not success:
raise HTTPException(status_code=404, detail="Generation not found")
return {"message": "Generation deleted successfully"}
@router.get("/history/{generation_id}/export")
async def export_generation(
generation_id: str,
db: Session = Depends(get_db),
):
"""Export a generation as a ZIP archive."""
generation = db.query(DBGeneration).filter_by(id=generation_id).first()
if not generation:
raise HTTPException(status_code=404, detail="Generation not found")
try:
zip_bytes = export_import.export_generation_to_zip(generation_id, db)
except ValueError as e:
raise HTTPException(status_code=400, detail=str(e))
except Exception as e:
raise HTTPException(status_code=500, detail=str(e))
safe_text = "".join(c for c in generation.text[:30] if c.isalnum() or c in (" ", "-", "_")).strip()
if not safe_text:
safe_text = "generation"
filename = f"generation-{safe_text}.voicebox.zip"
return StreamingResponse(
io.BytesIO(zip_bytes),
media_type="application/zip",
headers={"Content-Disposition": safe_content_disposition("attachment", filename)},
)
@router.get("/history/{generation_id}/export-audio")
async def export_generation_audio(
generation_id: str,
db: Session = Depends(get_db),
):
"""Export only the audio file from a generation."""
generation = db.query(DBGeneration).filter_by(id=generation_id).first()
if not generation:
raise HTTPException(status_code=404, detail="Generation not found")
if not generation.audio_path:
raise HTTPException(status_code=404, detail="Generation has no audio file")
audio_path = Path(generation.audio_path)
if not audio_path.is_file():
raise HTTPException(status_code=404, detail="Audio file not found")
safe_text = "".join(c for c in generation.text[:30] if c.isalnum() or c in (" ", "-", "_")).strip()
if not safe_text:
safe_text = "generation"
filename = f"{safe_text}.wav"
return FileResponse(
audio_path,
media_type="audio/wav",
headers={"Content-Disposition": safe_content_disposition("attachment", filename)},
)
+474
View File
@@ -0,0 +1,474 @@
"""Model management endpoints."""
import asyncio
import shutil
from pathlib import Path
from fastapi import APIRouter, Depends, HTTPException
from fastapi.responses import StreamingResponse
from sqlalchemy.orm import Session
from .. import models
from ..utils.platform_detect import get_backend_type
from ..services.task_queue import create_background_task
from ..utils.progress import get_progress_manager
from ..utils.tasks import get_task_manager
router = APIRouter()
def _get_dir_size(path: Path) -> int:
"""Get total size of a directory in bytes."""
total = 0
for f in path.rglob("*"):
if f.is_file():
total += f.stat().st_size
return total
def _copy_with_progress(src: Path, dst: Path, progress_manager, copied_so_far: int, total_bytes: int) -> int:
"""Copy a directory tree with byte-level progress tracking."""
dst.mkdir(parents=True, exist_ok=True)
for item in src.iterdir():
dest_item = dst / item.name
if item.is_dir():
copied_so_far = _copy_with_progress(item, dest_item, progress_manager, copied_so_far, total_bytes)
else:
size = item.stat().st_size
shutil.copy2(str(item), str(dest_item))
copied_so_far += size
progress_manager.update_progress(
"migration",
copied_so_far,
total_bytes,
filename=item.name,
status="downloading",
)
return copied_so_far
@router.post("/models/load")
async def load_model(model_size: str = "1.7B"):
"""Manually load TTS model."""
from ..services import tts
try:
tts_model = tts.get_tts_model()
await tts_model.load_model_async(model_size)
return {"message": f"Model {model_size} loaded successfully"}
except Exception as e:
raise HTTPException(status_code=500, detail=str(e))
@router.post("/models/unload")
async def unload_model():
"""Unload the default Qwen TTS model to free memory."""
from ..services import tts
try:
tts.unload_tts_model()
return {"message": "Model unloaded successfully"}
except Exception as e:
raise HTTPException(status_code=500, detail=str(e))
@router.post("/models/{model_name}/unload")
async def unload_model_by_name(model_name: str):
"""Unload a specific model from memory without deleting it from disk."""
from ..backends import get_model_config, unload_model_by_config
config = get_model_config(model_name)
if not config:
raise HTTPException(status_code=400, detail=f"Unknown model: {model_name}")
try:
was_loaded = unload_model_by_config(config)
if not was_loaded:
return {"message": f"Model {model_name} is not loaded"}
return {"message": f"Model {model_name} unloaded successfully"}
except Exception as e:
raise HTTPException(status_code=500, detail=str(e)) from e
@router.get("/models/progress/{model_name}")
async def get_model_progress(model_name: str):
"""Get model download progress via Server-Sent Events."""
progress_manager = get_progress_manager()
async def event_generator():
async for event in progress_manager.subscribe(model_name):
yield event
return StreamingResponse(
event_generator(),
media_type="text/event-stream",
headers={
"Cache-Control": "no-cache",
"Connection": "keep-alive",
"X-Accel-Buffering": "no",
},
)
@router.get("/models/cache-dir")
async def get_models_cache_dir():
"""Get the path to the HuggingFace model cache directory."""
from huggingface_hub import constants as hf_constants
return {"path": str(Path(hf_constants.HF_HUB_CACHE))}
@router.post("/models/migrate")
async def migrate_models(request: models.ModelMigrateRequest):
"""Move all downloaded models to a new directory with byte-level progress via SSE."""
from huggingface_hub import constants as hf_constants
source = Path(hf_constants.HF_HUB_CACHE)
destination = Path(request.destination)
if not source.exists():
raise HTTPException(status_code=404, detail="Current model cache directory not found")
if source.resolve() == destination.resolve():
raise HTTPException(status_code=400, detail="Source and destination are the same directory")
if destination.resolve().is_relative_to(source.resolve()):
raise HTTPException(status_code=400, detail="Destination cannot be inside the current cache directory")
model_dirs = [d for d in source.iterdir() if d.name.startswith("models--") and d.is_dir()]
if not model_dirs:
return {"moved": 0, "errors": [], "source": str(source), "destination": str(destination)}
destination.mkdir(parents=True, exist_ok=True)
progress_manager = get_progress_manager()
same_fs = False
try:
same_fs = source.stat().st_dev == destination.stat().st_dev
except OSError:
pass
async def migrate_background():
moved = 0
errors = []
try:
if same_fs:
total = len(model_dirs)
for i, item in enumerate(model_dirs):
dest_item = destination / item.name
try:
if dest_item.exists():
shutil.rmtree(dest_item)
shutil.move(str(item), str(dest_item))
moved += 1
progress_manager.update_progress(
"migration",
i + 1,
total,
filename=item.name,
status="downloading",
)
except Exception as e:
errors.append(f"{item.name}: {str(e)}")
else:
total_bytes = sum(_get_dir_size(d) for d in model_dirs)
progress_manager.update_progress(
"migration", 0, total_bytes, filename="Calculating...", status="downloading"
)
copied = 0
for item in model_dirs:
dest_item = destination / item.name
try:
if dest_item.exists():
shutil.rmtree(dest_item)
copied = await asyncio.to_thread(
_copy_with_progress, item, dest_item, progress_manager, copied, total_bytes
)
await asyncio.to_thread(shutil.rmtree, str(item))
moved += 1
except Exception as e:
errors.append(f"{item.name}: {str(e)}")
progress_manager.update_progress("migration", 1, 1, status="complete")
progress_manager.mark_complete("migration")
except Exception as e:
progress_manager.update_progress("migration", 0, 0, status="error")
progress_manager.mark_error("migration", str(e))
create_background_task(migrate_background())
return {"source": str(source), "destination": str(destination)}
@router.get("/models/migrate/progress")
async def get_migration_progress():
"""Get model migration progress via Server-Sent Events."""
progress_manager = get_progress_manager()
async def event_generator():
async for event in progress_manager.subscribe("migration"):
yield event
return StreamingResponse(
event_generator(),
media_type="text/event-stream",
headers={
"Cache-Control": "no-cache",
"Connection": "keep-alive",
"X-Accel-Buffering": "no",
},
)
@router.get("/models/status", response_model=models.ModelStatusListResponse)
async def get_model_status():
"""Get status of all available models."""
from huggingface_hub import constants as hf_constants
backend_type = get_backend_type()
task_manager = get_task_manager()
active_download_names = {task.model_name for task in task_manager.get_active_downloads()}
try:
from huggingface_hub import scan_cache_dir
use_scan_cache = True
except ImportError:
use_scan_cache = False
from ..backends import get_all_model_configs, check_model_loaded
registry_configs = get_all_model_configs()
model_configs = [
{
"model_name": cfg.model_name,
"display_name": cfg.display_name,
"hf_repo_id": cfg.hf_repo_id,
"model_size": cfg.model_size,
"check_loaded": lambda c=cfg: check_model_loaded(c),
}
for cfg in registry_configs
]
model_to_repo = {cfg["model_name"]: cfg["hf_repo_id"] for cfg in model_configs}
active_download_repos = {model_to_repo.get(name) for name in active_download_names if name in model_to_repo}
cache_info = None
if use_scan_cache:
try:
cache_info = scan_cache_dir()
except Exception:
pass
statuses = []
for config in model_configs:
try:
downloaded = False
size_mb = None
loaded = False
if cache_info:
repo_id = config["hf_repo_id"]
for repo in cache_info.repos:
if repo.repo_id == repo_id:
has_model_weights = False
for rev in repo.revisions:
for f in rev.files:
fname = f.file_name.lower()
if fname.endswith((".safetensors", ".bin", ".pt", ".pth", ".npz")):
has_model_weights = True
break
if has_model_weights:
break
has_incomplete = False
try:
cache_dir = hf_constants.HF_HUB_CACHE
blobs_dir = Path(cache_dir) / ("models--" + repo_id.replace("/", "--")) / "blobs"
if blobs_dir.exists():
has_incomplete = any(blobs_dir.glob("*.incomplete"))
except Exception:
pass
if has_model_weights and not has_incomplete:
downloaded = True
try:
total_size = sum(revision.size_on_disk for revision in repo.revisions)
size_mb = total_size / (1024 * 1024)
except Exception:
pass
break
if not downloaded:
try:
cache_dir = hf_constants.HF_HUB_CACHE
repo_cache = Path(cache_dir) / ("models--" + config["hf_repo_id"].replace("/", "--"))
if repo_cache.exists():
blobs_dir = repo_cache / "blobs"
has_incomplete = blobs_dir.exists() and any(blobs_dir.glob("*.incomplete"))
if not has_incomplete:
snapshots_dir = repo_cache / "snapshots"
has_model_files = False
if snapshots_dir.exists():
has_model_files = (
any(snapshots_dir.rglob("*.bin"))
or any(snapshots_dir.rglob("*.safetensors"))
or any(snapshots_dir.rglob("*.pt"))
or any(snapshots_dir.rglob("*.pth"))
or any(snapshots_dir.rglob("*.npz"))
)
if has_model_files:
downloaded = True
try:
total_size = sum(
f.stat().st_size
for f in repo_cache.rglob("*")
if f.is_file() and not f.name.endswith(".incomplete")
)
size_mb = total_size / (1024 * 1024)
except Exception:
pass
except Exception:
pass
try:
loaded = config["check_loaded"]()
except Exception:
loaded = False
is_downloading = config["hf_repo_id"] in active_download_repos
if is_downloading:
downloaded = False
size_mb = None
statuses.append(
models.ModelStatus(
model_name=config["model_name"],
display_name=config["display_name"],
hf_repo_id=config["hf_repo_id"],
downloaded=downloaded,
downloading=is_downloading,
size_mb=size_mb,
loaded=loaded,
)
)
except Exception:
try:
loaded = config["check_loaded"]()
except Exception:
loaded = False
is_downloading = config["hf_repo_id"] in active_download_repos
statuses.append(
models.ModelStatus(
model_name=config["model_name"],
display_name=config["display_name"],
hf_repo_id=config["hf_repo_id"],
downloaded=False,
downloading=is_downloading,
size_mb=None,
loaded=loaded,
)
)
return models.ModelStatusListResponse(models=statuses)
@router.post("/models/download")
async def trigger_model_download(request: models.ModelDownloadRequest):
"""Trigger download of a specific model."""
from ..backends import get_model_config, get_model_load_func
task_manager = get_task_manager()
progress_manager = get_progress_manager()
config = get_model_config(request.model_name)
if not config:
raise HTTPException(status_code=400, detail=f"Unknown model: {request.model_name}")
load_func = get_model_load_func(config)
async def download_in_background():
try:
result = load_func()
if asyncio.iscoroutine(result):
await result
task_manager.complete_download(request.model_name)
except Exception as e:
task_manager.error_download(request.model_name, str(e))
task_manager.start_download(request.model_name)
progress_manager.update_progress(
model_name=request.model_name,
current=0,
total=0,
filename="Connecting to HuggingFace...",
status="downloading",
)
create_background_task(download_in_background())
return {"message": f"Model {request.model_name} download started"}
@router.post("/models/download/cancel")
async def cancel_model_download(request: models.ModelDownloadRequest):
"""Cancel or dismiss an errored/stale download task."""
task_manager = get_task_manager()
progress_manager = get_progress_manager()
removed = task_manager.cancel_download(request.model_name)
progress_removed = False
with progress_manager._lock:
if request.model_name in progress_manager._progress:
del progress_manager._progress[request.model_name]
progress_removed = True
if removed or progress_removed:
return {"message": f"Download task for {request.model_name} cancelled"}
return {"message": f"No active task found for {request.model_name}"}
@router.delete("/models/{model_name}")
async def delete_model(model_name: str):
"""Delete a downloaded model from the HuggingFace cache."""
from huggingface_hub import constants as hf_constants
from ..backends import get_model_config, unload_model_by_config
config = get_model_config(model_name)
if not config:
raise HTTPException(status_code=400, detail=f"Unknown model: {model_name}")
hf_repo_id = config.hf_repo_id
try:
unload_model_by_config(config)
cache_dir = hf_constants.HF_HUB_CACHE
repo_cache_dir = Path(cache_dir) / ("models--" + hf_repo_id.replace("/", "--"))
if not repo_cache_dir.exists():
raise HTTPException(status_code=404, detail=f"Model {model_name} not found in cache")
try:
shutil.rmtree(repo_cache_dir)
except OSError as e:
raise HTTPException(status_code=500, detail=f"Failed to delete model cache directory: {str(e)}")
return {"message": f"Model {model_name} deleted successfully"}
except HTTPException:
raise
except Exception as e:
raise HTTPException(status_code=500, detail=f"Failed to delete model: {str(e)}")
+309
View File
@@ -0,0 +1,309 @@
"""Voice profile endpoints."""
import io
import tempfile
from datetime import datetime
from pathlib import Path
from fastapi import APIRouter, Depends, File, Form, HTTPException, UploadFile
from fastapi.responses import FileResponse, StreamingResponse
from sqlalchemy.orm import Session
from .. import config, models
from ..app import safe_content_disposition
from ..database import VoiceProfile as DBVoiceProfile, get_db
from ..services import channels, export_import, profiles
from ..services.profiles import _profile_to_response
router = APIRouter()
@router.post("/profiles", response_model=models.VoiceProfileResponse)
async def create_profile(
data: models.VoiceProfileCreate,
db: Session = Depends(get_db),
):
"""Create a new voice profile."""
try:
return await profiles.create_profile(data, db)
except ValueError as e:
raise HTTPException(status_code=400, detail=str(e))
except Exception as e:
raise HTTPException(status_code=400, detail=str(e))
@router.get("/profiles", response_model=list[models.VoiceProfileResponse])
async def list_profiles(db: Session = Depends(get_db)):
"""List all voice profiles."""
return await profiles.list_profiles(db)
@router.post("/profiles/import", response_model=models.VoiceProfileResponse)
async def import_profile(
file: UploadFile = File(...),
db: Session = Depends(get_db),
):
"""Import a voice profile from a ZIP archive."""
MAX_FILE_SIZE = 100 * 1024 * 1024
content = await file.read()
if len(content) > MAX_FILE_SIZE:
raise HTTPException(
status_code=400, detail=f"File too large. Maximum size is {MAX_FILE_SIZE / (1024 * 1024)}MB"
)
try:
profile = await export_import.import_profile_from_zip(content, db)
return profile
except ValueError as e:
raise HTTPException(status_code=400, detail=str(e))
except Exception as e:
raise HTTPException(status_code=500, detail=str(e))
@router.get("/profiles/{profile_id}", response_model=models.VoiceProfileResponse)
async def get_profile(
profile_id: str,
db: Session = Depends(get_db),
):
"""Get a voice profile by ID."""
profile = await profiles.get_profile(profile_id, db)
if not profile:
raise HTTPException(status_code=404, detail="Profile not found")
return profile
@router.put("/profiles/{profile_id}", response_model=models.VoiceProfileResponse)
async def update_profile(
profile_id: str,
data: models.VoiceProfileCreate,
db: Session = Depends(get_db),
):
"""Update a voice profile."""
try:
profile = await profiles.update_profile(profile_id, data, db)
if not profile:
raise HTTPException(status_code=404, detail="Profile not found")
return profile
except ValueError as e:
raise HTTPException(status_code=400, detail=str(e))
@router.delete("/profiles/{profile_id}")
async def delete_profile(
profile_id: str,
db: Session = Depends(get_db),
):
"""Delete a voice profile."""
success = await profiles.delete_profile(profile_id, db)
if not success:
raise HTTPException(status_code=404, detail="Profile not found")
return {"message": "Profile deleted successfully"}
@router.post("/profiles/{profile_id}/samples", response_model=models.ProfileSampleResponse)
async def add_profile_sample(
profile_id: str,
file: UploadFile = File(...),
reference_text: str = Form(...),
db: Session = Depends(get_db),
):
"""Add a sample to a voice profile."""
_allowed_audio_exts = {".wav", ".mp3", ".m4a", ".ogg", ".flac", ".aac", ".webm", ".opus"}
_uploaded_ext = Path(file.filename or "").suffix.lower()
file_suffix = _uploaded_ext if _uploaded_ext in _allowed_audio_exts else ".wav"
with tempfile.NamedTemporaryFile(suffix=file_suffix, delete=False) as tmp:
content = await file.read()
tmp.write(content)
tmp_path = tmp.name
try:
sample = await profiles.add_profile_sample(
profile_id,
tmp_path,
reference_text,
db,
)
return sample
except ValueError as e:
raise HTTPException(status_code=400, detail=str(e))
except Exception as e:
raise HTTPException(status_code=500, detail=f"Failed to process audio file: {str(e)}")
finally:
Path(tmp_path).unlink(missing_ok=True)
@router.get("/profiles/{profile_id}/samples", response_model=list[models.ProfileSampleResponse])
async def get_profile_samples(
profile_id: str,
db: Session = Depends(get_db),
):
"""Get all samples for a profile."""
return await profiles.get_profile_samples(profile_id, db)
@router.delete("/profiles/samples/{sample_id}")
async def delete_profile_sample(
sample_id: str,
db: Session = Depends(get_db),
):
"""Delete a profile sample."""
success = await profiles.delete_profile_sample(sample_id, db)
if not success:
raise HTTPException(status_code=404, detail="Sample not found")
return {"message": "Sample deleted successfully"}
@router.put("/profiles/samples/{sample_id}", response_model=models.ProfileSampleResponse)
async def update_profile_sample(
sample_id: str,
data: models.ProfileSampleUpdate,
db: Session = Depends(get_db),
):
"""Update a profile sample's reference text."""
sample = await profiles.update_profile_sample(sample_id, data.reference_text, db)
if not sample:
raise HTTPException(status_code=404, detail="Sample not found")
return sample
@router.post("/profiles/{profile_id}/avatar", response_model=models.VoiceProfileResponse)
async def upload_profile_avatar(
profile_id: str,
file: UploadFile = File(...),
db: Session = Depends(get_db),
):
"""Upload or update avatar image for a profile."""
with tempfile.NamedTemporaryFile(delete=False, suffix=Path(file.filename).suffix) as tmp:
content = await file.read()
tmp.write(content)
tmp_path = tmp.name
try:
profile = await profiles.upload_avatar(profile_id, tmp_path, db)
return profile
except ValueError as e:
raise HTTPException(status_code=400, detail=str(e))
finally:
Path(tmp_path).unlink(missing_ok=True)
@router.get("/profiles/{profile_id}/avatar")
async def get_profile_avatar(
profile_id: str,
db: Session = Depends(get_db),
):
"""Get avatar image for a profile."""
profile = await profiles.get_profile(profile_id, db)
if not profile:
raise HTTPException(status_code=404, detail="Profile not found")
if not profile.avatar_path:
raise HTTPException(status_code=404, detail="No avatar found for this profile")
avatar_path = Path(profile.avatar_path)
if not avatar_path.exists():
raise HTTPException(status_code=404, detail="Avatar file not found")
return FileResponse(avatar_path)
@router.delete("/profiles/{profile_id}/avatar")
async def delete_profile_avatar(
profile_id: str,
db: Session = Depends(get_db),
):
"""Delete avatar image for a profile."""
success = await profiles.delete_avatar(profile_id, db)
if not success:
raise HTTPException(status_code=404, detail="Profile not found or no avatar to delete")
return {"message": "Avatar deleted successfully"}
@router.get("/profiles/{profile_id}/export")
async def export_profile(
profile_id: str,
db: Session = Depends(get_db),
):
"""Export a voice profile as a ZIP archive."""
try:
profile = await profiles.get_profile(profile_id, db)
if not profile:
raise HTTPException(status_code=404, detail="Profile not found")
zip_bytes = export_import.export_profile_to_zip(profile_id, db)
safe_name = "".join(c for c in profile.name if c.isalnum() or c in (" ", "-", "_")).strip()
if not safe_name:
safe_name = "profile"
filename = f"profile-{safe_name}.voicebox.zip"
return StreamingResponse(
io.BytesIO(zip_bytes),
media_type="application/zip",
headers={"Content-Disposition": safe_content_disposition("attachment", filename)},
)
except ValueError as e:
raise HTTPException(status_code=400, detail=str(e))
except Exception as e:
raise HTTPException(status_code=500, detail=str(e))
@router.get("/profiles/{profile_id}/channels")
async def get_profile_channels(
profile_id: str,
db: Session = Depends(get_db),
):
"""Get list of channel IDs assigned to a profile."""
try:
channel_ids = await channels.get_profile_channels(profile_id, db)
return {"channel_ids": channel_ids}
except ValueError as e:
raise HTTPException(status_code=400, detail=str(e))
@router.put("/profiles/{profile_id}/channels")
async def set_profile_channels(
profile_id: str,
data: models.ProfileChannelAssignment,
db: Session = Depends(get_db),
):
"""Set which channels a profile is assigned to."""
try:
await channels.set_profile_channels(profile_id, data, db)
return {"message": "Profile channels updated successfully"}
except ValueError as e:
raise HTTPException(status_code=400, detail=str(e))
@router.put("/profiles/{profile_id}/effects", response_model=models.VoiceProfileResponse)
async def update_profile_effects(
profile_id: str,
data: models.ProfileEffectsUpdate,
db: Session = Depends(get_db),
):
"""Set or clear the default effects chain for a voice profile."""
import json as _json
profile = db.query(DBVoiceProfile).filter_by(id=profile_id).first()
if not profile:
raise HTTPException(status_code=404, detail="Profile not found")
if data.effects_chain is not None:
from ..utils.effects import validate_effects_chain
chain_dicts = [e.model_dump() for e in data.effects_chain]
error = validate_effects_chain(chain_dicts)
if error:
raise HTTPException(status_code=400, detail=error)
profile.effects_chain = _json.dumps(chain_dicts)
else:
profile.effects_chain = None
profile.updated_at = datetime.utcnow()
db.commit()
db.refresh(profile)
return _profile_to_response(profile)
+223
View File
@@ -0,0 +1,223 @@
"""Story endpoints."""
import io
from fastapi import APIRouter, Depends, HTTPException
from fastapi.responses import StreamingResponse
from sqlalchemy.orm import Session
from .. import database, models
from ..services import stories
from ..app import safe_content_disposition
from ..database import get_db
router = APIRouter()
@router.get("/stories", response_model=list[models.StoryResponse])
async def list_stories(db: Session = Depends(get_db)):
"""List all stories."""
return await stories.list_stories(db)
@router.post("/stories", response_model=models.StoryResponse)
async def create_story(
data: models.StoryCreate,
db: Session = Depends(get_db),
):
"""Create a new story."""
try:
return await stories.create_story(data, db)
except Exception as e:
raise HTTPException(status_code=400, detail=str(e))
@router.get("/stories/{story_id}", response_model=models.StoryDetailResponse)
async def get_story(
story_id: str,
db: Session = Depends(get_db),
):
"""Get a story with all its items."""
story = await stories.get_story(story_id, db)
if not story:
raise HTTPException(status_code=404, detail="Story not found")
return story
@router.put("/stories/{story_id}", response_model=models.StoryResponse)
async def update_story(
story_id: str,
data: models.StoryCreate,
db: Session = Depends(get_db),
):
"""Update a story."""
story = await stories.update_story(story_id, data, db)
if not story:
raise HTTPException(status_code=404, detail="Story not found")
return story
@router.delete("/stories/{story_id}")
async def delete_story(
story_id: str,
db: Session = Depends(get_db),
):
"""Delete a story."""
success = await stories.delete_story(story_id, db)
if not success:
raise HTTPException(status_code=404, detail="Story not found")
return {"message": "Story deleted successfully"}
@router.post("/stories/{story_id}/items", response_model=models.StoryItemDetail)
async def add_story_item(
story_id: str,
data: models.StoryItemCreate,
db: Session = Depends(get_db),
):
"""Add a generation to a story."""
item = await stories.add_item_to_story(story_id, data, db)
if not item:
raise HTTPException(status_code=404, detail="Story or generation not found")
return item
@router.delete("/stories/{story_id}/items/{item_id}")
async def remove_story_item(
story_id: str,
item_id: str,
db: Session = Depends(get_db),
):
"""Remove a story item from a story."""
success = await stories.remove_item_from_story(story_id, item_id, db)
if not success:
raise HTTPException(status_code=404, detail="Story item not found")
return {"message": "Item removed successfully"}
@router.put("/stories/{story_id}/items/times")
async def update_story_item_times(
story_id: str,
data: models.StoryItemBatchUpdate,
db: Session = Depends(get_db),
):
"""Update story item timecodes."""
success = await stories.update_story_item_times(story_id, data, db)
if not success:
raise HTTPException(status_code=400, detail="Invalid timecode update request")
return {"message": "Item timecodes updated successfully"}
@router.put("/stories/{story_id}/items/reorder", response_model=list[models.StoryItemDetail])
async def reorder_story_items(
story_id: str,
data: models.StoryItemReorder,
db: Session = Depends(get_db),
):
"""Reorder story items and recalculate timecodes."""
items = await stories.reorder_story_items(story_id, data.generation_ids, db)
if items is None:
raise HTTPException(
status_code=400, detail="Invalid reorder request - ensure all generation IDs belong to this story"
)
return items
@router.put("/stories/{story_id}/items/{item_id}/move", response_model=models.StoryItemDetail)
async def move_story_item(
story_id: str,
item_id: str,
data: models.StoryItemMove,
db: Session = Depends(get_db),
):
"""Move a story item (update position and/or track)."""
item = await stories.move_story_item(story_id, item_id, data, db)
if item is None:
raise HTTPException(status_code=404, detail="Story item not found")
return item
@router.put("/stories/{story_id}/items/{item_id}/trim", response_model=models.StoryItemDetail)
async def trim_story_item(
story_id: str,
item_id: str,
data: models.StoryItemTrim,
db: Session = Depends(get_db),
):
"""Trim a story item."""
item = await stories.trim_story_item(story_id, item_id, data, db)
if item is None:
raise HTTPException(status_code=404, detail="Story item not found or invalid trim values")
return item
@router.post("/stories/{story_id}/items/{item_id}/split", response_model=list[models.StoryItemDetail])
async def split_story_item(
story_id: str,
item_id: str,
data: models.StoryItemSplit,
db: Session = Depends(get_db),
):
"""Split a story item at a given time, creating two clips."""
items = await stories.split_story_item(story_id, item_id, data, db)
if items is None:
raise HTTPException(status_code=404, detail="Story item not found or invalid split point")
return items
@router.post("/stories/{story_id}/items/{item_id}/duplicate", response_model=models.StoryItemDetail)
async def duplicate_story_item(
story_id: str,
item_id: str,
db: Session = Depends(get_db),
):
"""Duplicate a story item."""
item = await stories.duplicate_story_item(story_id, item_id, db)
if item is None:
raise HTTPException(status_code=404, detail="Story item not found")
return item
@router.put("/stories/{story_id}/items/{item_id}/version", response_model=models.StoryItemDetail)
async def set_story_item_version(
story_id: str,
item_id: str,
data: models.StoryItemVersionUpdate,
db: Session = Depends(get_db),
):
"""Pin a story item to a specific generation version."""
item = await stories.set_story_item_version(story_id, item_id, data, db)
if item is None:
raise HTTPException(status_code=404, detail="Story item or version not found")
return item
@router.get("/stories/{story_id}/export-audio")
async def export_story_audio(
story_id: str,
db: Session = Depends(get_db),
):
"""Export story as single mixed audio file."""
try:
story = db.query(database.Story).filter_by(id=story_id).first()
if not story:
raise HTTPException(status_code=404, detail="Story not found")
audio_bytes = await stories.export_story_audio(story_id, db)
if not audio_bytes:
raise HTTPException(status_code=400, detail="Story has no audio items")
safe_name = "".join(c for c in story.name if c.isalnum() or c in (" ", "-", "_")).strip()
if not safe_name:
safe_name = "story"
filename = f"{safe_name}.wav"
return StreamingResponse(
io.BytesIO(audio_bytes),
media_type="audio/wav",
headers={"Content-Disposition": safe_content_disposition("attachment", filename)},
)
except HTTPException:
raise
except Exception as e:
raise HTTPException(status_code=500, detail=str(e))
+125
View File
@@ -0,0 +1,125 @@
"""Task and cache management endpoints."""
from datetime import datetime
from fastapi import APIRouter
from .. import models
from ..utils.cache import clear_voice_prompt_cache
from ..utils.progress import get_progress_manager
from ..utils.tasks import get_task_manager
from fastapi import HTTPException
router = APIRouter()
@router.post("/tasks/clear")
async def clear_all_tasks():
"""Clear all download tasks and progress state."""
task_manager = get_task_manager()
progress_manager = get_progress_manager()
task_manager.clear_all()
with progress_manager._lock:
progress_manager._progress.clear()
progress_manager._last_notify_time.clear()
progress_manager._last_notify_progress.clear()
return {"message": "All task state cleared"}
@router.post("/cache/clear")
async def clear_cache():
"""Clear all voice prompt caches (memory and disk)."""
try:
deleted_count = clear_voice_prompt_cache()
return {
"message": "Voice prompt cache cleared successfully",
"files_deleted": deleted_count,
}
except Exception as e:
raise HTTPException(status_code=500, detail=f"Failed to clear cache: {str(e)}")
@router.get("/tasks/active", response_model=models.ActiveTasksResponse)
async def get_active_tasks():
"""Return all currently active downloads and generations."""
task_manager = get_task_manager()
progress_manager = get_progress_manager()
active_downloads = []
task_manager_downloads = task_manager.get_active_downloads()
progress_active = progress_manager.get_all_active()
download_map = {task.model_name: task for task in task_manager_downloads}
progress_map = {p["model_name"]: p for p in progress_active}
all_model_names = set(download_map.keys()) | set(progress_map.keys())
for model_name in all_model_names:
task = download_map.get(model_name)
progress = progress_map.get(model_name)
if task:
error = task.error
if not error:
with progress_manager._lock:
pm_data = progress_manager._progress.get(model_name)
if pm_data:
error = pm_data.get("error")
prog = progress or {}
if not prog:
with progress_manager._lock:
pm_data = progress_manager._progress.get(model_name)
if pm_data:
prog = pm_data
active_downloads.append(
models.ActiveDownloadTask(
model_name=model_name,
status=task.status,
started_at=task.started_at,
error=error,
progress=prog.get("progress"),
current=prog.get("current"),
total=prog.get("total"),
filename=prog.get("filename"),
)
)
elif progress:
timestamp_str = progress.get("timestamp")
if timestamp_str:
try:
started_at = datetime.fromisoformat(timestamp_str.replace("Z", "+00:00"))
except (ValueError, AttributeError):
started_at = datetime.utcnow()
else:
started_at = datetime.utcnow()
active_downloads.append(
models.ActiveDownloadTask(
model_name=model_name,
status=progress.get("status", "downloading"),
started_at=started_at,
error=progress.get("error"),
progress=progress.get("progress"),
current=progress.get("current"),
total=progress.get("total"),
filename=progress.get("filename"),
)
)
active_generations = []
for gen_task in task_manager.get_active_generations():
active_generations.append(
models.ActiveGenerationTask(
task_id=gen_task.task_id,
profile_id=gen_task.profile_id,
text_preview=gen_task.text_preview,
started_at=gen_task.started_at,
)
)
return models.ActiveTasksResponse(
downloads=active_downloads,
generations=active_generations,
)
+74
View File
@@ -0,0 +1,74 @@
"""Transcription endpoints."""
import asyncio
import tempfile
from pathlib import Path
from fastapi import APIRouter, File, Form, HTTPException, UploadFile
from .. import models
from ..services import transcribe
from ..services.task_queue import create_background_task
from ..utils.tasks import get_task_manager
router = APIRouter()
UPLOAD_CHUNK_SIZE = 1024 * 1024 # 1MB
@router.post("/transcribe", response_model=models.TranscriptionResponse)
async def transcribe_audio(
file: UploadFile = File(...),
language: str | None = Form(None),
):
"""Transcribe audio file to text."""
with tempfile.NamedTemporaryFile(suffix=".wav", delete=False) as tmp:
while chunk := await file.read(UPLOAD_CHUNK_SIZE):
tmp.write(chunk)
tmp_path = tmp.name
try:
from ..utils.audio import load_audio
audio, sr = await asyncio.to_thread(load_audio, tmp_path)
duration = len(audio) / sr
whisper_model = transcribe.get_whisper_model()
model_size = whisper_model.model_size
if not whisper_model.is_loaded() and not whisper_model._is_model_cached(model_size):
progress_model_name = f"whisper-{model_size}"
task_manager = get_task_manager()
async def download_whisper_background():
try:
await whisper_model.load_model_async(model_size)
task_manager.complete_download(progress_model_name)
except Exception as e:
task_manager.error_download(progress_model_name, str(e))
task_manager.start_download(progress_model_name)
create_background_task(download_whisper_background())
raise HTTPException(
status_code=202,
detail={
"message": f"Whisper model {model_size} is being downloaded. Please wait and try again.",
"model_name": progress_model_name,
"downloading": True,
},
)
text = await whisper_model.transcribe(tmp_path, language)
return models.TranscriptionResponse(
text=text,
duration=duration,
)
except HTTPException:
raise
except Exception as e:
raise HTTPException(status_code=500, detail=str(e))
finally:
Path(tmp_path).unlink(missing_ok=True)
+167 -5
View File
@@ -6,6 +6,47 @@ absolute imports instead of relative imports.
""" """
import sys import sys
import os
# On Windows with --noconsole (PyInstaller), sys.stdout/stderr are None.
# They can also be broken file objects in some edge cases.
# Redirect to devnull to prevent crashes from print()/tqdm/logging.
def _is_writable(stream):
"""Check if a stream is usable for writing."""
if stream is None:
return False
try:
stream.write("")
return True
except Exception:
return False
if not _is_writable(sys.stdout):
sys.stdout = open(os.devnull, 'w')
if not _is_writable(sys.stderr):
sys.stderr = open(os.devnull, 'w')
# PyInstaller + multiprocessing: child processes re-execute the frozen binary
# with internal arguments. freeze_support() handles this and exits early.
import multiprocessing
multiprocessing.freeze_support()
# In frozen builds, piper_phonemize's espeak-ng C library falls back to
# /usr/share/espeak-ng-data/ which doesn't exist. Point it at the bundled
# data directory instead.
if getattr(sys, 'frozen', False):
_meipass = getattr(sys, '_MEIPASS', os.path.dirname(sys.executable))
_espeak_data = os.path.join(_meipass, 'piper_phonemize', 'espeak-ng-data')
if os.path.isdir(_espeak_data):
os.environ.setdefault('ESPEAK_DATA_PATH', _espeak_data)
# Fast path: handle --version before any heavy imports so the Rust
# version check doesn't block for 30+ seconds loading torch etc.
if "--version" in sys.argv:
from backend import __version__
print(f"voicebox-server {__version__}")
sys.exit(0)
import logging import logging
# Set up logging FIRST, before any imports that might fail # Set up logging FIRST, before any imports that might fail
@@ -43,6 +84,115 @@ except Exception as e:
logger.error(f"Failed to import required modules: {e}", exc_info=True) logger.error(f"Failed to import required modules: {e}", exc_info=True)
sys.exit(1) sys.exit(1)
_watchdog_disabled = False
def disable_watchdog():
"""Disable the parent watchdog so the server keeps running after parent exits."""
global _watchdog_disabled
_watchdog_disabled = True
# Ignore SIGHUP so the server survives when the parent Tauri process exits.
# On Unix, child processes receive SIGHUP when the parent's session leader
# exits, which would kill the server even though we want it to persist.
if sys.platform != "win32":
import signal
signal.signal(signal.SIGHUP, signal.SIG_IGN)
def _start_parent_watchdog(parent_pid, data_dir=None):
"""Monitor parent process and exit if it dies.
This is the clean shutdown mechanism: instead of the Tauri app trying to
forcefully kill the server (which spawns console windows on Windows),
the server monitors its parent and shuts itself down gracefully.
"""
import os
import signal
import threading
import time
# Set up a file logger so we can debug in production
watchdog_logger = logging.getLogger("watchdog")
if data_dir:
try:
log_dir = os.path.join(data_dir, "logs")
os.makedirs(log_dir, exist_ok=True)
fh = logging.FileHandler(os.path.join(log_dir, "watchdog.log"))
fh.setFormatter(logging.Formatter('%(asctime)s - %(message)s'))
watchdog_logger.addHandler(fh)
except Exception:
pass
watchdog_logger.setLevel(logging.INFO)
def _is_pid_alive(pid):
"""Check if a process with the given PID exists (cross-platform)."""
try:
if sys.platform == "win32":
import ctypes
kernel32 = ctypes.windll.kernel32
PROCESS_QUERY_LIMITED_INFORMATION = 0x1000
handle = kernel32.OpenProcess(PROCESS_QUERY_LIMITED_INFORMATION, False, pid)
if handle:
# Check if process has actually exited
STILL_ACTIVE = 259
exit_code = ctypes.c_ulong()
result = kernel32.GetExitCodeProcess(handle, ctypes.byref(exit_code))
kernel32.CloseHandle(handle)
if result and exit_code.value == STILL_ACTIVE:
return True
watchdog_logger.info(f"PID {pid}: exited with code {exit_code.value}")
return False
# OpenProcess failed — check if it's an access error (process exists
# but we can't open it) vs process not found
error = ctypes.GetLastError()
ACCESS_DENIED = 5
if error == ACCESS_DENIED:
return True # process exists, we just can't open it
watchdog_logger.info(f"PID {pid}: OpenProcess failed, error={error}")
return False
else:
os.kill(pid, 0)
return True
except (OSError, PermissionError):
return False
def _watch():
watchdog_logger.info(f"Parent watchdog started, monitoring PID {parent_pid}, server PID {os.getpid()}")
# Verify parent is alive before starting the loop
alive = _is_pid_alive(parent_pid)
watchdog_logger.info(f"Parent PID {parent_pid} initial check: alive={alive}")
if not alive:
watchdog_logger.warning(f"Parent PID {parent_pid} not found on first check — disabling watchdog")
return
while True:
if _watchdog_disabled:
watchdog_logger.info("Watchdog disabled (keep server running), stopping monitor")
return
if not _is_pid_alive(parent_pid):
# Parent is gone. Before shutting down, give the app a moment
# to send /watchdog/disable — there is a race where the Tauri
# RunEvent::Exit handler sends the disable request while we are
# mid-iteration (already past the _watchdog_disabled check above).
watchdog_logger.info(f"Parent process {parent_pid} gone, waiting for possible disable request...")
time.sleep(1)
if _watchdog_disabled:
watchdog_logger.info("Watchdog was disabled during grace period, keeping server alive")
return
watchdog_logger.info("Watchdog still enabled after grace period, shutting down server...")
if sys.platform == "win32":
# sys.exit triggers SystemExit, allowing uvicorn to run
# shutdown handlers. os.kill(SIGTERM) on Windows calls
# TerminateProcess which hard-kills without cleanup.
os._exit(0)
else:
os.kill(os.getpid(), signal.SIGTERM)
return
time.sleep(2)
t = threading.Thread(target=_watch, daemon=True)
t.start()
if __name__ == "__main__": if __name__ == "__main__":
try: try:
parser = argparse.ArgumentParser(description="voicebox backend server") parser = argparse.ArgumentParser(description="voicebox backend server")
@@ -64,17 +214,21 @@ if __name__ == "__main__":
default=None, default=None,
help="Data directory for database, profiles, and generated audio", help="Data directory for database, profiles, and generated audio",
) )
parser.add_argument(
"--parent-pid",
type=int,
default=None,
help="PID of parent process to monitor; server exits when parent dies",
)
parser.add_argument( parser.add_argument(
"--version", "--version",
action="store_true", action="store_true",
help="Print version and exit", help="Print version and exit (handled above, kept for argparse help)",
) )
args = parser.parse_args() args = parser.parse_args()
if args.version: if args.parent_pid is not None and args.parent_pid <= 0:
from backend import __version__ parser.error("--parent-pid must be a positive integer")
print(f"voicebox-server {__version__}")
sys.exit(0)
# Detect backend variant from binary name # Detect backend variant from binary name
# voicebox-server-cuda → sets VOICEBOX_BACKEND_VARIANT=cuda # voicebox-server-cuda → sets VOICEBOX_BACKEND_VARIANT=cuda
@@ -87,6 +241,14 @@ if __name__ == "__main__":
os.environ["VOICEBOX_BACKEND_VARIANT"] = "cpu" os.environ["VOICEBOX_BACKEND_VARIANT"] = "cpu"
logger.info("Backend variant: CPU") logger.info("Backend variant: CPU")
# Register parent watchdog to start after server is fully ready
if args.parent_pid is not None:
_parent_pid = args.parent_pid
_data_dir = args.data_dir
@app.on_event("startup")
async def _on_startup():
_start_parent_watchdog(_parent_pid, _data_dir)
logger.info(f"Parsed arguments: host={args.host}, port={args.port}, data_dir={args.data_dir}") logger.info(f"Parsed arguments: host={args.host}, port={args.port}, data_dir={args.data_dir}")
# Set data directory if provided # Set data directory if provided
+1
View File
@@ -0,0 +1 @@
# Services layer — generation orchestration and background task management.
@@ -7,14 +7,14 @@ from datetime import datetime
import uuid import uuid
from sqlalchemy.orm import Session from sqlalchemy.orm import Session
from .models import ( from ..models import (
AudioChannelCreate, AudioChannelCreate,
AudioChannelUpdate, AudioChannelUpdate,
AudioChannelResponse, AudioChannelResponse,
ChannelVoiceAssignment, ChannelVoiceAssignment,
ProfileChannelAssignment, ProfileChannelAssignment,
) )
from .database import ( from ..database import (
AudioChannel as DBAudioChannel, AudioChannel as DBAudioChannel,
ChannelDeviceMapping as DBChannelDeviceMapping, ChannelDeviceMapping as DBChannelDeviceMapping,
ProfileChannelMapping as DBProfileChannelMapping, ProfileChannelMapping as DBProfileChannelMapping,
@@ -14,9 +14,9 @@ import sys
from pathlib import Path from pathlib import Path
from typing import Optional from typing import Optional
from .config import get_data_dir from ..config import get_data_dir
from .utils.progress import get_progress_manager from ..utils.progress import get_progress_manager
from . import __version__ from .. import __version__
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
@@ -129,6 +129,17 @@ async def download_cuda_binary(version: Optional[str] = None):
except Exception as e: except Exception as e:
logger.warning(f"Could not fetch checksum file — skipping verification: {e}") logger.warning(f"Could not fetch checksum file — skipping verification: {e}")
# Get total size across all parts by issuing HEAD requests
total_size = 0
for part_name in parts:
try:
head_resp = await client.head(f"{base_url}/{part_name}")
content_length = int(head_resp.headers.get("content-length", 0))
total_size += content_length
except Exception:
pass
logger.info(f"Total download size: {total_size / 1024 / 1024:.1f} MB")
# Download and concatenate parts # Download and concatenate parts
total_downloaded = 0 total_downloaded = 0
with open(temp_path, "wb") as f: with open(temp_path, "wb") as f:
@@ -142,8 +153,8 @@ async def download_cuda_binary(version: Optional[str] = None):
f.write(chunk) f.write(chunk)
total_downloaded += len(chunk) total_downloaded += len(chunk)
progress.update_progress( progress.update_progress(
PROGRESS_KEY, current=total_downloaded, total=0, PROGRESS_KEY, current=total_downloaded, total=total_size,
filename=f"Part {i + 1}/{len(parts)}", filename=f"Downloading CUDA backend ({i + 1}/{len(parts)})",
status="downloading", status="downloading",
) )
@@ -188,6 +199,56 @@ async def download_cuda_binary(version: Optional[str] = None):
raise raise
def get_cuda_binary_version() -> Optional[str]:
"""Get the version of the installed CUDA binary, or None if not installed."""
import subprocess
cuda_path = get_cuda_binary_path()
if not cuda_path:
return None
try:
result = subprocess.run(
[str(cuda_path), "--version"],
capture_output=True, text=True, timeout=30,
)
# Output format: "voicebox-server 0.2.0"
for line in result.stdout.strip().splitlines():
if "voicebox-server" in line:
return line.split()[-1]
except Exception as e:
logger.warning(f"Could not get CUDA binary version: {e}")
return None
async def check_and_update_cuda_binary():
"""Check if the CUDA binary is outdated and auto-download if so.
Called on server startup. If a CUDA binary exists but its version
doesn't match the current app version, triggers a background download
of the updated CUDA binary. The download progress is visible to the
frontend via the existing SSE progress endpoint.
"""
cuda_path = get_cuda_binary_path()
if not cuda_path:
return # No CUDA binary installed, nothing to update
cuda_version = get_cuda_binary_version()
current_version = __version__
if cuda_version == current_version:
logger.info(f"CUDA binary is up to date (v{current_version})")
return
logger.info(
f"CUDA binary version mismatch: binary=v{cuda_version}, app=v{current_version}. "
f"Auto-downloading updated CUDA backend..."
)
try:
await download_cuda_binary()
except Exception as e:
logger.error(f"Auto-update of CUDA binary failed: {e}")
async def delete_cuda_binary() -> bool: async def delete_cuda_binary() -> bool:
"""Delete the downloaded CUDA binary. Returns True if deleted.""" """Delete the downloaded CUDA binary. Returns True if deleted."""
path = get_cuda_binary_path() path = get_cuda_binary_path()
+120
View File
@@ -0,0 +1,120 @@
"""
Effect presets CRUD operations.
"""
from __future__ import annotations
import json
import uuid
from typing import List, Optional
from sqlalchemy.orm import Session
from sqlalchemy.exc import IntegrityError
from ..database import EffectPreset as DBEffectPreset
from ..models import EffectPresetResponse, EffectPresetCreate, EffectPresetUpdate, EffectConfig
def _preset_response(p: DBEffectPreset) -> EffectPresetResponse:
"""Convert a DB preset row to a Pydantic response."""
effects_chain = [EffectConfig(**e) for e in json.loads(p.effects_chain)]
return EffectPresetResponse(
id=p.id,
name=p.name,
description=p.description,
effects_chain=effects_chain,
is_builtin=p.is_builtin or False,
created_at=p.created_at,
)
def list_presets(db: Session) -> List[EffectPresetResponse]:
"""List all effect presets (built-in + user-created)."""
presets = db.query(DBEffectPreset).order_by(DBEffectPreset.sort_order, DBEffectPreset.name).all()
return [_preset_response(p) for p in presets]
def get_preset(preset_id: str, db: Session) -> Optional[EffectPresetResponse]:
"""Get a preset by ID."""
p = db.query(DBEffectPreset).filter_by(id=preset_id).first()
if not p:
return None
return _preset_response(p)
def get_preset_by_name(name: str, db: Session) -> Optional[EffectPresetResponse]:
"""Get a preset by name."""
p = db.query(DBEffectPreset).filter_by(name=name).first()
if not p:
return None
return _preset_response(p)
def create_preset(data: EffectPresetCreate, db: Session) -> EffectPresetResponse:
"""Create a new user effect preset."""
from .utils.effects import validate_effects_chain
chain_dicts = [e.model_dump() for e in data.effects_chain]
error = validate_effects_chain(chain_dicts)
if error:
raise ValueError(error)
# Check for duplicate name before insert
existing = db.query(DBEffectPreset).filter_by(name=data.name).first()
if existing:
raise ValueError(f"A preset named '{data.name}' already exists")
preset = DBEffectPreset(
id=str(uuid.uuid4()),
name=data.name,
description=data.description,
effects_chain=json.dumps(chain_dicts),
is_builtin=False,
)
db.add(preset)
try:
db.commit()
except IntegrityError:
db.rollback()
raise ValueError(f"A preset named '{data.name}' already exists")
db.refresh(preset)
return _preset_response(preset)
def update_preset(preset_id: str, data: EffectPresetUpdate, db: Session) -> Optional[EffectPresetResponse]:
"""Update a user effect preset. Cannot modify built-in presets."""
preset = db.query(DBEffectPreset).filter_by(id=preset_id).first()
if not preset:
return None
if preset.is_builtin:
raise ValueError("Cannot modify built-in presets")
if data.name is not None:
preset.name = data.name
if data.description is not None:
preset.description = data.description
if data.effects_chain is not None:
from .utils.effects import validate_effects_chain
chain_dicts = [e.model_dump() for e in data.effects_chain]
error = validate_effects_chain(chain_dicts)
if error:
raise ValueError(error)
preset.effects_chain = json.dumps(chain_dicts)
db.commit()
db.refresh(preset)
return _preset_response(preset)
def delete_preset(preset_id: str, db: Session) -> bool:
"""Delete a user effect preset. Cannot delete built-in presets."""
preset = db.query(DBEffectPreset).filter_by(id=preset_id).first()
if not preset:
return False
if preset.is_builtin:
raise ValueError("Cannot delete built-in presets")
db.delete(preset)
db.commit()
return True
@@ -12,16 +12,11 @@ from pathlib import Path
from typing import Optional from typing import Optional
from sqlalchemy.orm import Session from sqlalchemy.orm import Session
from .models import VoiceProfileResponse from ..models import VoiceProfileResponse
from .database import VoiceProfile as DBVoiceProfile, ProfileSample as DBProfileSample, Generation as DBGeneration from ..database import VoiceProfile as DBVoiceProfile, ProfileSample as DBProfileSample, Generation as DBGeneration, GenerationVersion as DBGenerationVersion
from .profiles import create_profile, add_profile_sample from .profiles import create_profile, add_profile_sample
from .models import VoiceProfileCreate from ..models import VoiceProfileCreate
from . import config from .. import config
def _get_profiles_dir() -> Path:
"""Get profiles directory from config."""
return config.get_profiles_dir()
def _get_unique_profile_name(name: str, db: Session) -> str: def _get_unique_profile_name(name: str, db: Session) -> str:
@@ -99,7 +94,7 @@ def export_profile_to_zip(profile_id: str, db: Session) -> bytes:
# Create samples.json mapping # Create samples.json mapping
samples_data = {} samples_data = {}
profile_dir = _get_profiles_dir() / profile_id profile_dir = config.get_profiles_dir() / profile_id
for sample in samples: for sample in samples:
# Get filename from audio_path (should be {sample_id}.wav) # Get filename from audio_path (should be {sample_id}.wav)
@@ -181,7 +176,7 @@ async def import_profile_from_zip(file_bytes: bytes, db: Session) -> VoiceProfil
profile = await create_profile(profile_create, db) profile = await create_profile(profile_create, db)
# Extract and add samples # Extract and add samples
profile_dir = _get_profiles_dir() / profile.id profile_dir = config.get_profiles_dir() / profile.id
profile_dir.mkdir(parents=True, exist_ok=True) profile_dir.mkdir(parents=True, exist_ok=True)
# Handle avatar if present # Handle avatar if present
@@ -269,16 +264,33 @@ def export_generation_to_zip(generation_id: str, db: Session) -> bytes:
if not profile: if not profile:
raise ValueError(f"Profile {generation.profile_id} not found") raise ValueError(f"Profile {generation.profile_id} not found")
# Get audio file # Get all versions for this generation
audio_path = Path(generation.audio_path) versions = (
if not audio_path.exists(): db.query(DBGenerationVersion)
raise ValueError(f"Audio file not found: {audio_path}") .filter_by(generation_id=generation_id)
.order_by(DBGenerationVersion.created_at)
.all()
)
# Create ZIP in memory # Create ZIP in memory
zip_buffer = io.BytesIO() zip_buffer = io.BytesIO()
with zipfile.ZipFile(zip_buffer, 'w', zipfile.ZIP_DEFLATED) as zip_file: with zipfile.ZipFile(zip_buffer, 'w', zipfile.ZIP_DEFLATED) as zip_file:
# Create manifest.json # Build version manifest entries
version_entries = []
for v in versions:
v_path = Path(v.audio_path)
effects_chain = None
if v.effects_chain:
effects_chain = json.loads(v.effects_chain)
version_entries.append({
"id": v.id,
"label": v.label,
"is_default": v.is_default,
"effects_chain": effects_chain,
"filename": v_path.name,
})
manifest = { manifest = {
"version": "1.0", "version": "1.0",
"generation": { "generation": {
@@ -295,13 +307,22 @@ def export_generation_to_zip(generation_id: str, db: Session) -> bytes:
"name": profile.name, "name": profile.name,
"description": profile.description, "description": profile.description,
"language": profile.language, "language": profile.language,
} },
"versions": version_entries,
} }
zip_file.writestr("manifest.json", json.dumps(manifest, indent=2)) zip_file.writestr("manifest.json", json.dumps(manifest, indent=2))
# Add audio file # Add all version audio files
filename = audio_path.name for v in versions:
zip_file.write(audio_path, f"audio/{filename}") v_path = Path(v.audio_path)
if v_path.exists():
zip_file.write(v_path, f"audio/{v_path.name}")
# Fallback: if no versions exist, include the generation's main audio
if not versions:
audio_path = Path(generation.audio_path)
if audio_path.exists():
zip_file.write(audio_path, f"audio/{audio_path.name}")
zip_buffer.seek(0) zip_buffer.seek(0)
return zip_buffer.read() return zip_buffer.read()
@@ -325,7 +346,7 @@ async def import_generation_from_zip(file_bytes: bytes, db: Session) -> dict:
import tempfile import tempfile
import shutil import shutil
from datetime import datetime from datetime import datetime
from . import config from .. import config
zip_buffer = io.BytesIO(file_bytes) zip_buffer = io.BytesIO(file_bytes)
+253
View File
@@ -0,0 +1,253 @@
"""
Unified TTS generation orchestration.
Replaces the three near-identical closures (_run_generation, _run_retry,
_run_regenerate) that lived in main.py with a single ``run_generation()``
function parameterized by *mode*.
Mode differences:
- "generate" : full pipeline -- save clean version, optionally apply
effects and create a processed version.
- "retry" : re-runs a failed generation with the same seed.
No effects, no version creation.
- "regenerate" : re-runs with seed=None for variation. Creates a new
version with an auto-incremented "take-N" label.
"""
from __future__ import annotations
import traceback
from typing import Literal, Optional
from .. import config
from . import history, profiles
from ..database import get_db
from ..utils.tasks import get_task_manager
async def run_generation(
*,
generation_id: str,
profile_id: str,
text: str,
language: str,
engine: str,
model_size: str,
seed: Optional[int],
normalize: bool = False,
effects_chain: Optional[list] = None,
instruct: Optional[str] = None,
mode: Literal["generate", "retry", "regenerate"],
max_chunk_chars: Optional[int] = None,
crossfade_ms: Optional[int] = None,
version_id: Optional[str] = None,
) -> None:
"""Execute TTS inference and persist the result.
This is the single entry point for all background generation work.
It is designed to be enqueued via ``services.task_queue.enqueue_generation``.
"""
from ..backends import load_engine_model, get_tts_backend_for_engine, engine_needs_trim
from ..utils.chunked_tts import generate_chunked
from ..utils.audio import normalize_audio, save_audio, trim_tts_output
task_manager = get_task_manager()
bg_db = next(get_db())
try:
tts_model = get_tts_backend_for_engine(engine)
if not tts_model.is_loaded():
await history.update_generation_status(generation_id, "loading_model", bg_db)
await load_engine_model(engine, model_size)
voice_prompt = await profiles.create_voice_prompt_for_profile(
profile_id,
bg_db,
use_cache=True,
engine=engine,
)
await history.update_generation_status(generation_id, "generating", bg_db)
trim_fn = trim_tts_output if engine_needs_trim(engine) else None
gen_kwargs: dict = dict(
language=language,
seed=seed if mode != "regenerate" else None,
instruct=instruct,
trim_fn=trim_fn,
)
if max_chunk_chars is not None:
gen_kwargs["max_chunk_chars"] = max_chunk_chars
if crossfade_ms is not None:
gen_kwargs["crossfade_ms"] = crossfade_ms
audio, sample_rate = await generate_chunked(tts_model, text, voice_prompt, **gen_kwargs)
# --- Normalize (generate and regenerate always; retry skips) -----
if normalize or mode == "regenerate":
audio = normalize_audio(audio)
duration = len(audio) / sample_rate
# --- Persist audio and update status -----------------------------
if mode == "generate":
final_path = _save_generate(
generation_id=generation_id,
audio=audio,
sample_rate=sample_rate,
effects_chain=effects_chain,
save_audio=save_audio,
db=bg_db,
)
elif mode == "retry":
final_path = _save_retry(
generation_id=generation_id,
audio=audio,
sample_rate=sample_rate,
save_audio=save_audio,
)
elif mode == "regenerate":
final_path = _save_regenerate(
generation_id=generation_id,
version_id=version_id,
audio=audio,
sample_rate=sample_rate,
save_audio=save_audio,
db=bg_db,
)
await history.update_generation_status(
generation_id=generation_id,
status="completed",
db=bg_db,
audio_path=final_path,
duration=duration,
)
except Exception as e:
traceback.print_exc()
await history.update_generation_status(
generation_id=generation_id,
status="failed",
db=bg_db,
error=str(e),
)
finally:
task_manager.complete_generation(generation_id)
bg_db.close()
def _save_generate(
*,
generation_id: str,
audio,
sample_rate: int,
effects_chain: Optional[list],
save_audio,
db,
) -> str:
"""Save clean version and optionally an effects-processed version.
Returns the final audio path (processed if effects were applied,
otherwise clean).
"""
from . import versions as versions_mod
clean_audio_path = config.get_generations_dir() / f"{generation_id}.wav"
save_audio(audio, str(clean_audio_path), sample_rate)
has_effects = effects_chain and any(e.get("enabled", True) for e in effects_chain)
versions_mod.create_version(
generation_id=generation_id,
label="original",
audio_path=str(clean_audio_path),
db=db,
effects_chain=None,
is_default=not has_effects,
)
final_audio_path = str(clean_audio_path)
if has_effects:
from ..utils.effects import apply_effects, validate_effects_chain
error_msg = validate_effects_chain(effects_chain)
if error_msg:
import logging
logging.getLogger(__name__).warning("invalid effects chain, skipping: %s", error_msg)
versions_mod.set_default_version(
versions_mod.list_versions(generation_id, db)[0].id, db
)
else:
processed_audio = apply_effects(audio, sample_rate, effects_chain)
processed_path = config.get_generations_dir() / f"{generation_id}_processed.wav"
save_audio(processed_audio, str(processed_path), sample_rate)
final_audio_path = str(processed_path)
versions_mod.create_version(
generation_id=generation_id,
label="version-2",
audio_path=str(processed_path),
db=db,
effects_chain=effects_chain,
is_default=True,
)
return final_audio_path
def _save_retry(
*,
generation_id: str,
audio,
sample_rate: int,
save_audio,
) -> str:
"""Save retry output -- single file, no versions.
Returns the audio path.
"""
audio_path = config.get_generations_dir() / f"{generation_id}.wav"
save_audio(audio, str(audio_path), sample_rate)
return str(audio_path)
def _save_regenerate(
*,
generation_id: str,
version_id: Optional[str],
audio,
sample_rate: int,
save_audio,
db,
) -> str:
"""Save regeneration output as a new version with auto-label.
Returns the audio path.
"""
from . import versions as versions_mod
import uuid as _uuid
suffix = _uuid.uuid4().hex[:8]
audio_path = config.get_generations_dir() / f"{generation_id}_{suffix}.wav"
save_audio(audio, str(audio_path), sample_rate)
# Count via DB query rather than list length to avoid TOCTOU race
from ..database import GenerationVersion as DBGenerationVersion
count = db.query(DBGenerationVersion).filter_by(generation_id=generation_id).count()
label = f"take-{count + 1}"
versions_mod.create_version(
generation_id=generation_id,
label=label,
audio_path=str(audio_path),
db=db,
effects_chain=None,
is_default=True,
)
return str(audio_path)
@@ -10,14 +10,46 @@ from pathlib import Path
from sqlalchemy.orm import Session from sqlalchemy.orm import Session
from sqlalchemy import or_ from sqlalchemy import or_
from .models import GenerationRequest, GenerationResponse, HistoryQuery, HistoryResponse, HistoryListResponse from ..models import GenerationRequest, GenerationResponse, HistoryQuery, HistoryResponse, HistoryListResponse, GenerationVersionResponse, EffectConfig
from .database import Generation as DBGeneration, VoiceProfile as DBVoiceProfile from ..database import Generation as DBGeneration, GenerationVersion as DBGenerationVersion, VoiceProfile as DBVoiceProfile
from . import config from .. import config
def _get_generations_dir() -> Path: def _get_versions_for_generation(generation_id: str, db: Session) -> tuple:
"""Get generations directory from config.""" """Get versions list and active version ID for a generation."""
return config.get_generations_dir() import json
versions_rows = (
db.query(DBGenerationVersion)
.filter_by(generation_id=generation_id)
.order_by(DBGenerationVersion.created_at)
.all()
)
if not versions_rows:
return None, None
versions = []
active_version_id = None
for v in versions_rows:
effects_chain = None
if v.effects_chain:
try:
raw = json.loads(v.effects_chain)
effects_chain = [EffectConfig(**e) for e in raw]
except Exception:
pass
versions.append(GenerationVersionResponse(
id=v.id,
generation_id=v.generation_id,
label=v.label,
audio_path=v.audio_path,
effects_chain=effects_chain,
is_default=v.is_default,
created_at=v.created_at,
))
if v.is_default:
active_version_id = v.id
return versions, active_version_id
async def create_generation( async def create_generation(
@@ -170,6 +202,7 @@ async def list_generations(
# Convert to HistoryResponse with profile_name # Convert to HistoryResponse with profile_name
items = [] items = []
for generation, profile_name in results: for generation, profile_name in results:
versions, active_version_id = _get_versions_for_generation(generation.id, db)
items.append(HistoryResponse( items.append(HistoryResponse(
id=generation.id, id=generation.id,
profile_id=generation.profile_id, profile_id=generation.profile_id,
@@ -184,7 +217,10 @@ async def list_generations(
model_size=generation.model_size, model_size=generation.model_size,
status=generation.status or "completed", status=generation.status or "completed",
error=generation.error, error=generation.error,
is_favorited=bool(generation.is_favorited),
created_at=generation.created_at, created_at=generation.created_at,
versions=versions,
active_version_id=active_version_id,
)) ))
return HistoryListResponse( return HistoryListResponse(
@@ -210,12 +246,17 @@ async def delete_generation(
generation = db.query(DBGeneration).filter_by(id=generation_id).first() generation = db.query(DBGeneration).filter_by(id=generation_id).first()
if not generation: if not generation:
return False return False
# Delete audio file # Delete all version files and records
audio_path = Path(generation.audio_path) from . import versions as versions_mod
if audio_path.exists(): versions_mod.delete_versions_for_generation(generation_id, db)
audio_path.unlink()
# Delete main audio file (if not already removed by version cleanup)
if generation.audio_path:
audio_path = Path(generation.audio_path)
if audio_path.exists():
audio_path.unlink()
# Delete from database # Delete from database
db.delete(generation) db.delete(generation)
db.commit() db.commit()
@@ -8,28 +8,55 @@ import uuid
import shutil import shutil
from pathlib import Path from pathlib import Path
from sqlalchemy.orm import Session from sqlalchemy.orm import Session
from sqlalchemy import select from sqlalchemy import func, select
from .models import ( from ..models import (
VoiceProfileCreate, VoiceProfileCreate,
VoiceProfileResponse, VoiceProfileResponse,
ProfileSampleCreate, ProfileSampleCreate,
ProfileSampleResponse, ProfileSampleResponse,
) )
from .database import ( from ..database import (
VoiceProfile as DBVoiceProfile, VoiceProfile as DBVoiceProfile,
ProfileSample as DBProfileSample, ProfileSample as DBProfileSample,
Generation as DBGeneration,
) )
from .utils.audio import validate_reference_audio, load_audio, save_audio from ..models import EffectConfig
from .utils.images import validate_image, process_avatar from ..utils.audio import validate_reference_audio, load_audio, save_audio
from .utils.cache import _get_cache_dir, clear_profile_cache from ..utils.images import validate_image, process_avatar
from ..utils.cache import _get_cache_dir, clear_profile_cache
from .tts import get_tts_model from .tts import get_tts_model
from . import config from .. import config
import json as _json
def _get_profiles_dir() -> Path: def _profile_to_response(
"""Get profiles directory from config.""" profile: DBVoiceProfile,
return config.get_profiles_dir() generation_count: int = 0,
sample_count: int = 0,
) -> VoiceProfileResponse:
"""Convert a DB profile to a VoiceProfileResponse, deserializing effects_chain."""
effects_chain = None
if profile.effects_chain:
try:
raw = _json.loads(profile.effects_chain)
effects_chain = [EffectConfig(**e) for e in raw]
except Exception as e:
import logging
logging.warning(f"Failed to parse effects_chain for profile {profile.id}: {e}")
return VoiceProfileResponse(
id=profile.id,
name=profile.name,
description=profile.description,
language=profile.language,
avatar_path=profile.avatar_path,
effects_chain=effects_chain,
generation_count=generation_count,
sample_count=sample_count,
created_at=profile.created_at,
updated_at=profile.updated_at,
)
async def create_profile( async def create_profile(
@@ -49,12 +76,10 @@ async def create_profile(
Raises: Raises:
ValueError: If a profile with the same name already exists ValueError: If a profile with the same name already exists
""" """
# Check if profile name already exists
existing_profile = db.query(DBVoiceProfile).filter_by(name=data.name).first() existing_profile = db.query(DBVoiceProfile).filter_by(name=data.name).first()
if existing_profile: if existing_profile:
raise ValueError(f"A profile with the name '{data.name}' already exists. Please choose a different name.") raise ValueError(f"A profile with the name '{data.name}' already exists. Please choose a different name.")
# Create profile in database
db_profile = DBVoiceProfile( db_profile = DBVoiceProfile(
id=str(uuid.uuid4()), id=str(uuid.uuid4()),
name=data.name, name=data.name,
@@ -68,11 +93,10 @@ async def create_profile(
db.commit() db.commit()
db.refresh(db_profile) db.refresh(db_profile)
# Create profile directory profile_dir = config.get_profiles_dir() / db_profile.id
profile_dir = _get_profiles_dir() / db_profile.id
profile_dir.mkdir(parents=True, exist_ok=True) profile_dir.mkdir(parents=True, exist_ok=True)
return VoiceProfileResponse.model_validate(db_profile) return _profile_to_response(db_profile)
async def add_profile_sample( async def add_profile_sample(
@@ -83,56 +107,50 @@ async def add_profile_sample(
) -> ProfileSampleResponse: ) -> ProfileSampleResponse:
""" """
Add a sample to a voice profile. Add a sample to a voice profile.
Args: Args:
profile_id: Profile ID profile_id: Profile ID
audio_path: Path to temporary audio file audio_path: Path to temporary audio file
reference_text: Transcript of audio reference_text: Transcript of audio
db: Database session db: Database session
Returns: Returns:
Created sample Created sample
""" """
# Validate profile exists
profile = db.query(DBVoiceProfile).filter_by(id=profile_id).first() profile = db.query(DBVoiceProfile).filter_by(id=profile_id).first()
if not profile: if not profile:
raise ValueError(f"Profile {profile_id} not found") raise ValueError(f"Profile {profile_id} not found")
# Validate audio
is_valid, error_msg = validate_reference_audio(audio_path) is_valid, error_msg = validate_reference_audio(audio_path)
if not is_valid: if not is_valid:
raise ValueError(f"Invalid reference audio: {error_msg}") raise ValueError(f"Invalid reference audio: {error_msg}")
# Create sample ID and directory
sample_id = str(uuid.uuid4()) sample_id = str(uuid.uuid4())
profile_dir = _get_profiles_dir() / profile_id profile_dir = config.get_profiles_dir() / profile_id
profile_dir.mkdir(parents=True, exist_ok=True) profile_dir.mkdir(parents=True, exist_ok=True)
# Copy audio file to profile directory
dest_path = profile_dir / f"{sample_id}.wav" dest_path = profile_dir / f"{sample_id}.wav"
audio, sr = load_audio(audio_path) audio, sr = load_audio(audio_path)
save_audio(audio, str(dest_path), sr) save_audio(audio, str(dest_path), sr)
# Create database entry
db_sample = DBProfileSample( db_sample = DBProfileSample(
id=sample_id, id=sample_id,
profile_id=profile_id, profile_id=profile_id,
audio_path=str(dest_path), audio_path=str(dest_path),
reference_text=reference_text, reference_text=reference_text,
) )
db.add(db_sample) db.add(db_sample)
# Update profile timestamp
profile.updated_at = datetime.utcnow() profile.updated_at = datetime.utcnow()
db.commit() db.commit()
db.refresh(db_sample) db.refresh(db_sample)
# Invalidate combined audio cache for this profile # Invalidate combined audio cache for this profile
# Since a new sample was added, any cached combined audio is now stale # Since a new sample was added, any cached combined audio is now stale
clear_profile_cache(profile_id) clear_profile_cache(profile_id)
return ProfileSampleResponse.model_validate(db_sample) return ProfileSampleResponse.model_validate(db_sample)
@@ -142,19 +160,19 @@ async def get_profile(
) -> Optional[VoiceProfileResponse]: ) -> Optional[VoiceProfileResponse]:
""" """
Get a voice profile by ID. Get a voice profile by ID.
Args: Args:
profile_id: Profile ID profile_id: Profile ID
db: Database session db: Database session
Returns: Returns:
Profile or None if not found Profile or None if not found
""" """
profile = db.query(DBVoiceProfile).filter_by(id=profile_id).first() profile = db.query(DBVoiceProfile).filter_by(id=profile_id).first()
if not profile: if not profile:
return None return None
return VoiceProfileResponse.model_validate(profile) return _profile_to_response(profile)
async def get_profile_samples( async def get_profile_samples(
@@ -163,11 +181,11 @@ async def get_profile_samples(
) -> List[ProfileSampleResponse]: ) -> List[ProfileSampleResponse]:
""" """
Get all samples for a profile. Get all samples for a profile.
Args: Args:
profile_id: Profile ID profile_id: Profile ID
db: Database session db: Database session
Returns: Returns:
List of samples List of samples
""" """
@@ -177,19 +195,39 @@ async def get_profile_samples(
async def list_profiles(db: Session) -> List[VoiceProfileResponse]: async def list_profiles(db: Session) -> List[VoiceProfileResponse]:
""" """
List all voice profiles. List all voice profiles with generation and sample counts.
Args: Args:
db: Database session db: Database session
Returns: Returns:
List of profiles List of profiles
""" """
profiles = db.query(DBVoiceProfile).order_by( profiles = db.query(DBVoiceProfile).order_by(DBVoiceProfile.created_at.desc()).all()
DBVoiceProfile.created_at.desc()
).all() if not profiles:
return []
return [VoiceProfileResponse.model_validate(p) for p in profiles]
# Batch-fetch generation counts
gen_counts_rows = (
db.query(DBGeneration.profile_id, func.count(DBGeneration.id)).group_by(DBGeneration.profile_id).all()
)
gen_counts = {row[0]: row[1] for row in gen_counts_rows}
# Batch-fetch sample counts
sample_counts_rows = (
db.query(DBProfileSample.profile_id, func.count(DBProfileSample.id)).group_by(DBProfileSample.profile_id).all()
)
sample_counts = {row[0]: row[1] for row in sample_counts_rows}
return [
_profile_to_response(
p,
generation_count=gen_counts.get(p.id, 0),
sample_count=sample_counts.get(p.id, 0),
)
for p in profiles
]
async def update_profile( async def update_profile(
@@ -215,13 +253,11 @@ async def update_profile(
if not profile: if not profile:
return None return None
# Check if the new name conflicts with another profile
if profile.name != data.name: if profile.name != data.name:
existing_profile = db.query(DBVoiceProfile).filter_by(name=data.name).first() existing_profile = db.query(DBVoiceProfile).filter_by(name=data.name).first()
if existing_profile: if existing_profile:
raise ValueError(f"A profile with the name '{data.name}' already exists. Please choose a different name.") raise ValueError(f"A profile with the name '{data.name}' already exists. Please choose a different name.")
# Update fields
profile.name = data.name profile.name = data.name
profile.description = data.description profile.description = data.description
profile.language = data.language profile.language = data.language
@@ -230,7 +266,7 @@ async def update_profile(
db.commit() db.commit()
db.refresh(profile) db.refresh(profile)
return VoiceProfileResponse.model_validate(profile) return _profile_to_response(profile)
async def delete_profile( async def delete_profile(
@@ -239,33 +275,30 @@ async def delete_profile(
) -> bool: ) -> bool:
""" """
Delete a voice profile and all associated data. Delete a voice profile and all associated data.
Args: Args:
profile_id: Profile ID profile_id: Profile ID
db: Database session db: Database session
Returns: Returns:
True if deleted, False if not found True if deleted, False if not found
""" """
profile = db.query(DBVoiceProfile).filter_by(id=profile_id).first() profile = db.query(DBVoiceProfile).filter_by(id=profile_id).first()
if not profile: if not profile:
return False return False
# Delete samples from database
db.query(DBProfileSample).filter_by(profile_id=profile_id).delete() db.query(DBProfileSample).filter_by(profile_id=profile_id).delete()
# Delete profile from database
db.delete(profile) db.delete(profile)
db.commit() db.commit()
# Delete profile directory profile_dir = config.get_profiles_dir() / profile_id
profile_dir = _get_profiles_dir() / profile_id
if profile_dir.exists(): if profile_dir.exists():
shutil.rmtree(profile_dir) shutil.rmtree(profile_dir)
# Clean up combined audio cache files for this profile # Clean up combined audio cache files for this profile
clear_profile_cache(profile_id) clear_profile_cache(profile_id)
return True return True
@@ -275,34 +308,32 @@ async def delete_profile_sample(
) -> bool: ) -> bool:
""" """
Delete a profile sample. Delete a profile sample.
Args: Args:
sample_id: Sample ID sample_id: Sample ID
db: Database session db: Database session
Returns: Returns:
True if deleted, False if not found True if deleted, False if not found
""" """
sample = db.query(DBProfileSample).filter_by(id=sample_id).first() sample = db.query(DBProfileSample).filter_by(id=sample_id).first()
if not sample: if not sample:
return False return False
# Store profile_id before deleting # Store profile_id before deleting
profile_id = sample.profile_id profile_id = sample.profile_id
# Delete audio file
audio_path = Path(sample.audio_path) audio_path = Path(sample.audio_path)
if audio_path.exists(): if audio_path.exists():
audio_path.unlink() audio_path.unlink()
# Delete from database
db.delete(sample) db.delete(sample)
db.commit() db.commit()
# Invalidate combined audio cache for this profile # Invalidate combined audio cache for this profile
# Since the sample set changed, any cached combined audio is now stale # Since the sample set changed, any cached combined audio is now stale
clear_profile_cache(profile_id) clear_profile_cache(profile_id)
return True return True
@@ -313,30 +344,30 @@ async def update_profile_sample(
) -> Optional[ProfileSampleResponse]: ) -> Optional[ProfileSampleResponse]:
""" """
Update a profile sample's reference text. Update a profile sample's reference text.
Args: Args:
sample_id: Sample ID sample_id: Sample ID
reference_text: Updated reference text reference_text: Updated reference text
db: Database session db: Database session
Returns: Returns:
Updated sample or None if not found Updated sample or None if not found
""" """
sample = db.query(DBProfileSample).filter_by(id=sample_id).first() sample = db.query(DBProfileSample).filter_by(id=sample_id).first()
if not sample: if not sample:
return None return None
# Store profile_id before updating # Store profile_id before updating
profile_id = sample.profile_id profile_id = sample.profile_id
sample.reference_text = reference_text sample.reference_text = reference_text
db.commit() db.commit()
db.refresh(sample) db.refresh(sample)
# Invalidate combined audio cache for this profile # Invalidate combined audio cache for this profile
# Since the reference text changed, cache keys and combined text are now stale # Since the reference text changed, cache keys and combined text are now stale
clear_profile_cache(profile_id) clear_profile_cache(profile_id)
return ProfileSampleResponse.model_validate(sample) return ProfileSampleResponse.model_validate(sample)
@@ -358,9 +389,8 @@ async def create_voice_prompt_for_profile(
Returns: Returns:
Voice prompt dictionary Voice prompt dictionary
""" """
from .backends import get_tts_backend_for_engine from ..backends import get_tts_backend_for_engine
# Get all samples for profile
samples = db.query(DBProfileSample).filter_by(profile_id=profile_id).all() samples = db.query(DBProfileSample).filter_by(profile_id=profile_id).all()
if not samples: if not samples:
@@ -369,7 +399,6 @@ async def create_voice_prompt_for_profile(
tts_model = get_tts_backend_for_engine(engine) tts_model = get_tts_backend_for_engine(engine)
if len(samples) == 1: if len(samples) == 1:
# Single sample - use directly
sample = samples[0] sample = samples[0]
voice_prompt, _ = await tts_model.create_voice_prompt( voice_prompt, _ = await tts_model.create_voice_prompt(
sample.audio_path, sample.audio_path,
@@ -378,11 +407,9 @@ async def create_voice_prompt_for_profile(
) )
return voice_prompt return voice_prompt
else: else:
# Multiple samples - combine them
audio_paths = [s.audio_path for s in samples] audio_paths = [s.audio_path for s in samples]
reference_texts = [s.reference_text for s in samples] reference_texts = [s.reference_text for s in samples]
# Combine audio
combined_audio, combined_text = await tts_model.combine_voice_prompts( combined_audio, combined_text = await tts_model.combine_voice_prompts(
audio_paths, audio_paths,
reference_texts, reference_texts,
@@ -391,18 +418,16 @@ async def create_voice_prompt_for_profile(
# Save combined audio to cache directory (persistent) # Save combined audio to cache directory (persistent)
# Create a hash of sample IDs to identify this specific combination # Create a hash of sample IDs to identify this specific combination
import hashlib import hashlib
sample_ids_str = "-".join(sorted([s.id for s in samples])) sample_ids_str = "-".join(sorted([s.id for s in samples]))
combination_hash = hashlib.md5(sample_ids_str.encode()).hexdigest()[:12] combination_hash = hashlib.md5(sample_ids_str.encode()).hexdigest()[:12]
# Store in cache directory
cache_dir = _get_cache_dir() cache_dir = _get_cache_dir()
cache_dir.mkdir(parents=True, exist_ok=True) cache_dir.mkdir(parents=True, exist_ok=True)
combined_path = cache_dir / f"combined_{profile_id}_{combination_hash}.wav" combined_path = cache_dir / f"combined_{profile_id}_{combination_hash}.wav"
# Save combined audio
save_audio(combined_audio, str(combined_path), 24000) save_audio(combined_audio, str(combined_path), 24000)
# Create prompt from combined audio
voice_prompt, _ = await tts_model.create_voice_prompt( voice_prompt, _ = await tts_model.create_voice_prompt(
str(combined_path), str(combined_path),
combined_text, combined_text,
@@ -427,17 +452,14 @@ async def upload_avatar(
Returns: Returns:
Updated profile Updated profile
""" """
# Validate profile exists
profile = db.query(DBVoiceProfile).filter_by(id=profile_id).first() profile = db.query(DBVoiceProfile).filter_by(id=profile_id).first()
if not profile: if not profile:
raise ValueError(f"Profile {profile_id} not found") raise ValueError(f"Profile {profile_id} not found")
# Validate image
is_valid, error_msg = validate_image(image_path) is_valid, error_msg = validate_image(image_path)
if not is_valid: if not is_valid:
raise ValueError(error_msg) raise ValueError(error_msg)
# Delete existing avatar if present
if profile.avatar_path: if profile.avatar_path:
old_avatar = Path(profile.avatar_path) old_avatar = Path(profile.avatar_path)
if old_avatar.exists(): if old_avatar.exists():
@@ -445,34 +467,29 @@ async def upload_avatar(
# Determine file extension from uploaded file # Determine file extension from uploaded file
from PIL import Image from PIL import Image
with Image.open(image_path) as img: with Image.open(image_path) as img:
# Normalize JPEG variants (MPO is multi-picture format from some cameras) # Normalize JPEG variants (MPO is multi-picture format from some cameras)
img_format = img.format img_format = img.format
if img_format in ('MPO', 'JPG'): if img_format in ("MPO", "JPG"):
img_format = 'JPEG' img_format = "JPEG"
ext_map = {
'PNG': '.png',
'JPEG': '.jpg',
'WEBP': '.webp'
}
ext = ext_map.get(img_format, '.png')
# Save processed image to profile directory ext_map = {"PNG": ".png", "JPEG": ".jpg", "WEBP": ".webp"}
profile_dir = _get_profiles_dir() / profile_id ext = ext_map.get(img_format, ".png")
profile_dir = config.get_profiles_dir() / profile_id
profile_dir.mkdir(parents=True, exist_ok=True) profile_dir.mkdir(parents=True, exist_ok=True)
output_path = profile_dir / f"avatar{ext}" output_path = profile_dir / f"avatar{ext}"
process_avatar(image_path, str(output_path)) process_avatar(image_path, str(output_path))
# Update database
profile.avatar_path = str(output_path) profile.avatar_path = str(output_path)
profile.updated_at = datetime.utcnow() profile.updated_at = datetime.utcnow()
db.commit() db.commit()
db.refresh(profile) db.refresh(profile)
return VoiceProfileResponse.model_validate(profile) return _profile_to_response(profile)
async def delete_avatar( async def delete_avatar(
@@ -493,12 +510,10 @@ async def delete_avatar(
if not profile or not profile.avatar_path: if not profile or not profile.avatar_path:
return False return False
# Delete avatar file
avatar_path = Path(profile.avatar_path) avatar_path = Path(profile.avatar_path)
if avatar_path.exists(): if avatar_path.exists():
avatar_path.unlink() avatar_path.unlink()
# Update database
profile.avatar_path = None profile.avatar_path = None
profile.updated_at = datetime.utcnow() profile.updated_at = datetime.utcnow()
+256 -306
View File
@@ -10,7 +10,7 @@ from pathlib import Path
from sqlalchemy.orm import Session from sqlalchemy.orm import Session
from sqlalchemy import func from sqlalchemy import func
from .models import ( from ..models import (
StoryCreate, StoryCreate,
StoryResponse, StoryResponse,
StoryDetailResponse, StoryDetailResponse,
@@ -20,12 +20,60 @@ from .models import (
StoryItemMove, StoryItemMove,
StoryItemTrim, StoryItemTrim,
StoryItemSplit, StoryItemSplit,
StoryItemVersionUpdate,
) )
from .database import Story as DBStory, StoryItem as DBStoryItem, Generation as DBGeneration, VoiceProfile as DBVoiceProfile from ..database import (
from .utils.audio import load_audio, save_audio Story as DBStory,
StoryItem as DBStoryItem,
Generation as DBGeneration,
VoiceProfile as DBVoiceProfile,
)
from .history import _get_versions_for_generation
from ..utils.audio import load_audio, save_audio
import numpy as np import numpy as np
def _build_item_detail(
item: DBStoryItem,
generation: DBGeneration,
profile_name: str,
db: Session,
) -> StoryItemDetail:
"""Build a StoryItemDetail with version info from a story item and its generation."""
versions, active_version_id = _get_versions_for_generation(generation.id, db)
# Resolve the audio path: if version_id is set, use that version's audio
audio_path = generation.audio_path
if item.version_id and versions:
for v in versions:
if v.id == item.version_id:
audio_path = v.audio_path
break
return StoryItemDetail(
id=item.id,
story_id=item.story_id,
generation_id=item.generation_id,
version_id=getattr(item, "version_id", None),
start_time_ms=item.start_time_ms,
track=item.track,
trim_start_ms=getattr(item, "trim_start_ms", 0),
trim_end_ms=getattr(item, "trim_end_ms", 0),
created_at=item.created_at,
profile_id=generation.profile_id,
profile_name=profile_name,
text=generation.text,
language=generation.language,
audio_path=audio_path,
duration=generation.duration,
seed=generation.seed,
instruct=generation.instruct,
generation_created_at=generation.created_at,
versions=versions,
active_version_id=active_version_id,
)
async def create_story( async def create_story(
data: StoryCreate, data: StoryCreate,
db: Session, db: Session,
@@ -52,10 +100,7 @@ async def create_story(
db.commit() db.commit()
db.refresh(db_story) db.refresh(db_story)
# Get item count item_count = db.query(func.count(DBStoryItem.id)).filter(DBStoryItem.story_id == db_story.id).scalar()
item_count = db.query(func.count(DBStoryItem.id)).filter(
DBStoryItem.story_id == db_story.id
).scalar()
response = StoryResponse.model_validate(db_story) response = StoryResponse.model_validate(db_story)
response.item_count = item_count response.item_count = item_count
@@ -75,17 +120,15 @@ async def list_stories(
List of stories with item counts List of stories with item counts
""" """
stories = db.query(DBStory).order_by(DBStory.updated_at.desc()).all() stories = db.query(DBStory).order_by(DBStory.updated_at.desc()).all()
result = [] result = []
for story in stories: for story in stories:
item_count = db.query(func.count(DBStoryItem.id)).filter( item_count = db.query(func.count(DBStoryItem.id)).filter(DBStoryItem.story_id == story.id).scalar()
DBStoryItem.story_id == story.id
).scalar()
response = StoryResponse.model_validate(story) response = StoryResponse.model_validate(story)
response.item_count = item_count response.item_count = item_count
result.append(response) result.append(response)
return result return result
@@ -107,44 +150,18 @@ async def get_story(
if not story: if not story:
return None return None
# Get all items ordered by start_time_ms items = (
items = db.query( db.query(DBStoryItem, DBGeneration, DBVoiceProfile.name.label("profile_name"))
DBStoryItem, .join(DBGeneration, DBStoryItem.generation_id == DBGeneration.id)
DBGeneration, .join(DBVoiceProfile, DBGeneration.profile_id == DBVoiceProfile.id)
DBVoiceProfile.name.label('profile_name') .filter(DBStoryItem.story_id == story_id)
).join( .order_by(DBStoryItem.start_time_ms)
DBGeneration, .all()
DBStoryItem.generation_id == DBGeneration.id )
).join(
DBVoiceProfile,
DBGeneration.profile_id == DBVoiceProfile.id
).filter(
DBStoryItem.story_id == story_id
).order_by(DBStoryItem.start_time_ms).all()
# Build item details
item_details = [] item_details = []
for item, generation, profile_name in items: for item, generation, profile_name in items:
item_detail = StoryItemDetail( item_details.append(_build_item_detail(item, generation, profile_name, db))
id=item.id,
story_id=item.story_id,
generation_id=item.generation_id,
start_time_ms=item.start_time_ms,
track=item.track,
trim_start_ms=getattr(item, 'trim_start_ms', 0),
trim_end_ms=getattr(item, 'trim_end_ms', 0),
created_at=item.created_at,
profile_id=generation.profile_id,
profile_name=profile_name,
text=generation.text,
language=generation.language,
audio_path=generation.audio_path,
duration=generation.duration,
seed=generation.seed,
instruct=generation.instruct,
generation_created_at=generation.created_at,
)
item_details.append(item_detail)
response = StoryDetailResponse.model_validate(story) response = StoryDetailResponse.model_validate(story)
response.items = item_details response.items = item_details
@@ -178,10 +195,7 @@ async def update_story(
db.commit() db.commit()
db.refresh(story) db.refresh(story)
# Get item count item_count = db.query(func.count(DBStoryItem.id)).filter(DBStoryItem.story_id == story.id).scalar()
item_count = db.query(func.count(DBStoryItem.id)).filter(
DBStoryItem.story_id == story.id
).scalar()
response = StoryResponse.model_validate(story) response = StoryResponse.model_validate(story)
response.item_count = item_count response.item_count = item_count
@@ -243,32 +257,11 @@ async def add_item_to_story(
return None return None
# Check if generation is already in story # Check if generation is already in story
existing = db.query(DBStoryItem).filter_by( existing = db.query(DBStoryItem).filter_by(story_id=story_id, generation_id=data.generation_id).first()
story_id=story_id,
generation_id=data.generation_id
).first()
if existing: if existing:
# Return existing item # Return existing item
profile = db.query(DBVoiceProfile).filter_by(id=generation.profile_id).first() profile = db.query(DBVoiceProfile).filter_by(id=generation.profile_id).first()
return StoryItemDetail( return _build_item_detail(existing, generation, profile.name if profile else "Unknown", db)
id=existing.id,
story_id=existing.story_id,
generation_id=existing.generation_id,
start_time_ms=existing.start_time_ms,
track=existing.track,
trim_start_ms=getattr(existing, 'trim_start_ms', 0),
trim_end_ms=getattr(existing, 'trim_end_ms', 0),
created_at=existing.created_at,
profile_id=generation.profile_id,
profile_name=profile.name if profile else "Unknown",
text=generation.text,
language=generation.language,
audio_path=generation.audio_path,
duration=generation.duration,
seed=generation.seed,
instruct=generation.instruct,
generation_created_at=generation.created_at,
)
# Get track from data or default to 0 # Get track from data or default to 0
track = data.track if data.track is not None else 0 track = data.track if data.track is not None else 0
@@ -277,18 +270,16 @@ async def add_item_to_story(
if data.start_time_ms is not None: if data.start_time_ms is not None:
start_time_ms = data.start_time_ms start_time_ms = data.start_time_ms
else: else:
# Find the maximum end time on the target track only existing_items = (
existing_items = db.query( db.query(DBStoryItem, DBGeneration)
DBStoryItem, .join(DBGeneration, DBStoryItem.generation_id == DBGeneration.id)
DBGeneration .filter(
).join( DBStoryItem.story_id == story_id,
DBGeneration, DBStoryItem.track == track,
DBStoryItem.generation_id == DBGeneration.id )
).filter( .all()
DBStoryItem.story_id == story_id, )
DBStoryItem.track == track,
).all()
if not existing_items: if not existing_items:
start_time_ms = 0 start_time_ms = 0
else: else:
@@ -296,7 +287,7 @@ async def add_item_to_story(
for item, gen in existing_items: for item, gen in existing_items:
item_end_ms = item.start_time_ms + int(gen.duration * 1000) item_end_ms = item.start_time_ms + int(gen.duration * 1000)
max_end_time_ms = max(max_end_time_ms, item_end_ms) max_end_time_ms = max(max_end_time_ms, item_end_ms)
# Add 200ms gap after the last item # Add 200ms gap after the last item
start_time_ms = max_end_time_ms + 200 start_time_ms = max_end_time_ms + 200
@@ -311,35 +302,17 @@ async def add_item_to_story(
) )
db.add(item) db.add(item)
# Update story updated_at # Update story updated_at
story.updated_at = datetime.utcnow() story.updated_at = datetime.utcnow()
db.commit() db.commit()
db.refresh(item) db.refresh(item)
# Get profile name # Get profile name
profile = db.query(DBVoiceProfile).filter_by(id=generation.profile_id).first() profile = db.query(DBVoiceProfile).filter_by(id=generation.profile_id).first()
return StoryItemDetail( return _build_item_detail(item, generation, profile.name if profile else "Unknown", db)
id=item.id,
story_id=item.story_id,
generation_id=item.generation_id,
start_time_ms=item.start_time_ms,
track=item.track,
trim_start_ms=getattr(item, 'trim_start_ms', 0),
trim_end_ms=getattr(item, 'trim_end_ms', 0),
created_at=item.created_at,
profile_id=generation.profile_id,
profile_name=profile.name if profile else "Unknown",
text=generation.text,
language=generation.language,
audio_path=generation.audio_path,
duration=generation.duration,
seed=generation.seed,
instruct=generation.instruct,
generation_created_at=generation.created_at,
)
async def move_story_item( async def move_story_item(
@@ -361,10 +334,14 @@ async def move_story_item(
Updated item detail or None if not found Updated item detail or None if not found
""" """
# Get the item # Get the item
item = db.query(DBStoryItem).filter_by( item = (
id=item_id, db.query(DBStoryItem)
story_id=story_id, .filter_by(
).first() id=item_id,
story_id=story_id,
)
.first()
)
if not item: if not item:
return None return None
@@ -388,25 +365,7 @@ async def move_story_item(
# Get profile name # Get profile name
profile = db.query(DBVoiceProfile).filter_by(id=generation.profile_id).first() profile = db.query(DBVoiceProfile).filter_by(id=generation.profile_id).first()
return StoryItemDetail( return _build_item_detail(item, generation, profile.name if profile else "Unknown", db)
id=item.id,
story_id=item.story_id,
generation_id=item.generation_id,
start_time_ms=item.start_time_ms,
track=item.track,
trim_start_ms=getattr(item, 'trim_start_ms', 0),
trim_end_ms=getattr(item, 'trim_end_ms', 0),
created_at=item.created_at,
profile_id=generation.profile_id,
profile_name=profile.name if profile else "Unknown",
text=generation.text,
language=generation.language,
audio_path=generation.audio_path,
duration=generation.duration,
seed=generation.seed,
instruct=generation.instruct,
generation_created_at=generation.created_at,
)
async def remove_item_from_story( async def remove_item_from_story(
@@ -425,10 +384,14 @@ async def remove_item_from_story(
Returns: Returns:
True if removed, False if not found True if removed, False if not found
""" """
item = db.query(DBStoryItem).filter_by( item = (
id=item_id, db.query(DBStoryItem)
story_id=story_id, .filter_by(
).first() id=item_id,
story_id=story_id,
)
.first()
)
if not item: if not item:
return False return False
@@ -463,10 +426,14 @@ async def trim_story_item(
Updated item detail or None if not found Updated item detail or None if not found
""" """
# Get the item # Get the item
item = db.query(DBStoryItem).filter_by( item = (
id=item_id, db.query(DBStoryItem)
story_id=story_id, .filter_by(
).first() id=item_id,
story_id=story_id,
)
.first()
)
if not item: if not item:
return None return None
@@ -495,25 +462,7 @@ async def trim_story_item(
# Get profile name # Get profile name
profile = db.query(DBVoiceProfile).filter_by(id=generation.profile_id).first() profile = db.query(DBVoiceProfile).filter_by(id=generation.profile_id).first()
return StoryItemDetail( return _build_item_detail(item, generation, profile.name if profile else "Unknown", db)
id=item.id,
story_id=item.story_id,
generation_id=item.generation_id,
start_time_ms=item.start_time_ms,
track=item.track,
trim_start_ms=item.trim_start_ms,
trim_end_ms=item.trim_end_ms,
created_at=item.created_at,
profile_id=generation.profile_id,
profile_name=profile.name if profile else "Unknown",
text=generation.text,
language=generation.language,
audio_path=generation.audio_path,
duration=generation.duration,
seed=generation.seed,
instruct=generation.instruct,
generation_created_at=generation.created_at,
)
async def split_story_item( async def split_story_item(
@@ -535,10 +484,14 @@ async def split_story_item(
List of two updated item details (original and new) or None if not found/invalid List of two updated item details (original and new) or None if not found/invalid
""" """
# Get the item # Get the item
item = db.query(DBStoryItem).filter_by( item = (
id=item_id, db.query(DBStoryItem)
story_id=story_id, .filter_by(
).first() id=item_id,
story_id=story_id,
)
.first()
)
if not item: if not item:
return None return None
@@ -548,8 +501,8 @@ async def split_story_item(
return None return None
# Calculate effective duration and validate split point # Calculate effective duration and validate split point
current_trim_start = getattr(item, 'trim_start_ms', 0) current_trim_start = getattr(item, "trim_start_ms", 0)
current_trim_end = getattr(item, 'trim_end_ms', 0) current_trim_end = getattr(item, "trim_end_ms", 0)
original_duration_ms = int(generation.duration * 1000) original_duration_ms = int(generation.duration * 1000)
effective_duration_ms = original_duration_ms - current_trim_start - current_trim_end effective_duration_ms = original_duration_ms - current_trim_start - current_trim_end
@@ -568,6 +521,7 @@ async def split_story_item(
id=str(uuid.uuid4()), id=str(uuid.uuid4()),
story_id=story_id, story_id=story_id,
generation_id=item.generation_id, # Same generation, different trim generation_id=item.generation_id, # Same generation, different trim
version_id=getattr(item, "version_id", None), # Preserve pinned version
start_time_ms=item.start_time_ms + data.split_time_ms, start_time_ms=item.start_time_ms + data.split_time_ms,
track=item.track, track=item.track,
trim_start_ms=absolute_split_ms, trim_start_ms=absolute_split_ms,
@@ -590,48 +544,10 @@ async def split_story_item(
profile = db.query(DBVoiceProfile).filter_by(id=generation.profile_id).first() profile = db.query(DBVoiceProfile).filter_by(id=generation.profile_id).first()
profile_name = profile.name if profile else "Unknown" profile_name = profile.name if profile else "Unknown"
# Build response items return [
original_item_detail = StoryItemDetail( _build_item_detail(item, generation, profile_name, db),
id=item.id, _build_item_detail(new_item, generation, profile_name, db),
story_id=item.story_id, ]
generation_id=item.generation_id,
start_time_ms=item.start_time_ms,
track=item.track,
trim_start_ms=item.trim_start_ms,
trim_end_ms=item.trim_end_ms,
created_at=item.created_at,
profile_id=generation.profile_id,
profile_name=profile_name,
text=generation.text,
language=generation.language,
audio_path=generation.audio_path,
duration=generation.duration,
seed=generation.seed,
instruct=generation.instruct,
generation_created_at=generation.created_at,
)
new_item_detail = StoryItemDetail(
id=new_item.id,
story_id=new_item.story_id,
generation_id=new_item.generation_id,
start_time_ms=new_item.start_time_ms,
track=new_item.track,
trim_start_ms=new_item.trim_start_ms,
trim_end_ms=new_item.trim_end_ms,
created_at=new_item.created_at,
profile_id=generation.profile_id,
profile_name=profile_name,
text=generation.text,
language=generation.language,
audio_path=generation.audio_path,
duration=generation.duration,
seed=generation.seed,
instruct=generation.instruct,
generation_created_at=generation.created_at,
)
return [original_item_detail, new_item_detail]
async def duplicate_story_item( async def duplicate_story_item(
@@ -651,10 +567,14 @@ async def duplicate_story_item(
New item detail or None if not found New item detail or None if not found
""" """
# Get the original item # Get the original item
original_item = db.query(DBStoryItem).filter_by( original_item = (
id=item_id, db.query(DBStoryItem)
story_id=story_id, .filter_by(
).first() id=item_id,
story_id=story_id,
)
.first()
)
if not original_item: if not original_item:
return None return None
@@ -664,8 +584,8 @@ async def duplicate_story_item(
return None return None
# Calculate effective duration # Calculate effective duration
current_trim_start = getattr(original_item, 'trim_start_ms', 0) current_trim_start = getattr(original_item, "trim_start_ms", 0)
current_trim_end = getattr(original_item, 'trim_end_ms', 0) current_trim_end = getattr(original_item, "trim_end_ms", 0)
original_duration_ms = int(generation.duration * 1000) original_duration_ms = int(generation.duration * 1000)
effective_duration_ms = original_duration_ms - current_trim_start - current_trim_end effective_duration_ms = original_duration_ms - current_trim_start - current_trim_end
@@ -674,6 +594,7 @@ async def duplicate_story_item(
id=str(uuid.uuid4()), id=str(uuid.uuid4()),
story_id=story_id, story_id=story_id,
generation_id=original_item.generation_id, # Same generation as original generation_id=original_item.generation_id, # Same generation as original
version_id=getattr(original_item, "version_id", None), # Preserve pinned version
start_time_ms=original_item.start_time_ms + effective_duration_ms + 200, # 200ms gap start_time_ms=original_item.start_time_ms + effective_duration_ms + 200, # 200ms gap
track=original_item.track, track=original_item.track,
trim_start_ms=current_trim_start, trim_start_ms=current_trim_start,
@@ -694,25 +615,7 @@ async def duplicate_story_item(
# Get profile name # Get profile name
profile = db.query(DBVoiceProfile).filter_by(id=generation.profile_id).first() profile = db.query(DBVoiceProfile).filter_by(id=generation.profile_id).first()
return StoryItemDetail( return _build_item_detail(new_item, generation, profile.name if profile else "Unknown", db)
id=new_item.id,
story_id=new_item.story_id,
generation_id=new_item.generation_id,
start_time_ms=new_item.start_time_ms,
track=new_item.track,
trim_start_ms=new_item.trim_start_ms,
trim_end_ms=new_item.trim_end_ms,
created_at=new_item.created_at,
profile_id=generation.profile_id,
profile_name=profile.name if profile else "Unknown",
text=generation.text,
language=generation.language,
audio_path=generation.audio_path,
duration=generation.duration,
seed=generation.seed,
instruct=generation.instruct,
generation_created_at=generation.created_at,
)
async def update_story_item_times( async def update_story_item_times(
@@ -775,19 +678,13 @@ async def reorder_story_items(
return None return None
# Get all items for this story with their generation data # Get all items for this story with their generation data
items_with_gen = db.query( items_with_gen = (
DBStoryItem, db.query(DBStoryItem, DBGeneration, DBVoiceProfile.name.label("profile_name"))
DBGeneration, .join(DBGeneration, DBStoryItem.generation_id == DBGeneration.id)
DBVoiceProfile.name.label('profile_name') .join(DBVoiceProfile, DBGeneration.profile_id == DBVoiceProfile.id)
).join( .filter(DBStoryItem.story_id == story_id)
DBGeneration, .all()
DBStoryItem.generation_id == DBGeneration.id )
).join(
DBVoiceProfile,
DBGeneration.profile_id == DBVoiceProfile.id
).filter(
DBStoryItem.story_id == story_id
).all()
# Create maps for quick lookup # Create maps for quick lookup
item_map = {item.generation_id: (item, gen, profile_name) for item, gen, profile_name in items_with_gen} item_map = {item.generation_id: (item, gen, profile_name) for item, gen, profile_name in items_with_gen}
@@ -802,36 +699,18 @@ async def reorder_story_items(
for gen_id in generation_ids: for gen_id in generation_ids:
item, generation, profile_name = item_map[gen_id] item, generation, profile_name = item_map[gen_id]
# Update the item's start time # Update the item's start time
item.start_time_ms = current_time_ms item.start_time_ms = current_time_ms
# Calculate the duration in ms # Calculate the duration in ms
duration_ms = int(generation.duration * 1000) duration_ms = int(generation.duration * 1000)
# Move to next position (current end + gap) # Move to next position (current end + gap)
current_time_ms += duration_ms + gap_ms current_time_ms += duration_ms + gap_ms
# Build the response item # Build the response item
updated_items.append(StoryItemDetail( updated_items.append(_build_item_detail(item, generation, profile_name, db))
id=item.id,
story_id=item.story_id,
generation_id=item.generation_id,
start_time_ms=item.start_time_ms,
track=item.track,
trim_start_ms=getattr(item, 'trim_start_ms', 0),
trim_end_ms=getattr(item, 'trim_end_ms', 0),
created_at=item.created_at,
profile_id=generation.profile_id,
profile_name=profile_name,
text=generation.text,
language=generation.language,
audio_path=generation.audio_path,
duration=generation.duration,
seed=generation.seed,
instruct=generation.instruct,
generation_created_at=generation.created_at,
))
# Update story updated_at # Update story updated_at
story.updated_at = datetime.utcnow() story.updated_at = datetime.utcnow()
@@ -840,6 +719,69 @@ async def reorder_story_items(
return updated_items return updated_items
async def set_story_item_version(
story_id: str,
item_id: str,
data: StoryItemVersionUpdate,
db: Session,
) -> Optional[StoryItemDetail]:
"""
Pin a story item to a specific generation version.
Args:
story_id: Story ID
item_id: Story item ID
data: Version update data (version_id or null for default)
db: Database session
Returns:
Updated item detail or None if not found
"""
item = (
db.query(DBStoryItem)
.filter_by(
id=item_id,
story_id=story_id,
)
.first()
)
if not item:
return None
generation = db.query(DBGeneration).filter_by(id=item.generation_id).first()
if not generation:
return None
# Validate version_id belongs to this generation if provided
if data.version_id:
from ..database import GenerationVersion as DBGenerationVersion
version = (
db.query(DBGenerationVersion)
.filter_by(
id=data.version_id,
generation_id=item.generation_id,
)
.first()
)
if not version:
return None
item.version_id = data.version_id
# Update story updated_at
story = db.query(DBStory).filter_by(id=story_id).first()
if story:
story.updated_at = datetime.utcnow()
db.commit()
db.refresh(item)
profile = db.query(DBVoiceProfile).filter_by(id=generation.profile_id).first()
return _build_item_detail(item, generation, profile.name if profile else "Unknown", db)
async def export_story_audio( async def export_story_audio(
story_id: str, story_id: str,
db: Session, db: Session,
@@ -859,15 +801,13 @@ async def export_story_audio(
return None return None
# Get all items ordered by start_time_ms # Get all items ordered by start_time_ms
items = db.query( items = (
DBStoryItem, db.query(DBStoryItem, DBGeneration)
DBGeneration .join(DBGeneration, DBStoryItem.generation_id == DBGeneration.id)
).join( .filter(DBStoryItem.story_id == story_id)
DBGeneration, .order_by(DBStoryItem.start_time_ms)
DBStoryItem.generation_id == DBGeneration.id .all()
).filter( )
DBStoryItem.story_id == story_id
).order_by(DBStoryItem.start_time_ms).all()
if not items: if not items:
return None return None
@@ -877,40 +817,53 @@ async def export_story_audio(
sample_rate = 24000 # Default sample rate sample_rate = 24000 # Default sample rate
for item, generation in items: for item, generation in items:
audio_path = Path(generation.audio_path) # Resolve audio path: use pinned version if set, otherwise generation default
resolved_audio_path = generation.audio_path
if getattr(item, "version_id", None):
from ..database import GenerationVersion as DBGenerationVersion
version = db.query(DBGenerationVersion).filter_by(id=item.version_id).first()
if version:
resolved_audio_path = version.audio_path
audio_path = Path(resolved_audio_path)
if not audio_path.exists(): if not audio_path.exists():
continue continue
try: try:
audio, sr = load_audio(str(audio_path), sample_rate=sample_rate) audio, sr = load_audio(str(audio_path), sample_rate=sample_rate)
sample_rate = sr # Use actual sample rate from first file sample_rate = sr # Use actual sample rate from first file
# Get trim values # Get trim values
trim_start_ms = getattr(item, 'trim_start_ms', 0) trim_start_ms = getattr(item, "trim_start_ms", 0)
trim_end_ms = getattr(item, 'trim_end_ms', 0) trim_end_ms = getattr(item, "trim_end_ms", 0)
# Calculate effective duration # Calculate effective duration
original_duration_ms = int(generation.duration * 1000) original_duration_ms = int(generation.duration * 1000)
effective_duration_ms = original_duration_ms - trim_start_ms - trim_end_ms effective_duration_ms = original_duration_ms - trim_start_ms - trim_end_ms
# Slice audio based on trim values # Slice audio based on trim values
trim_start_sample = int((trim_start_ms / 1000.0) * sample_rate) trim_start_sample = int((trim_start_ms / 1000.0) * sample_rate)
trim_end_sample = int((trim_end_ms / 1000.0) * sample_rate) trim_end_sample = int((trim_end_ms / 1000.0) * sample_rate)
# Extract the trimmed portion # Extract the trimmed portion
if trim_end_ms > 0: if trim_end_ms > 0:
trimmed_audio = audio[trim_start_sample:-trim_end_sample] if trim_end_sample > 0 else audio[trim_start_sample:] trimmed_audio = (
audio[trim_start_sample:-trim_end_sample] if trim_end_sample > 0 else audio[trim_start_sample:]
)
else: else:
trimmed_audio = audio[trim_start_sample:] trimmed_audio = audio[trim_start_sample:]
# Store audio with its timecode info # Store audio with its timecode info
start_time_ms = item.start_time_ms start_time_ms = item.start_time_ms
audio_data.append({ audio_data.append(
'audio': trimmed_audio, {
'start_time_ms': start_time_ms, "audio": trimmed_audio,
'duration_ms': effective_duration_ms, "start_time_ms": start_time_ms,
}) "duration_ms": effective_duration_ms,
}
)
except Exception: except Exception:
# Skip files that can't be loaded # Skip files that can't be loaded
continue continue
@@ -919,33 +872,30 @@ async def export_story_audio(
return None return None
# Calculate total duration: max(start_time_ms + duration_ms) # Calculate total duration: max(start_time_ms + duration_ms)
max_end_time_ms = max( max_end_time_ms = max((data["start_time_ms"] + data["duration_ms"] for data in audio_data), default=0)
(data['start_time_ms'] + data['duration_ms'] for data in audio_data),
default=0
)
# Convert to samples # Convert to samples
total_samples = int((max_end_time_ms / 1000.0) * sample_rate) total_samples = int((max_end_time_ms / 1000.0) * sample_rate)
# Create output buffer initialized to zeros # Create output buffer initialized to zeros
final_audio = np.zeros(total_samples, dtype=np.float32) final_audio = np.zeros(total_samples, dtype=np.float32)
# Mix each audio segment at its timecode position # Mix each audio segment at its timecode position
for data in audio_data: for data in audio_data:
audio = data['audio'] audio = data["audio"]
start_time_ms = data['start_time_ms'] start_time_ms = data["start_time_ms"]
# Calculate start sample index # Calculate start sample index
start_sample = int((start_time_ms / 1000.0) * sample_rate) start_sample = int((start_time_ms / 1000.0) * sample_rate)
# Ensure we don't exceed buffer bounds # Ensure we don't exceed buffer bounds
audio_length = len(audio) audio_length = len(audio)
end_sample = min(start_sample + audio_length, total_samples) end_sample = min(start_sample + audio_length, total_samples)
if start_sample < total_samples: if start_sample < total_samples:
# Trim audio if it extends beyond buffer # Trim audio if it extends beyond buffer
audio_to_mix = audio[:end_sample - start_sample] audio_to_mix = audio[: end_sample - start_sample]
# Mix: add audio to existing buffer (overlapping audio will sum) # Mix: add audio to existing buffer (overlapping audio will sum)
# Normalize to prevent clipping (simple approach: divide by max) # Normalize to prevent clipping (simple approach: divide by max)
final_audio[start_sample:end_sample] += audio_to_mix final_audio[start_sample:end_sample] += audio_to_mix
@@ -956,14 +906,14 @@ async def export_story_audio(
final_audio = final_audio / max_val final_audio = final_audio / max_val
# Save to temporary file # Save to temporary file
with tempfile.NamedTemporaryFile(suffix='.wav', delete=False) as tmp: with tempfile.NamedTemporaryFile(suffix=".wav", delete=False) as tmp:
tmp_path = tmp.name tmp_path = tmp.name
try: try:
save_audio(final_audio, tmp_path, sample_rate) save_audio(final_audio, tmp_path, sample_rate)
# Read file bytes # Read file bytes
with open(tmp_path, 'rb') as f: with open(tmp_path, "rb") as f:
audio_bytes = f.read() audio_bytes = f.read()
return audio_bytes return audio_bytes
+48
View File
@@ -0,0 +1,48 @@
"""
Serial generation queue — ensures only one TTS inference runs at a time
to avoid GPU contention.
"""
import asyncio
import traceback
# Keep references to fire-and-forget background tasks to prevent GC
_background_tasks: set = set()
# Generation queue — serializes TTS inference to avoid GPU contention
_generation_queue: asyncio.Queue = None # type: ignore # initialized at startup
def create_background_task(coro) -> asyncio.Task:
"""Create a background task and prevent it from being garbage collected."""
task = asyncio.create_task(coro)
_background_tasks.add(task)
task.add_done_callback(_background_tasks.discard)
return task
async def _generation_worker():
"""Worker that processes generation tasks one at a time."""
while True:
coro = await _generation_queue.get()
try:
await coro
except Exception:
traceback.print_exc()
finally:
_generation_queue.task_done()
def enqueue_generation(coro):
"""Add a generation coroutine to the serial queue."""
_generation_queue.put_nowait(coro)
def init_queue():
"""Initialize the generation queue and start the worker.
Must be called once during application startup (inside a running event loop).
"""
global _generation_queue
_generation_queue = asyncio.Queue()
create_background_task(_generation_worker())
@@ -3,7 +3,7 @@ STT (Speech-to-Text) module - delegates to backend abstraction layer.
""" """
from typing import Optional from typing import Optional
from .backends import get_stt_backend, STTBackend from ..backends import get_stt_backend, STTBackend
def get_whisper_model() -> STTBackend: def get_whisper_model() -> STTBackend:
+1 -1
View File
@@ -7,7 +7,7 @@ import numpy as np
import io import io
import soundfile as sf import soundfile as sf
from .backends import get_tts_backend, TTSBackend from ..backends import get_tts_backend, TTSBackend
def get_tts_model() -> TTSBackend: def get_tts_model() -> TTSBackend:
+211
View File
@@ -0,0 +1,211 @@
"""
Generation versions management module.
Each generation can have multiple audio versions: a clean (unprocessed)
version and any number of processed versions with different effects chains.
"""
from __future__ import annotations
import json
import uuid
from pathlib import Path
from typing import List, Optional
from sqlalchemy.orm import Session
from ..database import (
GenerationVersion as DBGenerationVersion,
Generation as DBGeneration,
)
from ..models import GenerationVersionResponse, EffectConfig
from .. import config
def _version_response(v: DBGenerationVersion) -> GenerationVersionResponse:
"""Convert a DB version row to a Pydantic response."""
effects_chain = None
if v.effects_chain:
raw = json.loads(v.effects_chain)
effects_chain = [EffectConfig(**e) for e in raw]
return GenerationVersionResponse(
id=v.id,
generation_id=v.generation_id,
label=v.label,
audio_path=v.audio_path,
effects_chain=effects_chain,
source_version_id=v.source_version_id,
is_default=v.is_default,
created_at=v.created_at,
)
def list_versions(generation_id: str, db: Session) -> List[GenerationVersionResponse]:
"""List all versions for a generation."""
versions = (
db.query(DBGenerationVersion)
.filter_by(generation_id=generation_id)
.order_by(DBGenerationVersion.created_at)
.all()
)
return [_version_response(v) for v in versions]
def get_version(version_id: str, db: Session) -> Optional[GenerationVersionResponse]:
"""Get a specific version by ID."""
v = db.query(DBGenerationVersion).filter_by(id=version_id).first()
if not v:
return None
return _version_response(v)
def get_default_version(generation_id: str, db: Session) -> Optional[GenerationVersionResponse]:
"""Get the default version for a generation."""
v = (
db.query(DBGenerationVersion)
.filter_by(generation_id=generation_id, is_default=True)
.first()
)
if not v:
# Fallback: return the first version
v = (
db.query(DBGenerationVersion)
.filter_by(generation_id=generation_id)
.order_by(DBGenerationVersion.created_at)
.first()
)
if not v:
return None
return _version_response(v)
def create_version(
generation_id: str,
label: str,
audio_path: str,
db: Session,
effects_chain: Optional[List[dict]] = None,
is_default: bool = False,
source_version_id: Optional[str] = None,
) -> GenerationVersionResponse:
"""Create a new version for a generation.
If ``is_default`` is True, all other versions for this generation
are un-defaulted first.
"""
if is_default:
_clear_defaults(generation_id, db)
version = DBGenerationVersion(
id=str(uuid.uuid4()),
generation_id=generation_id,
label=label,
audio_path=audio_path,
effects_chain=json.dumps(effects_chain) if effects_chain else None,
source_version_id=source_version_id,
is_default=is_default,
)
db.add(version)
db.commit()
db.refresh(version)
# If this version is the default, update the generation's audio_path
if is_default:
gen = db.query(DBGeneration).filter_by(id=generation_id).first()
if gen:
gen.audio_path = audio_path
db.commit()
return _version_response(version)
def set_default_version(version_id: str, db: Session) -> Optional[GenerationVersionResponse]:
"""Set a version as the default for its generation."""
version = db.query(DBGenerationVersion).filter_by(id=version_id).first()
if not version:
return None
_clear_defaults(version.generation_id, db)
version.is_default = True
db.commit()
db.refresh(version)
# Update generation's audio_path to point to this version
gen = db.query(DBGeneration).filter_by(id=version.generation_id).first()
if gen:
gen.audio_path = version.audio_path
db.commit()
return _version_response(version)
def delete_version(version_id: str, db: Session) -> bool:
"""Delete a version. Cannot delete the last remaining version."""
version = db.query(DBGenerationVersion).filter_by(id=version_id).first()
if not version:
return False
# Don't allow deleting the last version
count = (
db.query(DBGenerationVersion)
.filter_by(generation_id=version.generation_id)
.count()
)
if count <= 1:
return False
was_default = version.is_default
gen_id = version.generation_id
# Delete audio file
audio_path = Path(version.audio_path)
if audio_path.exists():
audio_path.unlink()
db.delete(version)
db.commit()
# If this was the default, promote the first remaining version
if was_default:
first = (
db.query(DBGenerationVersion)
.filter_by(generation_id=gen_id)
.order_by(DBGenerationVersion.created_at)
.first()
)
if first:
first.is_default = True
db.commit()
gen = db.query(DBGeneration).filter_by(id=gen_id).first()
if gen:
gen.audio_path = first.audio_path
db.commit()
return True
def delete_versions_for_generation(generation_id: str, db: Session) -> int:
"""Delete all versions for a generation (used when deleting a generation)."""
versions = (
db.query(DBGenerationVersion)
.filter_by(generation_id=generation_id)
.all()
)
count = 0
for v in versions:
audio_path = Path(v.audio_path)
if audio_path.exists():
audio_path.unlink()
db.delete(v)
count += 1
if count > 0:
db.commit()
return count
def _clear_defaults(generation_id: str, db: Session) -> None:
"""Clear the is_default flag on all versions for a generation."""
db.query(DBGenerationVersion).filter_by(
generation_id=generation_id, is_default=True
).update({"is_default": False})
db.flush()
-66
View File
@@ -1,66 +0,0 @@
"""
Audio studio module for timeline editing.
"""
from typing import List, Dict, Optional
import numpy as np
class AudioStudio:
"""Audio editing and timeline management."""
async def get_word_timestamps(
self,
audio_path: str,
text: str,
) -> List[Dict[str, float]]:
"""
Get word-level timestamps for audio.
Args:
audio_path: Path to audio file
text: Corresponding text
Returns:
List of word timestamps: [{"word": "...", "start": 0.0, "end": 0.5}, ...]
"""
# TODO: Implement Whisper alignment
raise NotImplementedError("Word timestamps not yet implemented")
async def mix_audio(
self,
audio_paths: List[str],
volumes: Optional[List[float]] = None,
) -> bytes:
"""
Mix multiple audio files together.
Args:
audio_paths: List of audio file paths
volumes: Optional volume levels (0.0-1.0) for each track
Returns:
Mixed audio bytes (WAV format)
"""
# TODO: Implement audio mixing
raise NotImplementedError("Audio mixing not yet implemented")
async def trim_audio(
self,
audio_path: str,
start: float,
end: float,
) -> bytes:
"""
Trim audio to specified time range.
Args:
audio_path: Path to audio file
start: Start time in seconds
end: End time in seconds
Returns:
Trimmed audio bytes (WAV format)
"""
# TODO: Implement audio trimming
raise NotImplementedError("Audio trimming not yet implemented")
+15 -31
View File
@@ -37,11 +37,10 @@ async def monitor_sse_stream(model_name: str, timeout: int = 120):
if line.startswith("data: "): if line.startswith("data: "):
try: try:
data = json.loads(line[6:]) data = json.loads(line[6:])
print(f"[{timestamp}] → SSE Event: {data['status']:12} {data.get('progress', 0):6.1f}% {data.get('filename', '')}") print(
events.append({ f"[{timestamp}] → SSE Event: {data['status']:12} {data.get('progress', 0):6.1f}% {data.get('filename', '')}"
**data, )
"_timestamp": timestamp events.append({**data, "_timestamp": timestamp})
})
# Stop if complete or error # Stop if complete or error
if data.get("status") in ("complete", "error"): if data.get("status") in ("complete", "error"):
@@ -74,12 +73,15 @@ async def trigger_generation(profile_id: str, text: str, model_size: str = "1.7B
try: try:
async with httpx.AsyncClient(timeout=120) as client: async with httpx.AsyncClient(timeout=120) as client:
response = await client.post(url, json={ response = await client.post(
"profile_id": profile_id, url,
"text": text, json={
"language": "en", "profile_id": profile_id,
"model_size": model_size, "text": text,
}) "language": "en",
"model_size": model_size,
},
)
print(f"[{_timestamp()}] Response: {response.status_code}") print(f"[{_timestamp()}] Response: {response.status_code}")
@@ -140,7 +142,7 @@ def _timestamp():
async def test_generation_with_cached_model(): async def test_generation_with_cached_model():
""" """
Test Case 1: Generation when model is already cached. Test Case 1: Generation when model is already cached.
This should NOT show any download progress events. This should NOT show any download progress events.
If it does, that's the UX bug we're trying to fix. If it does, that's the UX bug we're trying to fix.
""" """
@@ -194,7 +196,7 @@ async def test_generation_with_cached_model():
async def test_generation_with_fresh_download(): async def test_generation_with_fresh_download():
""" """
Test Case 2: Generation when model needs to be downloaded. Test Case 2: Generation when model needs to be downloaded.
This SHOULD show download progress events. This SHOULD show download progress events.
""" """
print("\n" + "=" * 80) print("\n" + "=" * 80)
@@ -292,24 +294,6 @@ async def main():
print(" Users see progress events even when the model is already cached,") print(" Users see progress events even when the model is already cached,")
print(" making them think the model is downloading again.") print(" making them think the model is downloading again.")
# Test Case 2: Fresh download (optional, commented out by default)
# Uncomment if you want to test download progress
# print("\n" + "🧪 " * 20)
# events_download = await test_generation_with_fresh_download()
#
# print("\n" + "=" * 80)
# print("TEST CASE 2 RESULTS: Generation with Model Download")
# print("=" * 80)
#
# if not events_download:
# print("ℹ Model was already cached, no download occurred")
# else:
# print(f"✓ Received {len(events_download)} download progress events")
# print("\nDownload Timeline:")
# for i, event in enumerate(events_download, 1):
# timestamp = event.pop("_timestamp", "??:??:??.???")
# print(f" {i}. [{timestamp}] {event}")
print("\n" + "=" * 80) print("\n" + "=" * 80)
print("Test Complete!") print("Test Complete!")
print("=" * 80) print("=" * 80)

Some files were not shown because too many files have changed in this diff Show More