Compare commits

...
Author SHA1 Message Date
Jamie PineandGitHub 6d261c44a1 Merge branch 'main' into feat/post-processing-effects 2026-03-14 12:14:26 -07:00
Jamie Pine 103e98b38f github runners suck 2026-03-14 12:13:45 -07:00
Jamie Pine 1c61b47a64 Glassmorphic active state for sidebar buttons with accent border shine 2026-03-14 12:11:07 -07:00
Jamie Pine 626e3740e1 Auto-select first story when navigating to Stories tab 2026-03-14 11:14:46 -07:00
Jamie Pine 310a4acb02 Add source version selection when applying effects, voices tab overhaul with inline inspector 2026-03-14 11:07:32 -07:00
Jamie Pine 899b90202b Add version control to track editor, restyle story list
- Story items can be pinned to a specific generation version via
  toolbar dropdown (shows when clip is selected and has >1 version)
- version_id column on story_items with migration, validated against
  the generation's versions before saving
- Split/duplicate preserve the source clip's pinned version
- Export and playback resolve version-specific audio paths
- Extracted _build_item_detail helper in stories.py (DRY cleanup)
- Story list restyled from rounded cards to flat rows with rounded
  hover/active states, gradient header fade, and dynamic bottom
  padding that accounts for track editor + generate box
2026-03-14 09:56:27 -07:00
Jamie Pine e8d54d52d3 Add favorites, effects badge on profiles, UI polish
- Add is_favorited column with toggle endpoint and star button on history
- Show sparkles icon on profile cards that have effects configured
- Gold ring on selected profile cards
- Smaller, gray action buttons with brighter hover
- Clamp player time to duration to prevent runaway playback
- Align profile card icon to top for wrapped names
- Flush bottom corners on history card when versions expanded
- Simplify .gitignore data/ rule
2026-03-14 09:10:56 -07:00
Jamie Pine 00c5b75ffb Regenerate as new version, UI polish, and bugfixes
- Add /generate/{id}/regenerate endpoint that creates a new take version
- Wire regenerate into history dropdown with SSE progress + autoplay
- Add normalize_audio to regenerate path
- Remove duplicate Regenerate menu item
- Show disabled ellipsis menu during generation instead of hiding it
- Fix sf.write format kwarg broken by asyncio.to_thread migration
- Rename clean version label to 'original', effects to 'version-N'
- Show effects chain names in version list instead of 'N fx'
- Include all versions in export package
- Clean up version panel padding
2026-03-14 08:34:58 -07:00
Jamie Pine 25134b4ba9 Fix player not loading new version after applying effects
Reload the player with the version-specific audio URL when effects are
applied to the currently playing generation. Also consolidate the
instruct/effects buttons into a single button with the effects editor
shown inline when instruct mode is open.
2026-03-14 08:01:45 -07:00
Jamie Pine 3d922ec846 Fix review findings: toggle logic, preset saving, version lookup, async audio ops
- Fix inverted effects toggle in FloatingGenerateBox
- Add Save button + API method for editing custom effect presets
- Return early on effects save failure in ProfileForm
- Fix no-op ternary in effectsStore
- Handle duplicate preset names with proper 400 response
- Use effects_chain is None instead of label for clean version lookup
- Move blocking audio ops to asyncio.to_thread in async endpoints
- Log warnings instead of silently swallowing parse errors
2026-03-14 07:47:06 -07:00
James Pine 638820c839 Add post-processing audio effects system
Adds a full effects pipeline powered by Spotify's pedalboard library,
enabling users to apply professional DSP effects (flanger, reverb, delay,
compressor, pitch shift, filters, gain) to generated audio.

Key features:
- Effects chain editor with drag-and-drop reordering (dnd-kit)
- Generation versions: clean copy always saved, processed versions created on top
- Built-in presets (Robotic, Radio, Echo Chamber, Deep Voice) + custom user presets
- Per-profile default effects chain (auto-applied to new generations)
- Per-generation effects override from the generation form
- Apply effects to existing generations from history (creates new version)
- Ephemeral preview endpoint for auditioning effects without persisting
- Dedicated Effects sidebar tab with preset management and live preview
- Version switcher in history cards with expandable panel
- Backward-compatible: existing generations backfilled as clean versions
2026-03-14 07:11:56 -07:00
Jamie Pine 7cbf5a1ded Use Namespace runner for Linux release build 2026-03-14 05:38:25 -07:00
Jamie Pine e89a7eb7e7 Windows dev experience: CUDA detection, GPU display cleanup, hide drag region, dev-mode update status 2026-03-14 03:24:39 -07:00
Jamie Pine e18757bab3 Skip AppImage on Linux, build deb/rpm only 2026-03-14 03:04:59 -07:00
Jamie Pine 0e6c678fc3 Add Windows support to justfile 2026-03-14 02:41:59 -07:00
Jamie Pine 942dabbcac Install CPU-only PyTorch before requirements.txt on Linux
The previous ordering let requirements.txt pull CUDA-enabled torch
with all nvidia deps first, then the CPU override only swapped the
torch wheel while leaving ~3GB of nvidia packages installed.
2026-03-14 00:55:43 -07:00
Jamie Pine b915825165 Add libasound2-dev to Linux release deps 2026-03-13 21:36:46 -07:00
Jamie Pine f3fc63942f Fix Linux release build exceeding 4GB PyInstaller limit
Install CPU-only PyTorch on Linux and exclude nvidia modules from
non-CUDA PyInstaller builds.
2026-03-13 21:19:24 -07:00
Jamie Pine 9044b986f3 Move tauri imports behind platform abstraction, merge CUDA build into release workflow
- Add openPath and pickDirectory to PlatformFilesystem interface
- Remove direct @tauri-apps/plugin-shell and plugin-dialog imports from app
- Merge build-cuda.yml into release.yml as a parallel job
2026-03-13 11:33:32 -07:00
Jamie Pine b01076b6b3 Bump version: 0.1.13 → 0.2.0 2026-03-13 11:06:40 -07:00
Jamie PineandGitHub 5121c76e39 Merge pull request #269 from jamiepine/feat/async-generation-queue
feat: async generation queue
2026-03-13 10:58:18 -07:00
Jamie Pine 49ebf6222e fix SSE cleanup, filter add popover to completed only, derive isGenerating from pending set, handle not_found status 2026-03-13 10:57:28 -07:00
Jamie Pine 509b0e71cc responsive layout fixes, version in sidebar, fixed voice card height, hide player title at small widths 2026-03-13 10:44:53 -07:00
Jamie Pine 81f8be1a94 defer story add until TTS completes, add generating pill to story editor, fix item placement per-track 2026-03-13 10:28:20 -07:00
Jamie Pine 655a60ca81 feat: async generation queue with serial execution
Generations now return immediately with a 'generating' status and appear
in history right away. TTS runs in a serial background queue to avoid
GPU contention. Users can kick off multiple generations without blocking.

- Async POST /generate creates DB record immediately, queues TTS work
- Serial generation queue prevents concurrent GPU access (Metal/CUDA/CPU)
- SSE endpoint GET /generate/{id}/status for real-time completion tracking
- Retry endpoint POST /generate/{id}/retry for failed generations
- Store engine and model_size on generation records for retry support
- History cards show animated loader (react-loaders) for generating/playing
- Failed generations show retry button instead of actions menu
- Model downloads happen inline in the queue instead of rejecting with 202
- Stale 'generating' records marked as failed on server startup
- Autoplay on generate setting (default: on)
- Show engine name on generation cards
- Remove sidebar generation spinner
- Checkbox alignment fix in settings
2026-03-13 10:02:41 -07:00
Jamie PineandGitHub 52285362ce Merge pull request #268 from jamiepine/feat/model-management-improvements
feat: model management improvements and folder migration
2026-03-13 09:16:43 -07:00
Jamie Pine 3ea587797f feat: model management improvements and folder migration
- Add model folder migration with byte-level progress tracking (backend + UI)
- Custom models directory support via VOICEBOX_MODELS_DIR env var passed to sidecar
- Hardcoded model descriptions displayed in model detail cards
- Open model folder button in storage location row
- Remove 'not downloaded' badge from model cards
- Fix server settings scroll offset for audio player
- Fix shell open permission to allow file paths
- Add normalize toggle to generation settings
2026-03-13 08:38:20 -07:00
Jamie PineandGitHub 325714bb83 Merge pull request #266 from jamiepine/feat/chunked-tts
feat: chunked TTS generation for long text (engine-agnostic)
2026-03-13 08:23:53 -07:00
James Pine 9aa7080c51 refactor: restructure server settings and models UI
- Split chunking/crossfade sliders into dedicated GenerationSettings card
- Merge connection status badges into ConnectionForm (remove ServerStatus card)
- 2-column grid layout for the entire settings page
- GPU Acceleration: remove icon, badge, and MLX info card
- Models: merge 'Other Voice Models' into single 'Voice Generation' list
- Model detail: remove 'Downloaded' badge, border above actions, swap
  badges above stats row, match disk size font to stats
2026-03-13 07:26:46 -07:00
James Pine 97292ecef7 feat: add chunk crossfade slider (0ms = hard cut)
Persisted setting (default 50ms) controls how audio chunks are blended
together.  Set to 0 for a clean hard cut with no overlap.
2026-03-13 06:48:06 -07:00
James Pine 837f8525d8 feat: add auto-chunking limit slider to settings
Persisted setting (default 800 chars) controls how long text is split
before generation.  Lower values improve quality for long outputs by
keeping each chunk well within the model's context window.

- Slider in Server Connection settings (100–2000 chars, step 50)
- Stored in localStorage via Zustand persist
- Passed as max_chunk_chars on every generation request
- Frontend text limit raised to 50,000 to match backend
2026-03-13 06:35:39 -07:00
James Pine 70ca7f66cb feat: chunked TTS generation for long text (engine-agnostic)
Text exceeding max_chunk_chars (default 800) is automatically split at
sentence boundaries, generated per-chunk, and concatenated with a 50ms
crossfade.  Works with all engines (Qwen, LuxTTS, Chatterbox, Turbo).

- Abbreviation-aware sentence splitter (Dr., Mr., e.g., decimals)
- CJK sentence-ending punctuation support
- Paralinguistic tag preservation ([laugh], [cough], etc.)
- Per-chunk seed variation to avoid correlated RNG artefacts
- Per-chunk Chatterbox trim (catches hallucination at each boundary)
- max_chunk_chars exposed as per-request param on GenerationRequest
- Text max_length raised to 50,000 characters

Closes #99
2026-03-13 06:21:34 -07:00
Jamie PineandGitHub c12b5d6f0a Merge pull request #265 from jamiepine/feat/paralinguistic-tags
feat: paralinguistic tag autocomplete for Chatterbox Turbo
2026-03-13 05:55:06 -07:00
James Pine 139fa38e3f fix: address review feedback for ParalinguisticInput
- Initialize lastSerializedRef to empty string so first-mount hydration
  always runs (fixes initial value not rendering)
- Guard arrow-key menu nav against empty filteredTags (avoids NaN index)
- Disable ARIA role/multiline and detach event handlers when disabled
- Add onBlur to close autocomplete dropdown when editor loses focus
- Chain exception with 'from e' in unload endpoint for better tracebacks
2026-03-13 05:52:06 -07:00
Jamie PineandGitHub 0e9f5db40f Merge pull request #264 from jamiepine/fix/chatterbox-float64-dtype
fix: Chatterbox float64 dtype mismatch + model unload button
2026-03-13 05:40:46 -07:00
James Pine 2f535a772f fix: load model into local var before patching to avoid half-initialised state
Apply local-var-then-assign pattern to chatterbox_backend.py (multilingual)
to match the turbo backend. Also use _current_model_size fallback in
unload, delete, and status endpoints for consistent Qwen model size checks.
2026-03-13 05:40:18 -07:00
James Pine b420637957 feat: paralinguistic tag autocomplete for Chatterbox Turbo
Type / in the text input when using Chatterbox Turbo to open an
autocomplete dropdown with 9 supported paralinguistic tags ([laugh],
[chuckle], [gasp], [cough], [sigh], [groan], [sniff], [shush],
[clear throat]).

- contentEditable div replaces textarea for Turbo engine only
- Tags render as inline styled badges
- Pasting text with [tag] patterns auto-converts to badges
- Badges serialize back to plain [tag] text for the API
- Dropdown portalled to body, opens above caret to avoid overflow
2026-03-13 05:19:23 -07:00
James Pine bfd7b815a5 fix: patch S3Tokenizer.log_mel_spectrogram for float64→float32 cast
The actual dtype mismatch was in S3Tokenizer.log_mel_spectrogram, not
VoiceEncoder.forward. librosa.load returns float64 numpy, which
torch.from_numpy preserves as double. The STFT output (double) then
hits _mel_filters (float32) in a matmul at s3tokenizer.py:163.

Now patching both entry points after model load:
1. S3Tokenizer.log_mel_spectrogram — cast audio to float32 before STFT
2. VoiceEncoder.forward — cast mels to float32 before LSTM

Remove debug traceback logging (no longer needed).
2026-03-13 05:04:29 -07:00
James Pine cac80f6af0 feat: add per-model unload endpoint and UI button
- POST /models/{model_name}/unload — unloads a specific model from
  memory without deleting from disk, supports all engine types
- Frontend: Unload button in model detail dialog when model is loaded
- Delete button remains disabled while loaded (unload first)
2026-03-13 04:50:56 -07:00
James Pine 47ce4cafdf fix: patch VoiceEncoder.forward to cast float64 mels to float32
The previous approach of patching librosa.load didn't work because
melspectrogram itself performs float64 math (numpy dot, signal.lfilter)
regardless of input dtype. The actual mismatch happens when pack()
creates a float64 tensor from the mel arrays and passes it into the
float32 LSTM weights in VoiceEncoder.forward().

Fix by monkey-patching VoiceEncoder.forward() to call mels.float()
before the LSTM, ensuring the input always matches the model dtype.
2026-03-13 04:41:43 -07:00
James Pine bfe912e41a fix: specify WAV format for atomic save temp file
soundfile cannot infer format from .tmp extension, causing all
generations to fail with 'No format specified and unable to get
format from file extension'
2026-03-13 04:34:26 -07:00
James Pine 5ccf79a8f7 Revert "fix: cast librosa float64 audio to float32 for Chatterbox voice encoder"
This reverts commit 1d32170c2e.
2026-03-13 04:28:00 -07:00
James Pine 1d32170c2e fix: cast librosa float64 audio to float32 for Chatterbox voice encoder
The upstream VoiceEncoder's melspectrogram only casts to float32 when
hp.normalized_mels is True (it defaults to False), so librosa's float64
output flows through as double tensors into float32 model weights,
causing 'expected m1 and m2 to have the same dtype, but got: float !=
double'. Fix by monkey-patching prepare_conditionals in both Chatterbox
and Chatterbox Turbo backends to ensure librosa.load returns float32.
2026-03-13 04:15:20 -07:00
James Pine ca74c155e2 fix: pass language parameter to Qwen TTS models and sync form with profile language
Both PyTorch and MLX backends silently dropped the language parameter —
it was accepted by generate() but never forwarded to the underlying
Qwen3-TTS model, causing it to default to auto-detection which
frequently confuses similar languages (e.g. Portuguese for Spanish).

- Add LANGUAGE_CODE_TO_NAME mapping (ISO 639-1 to full name) to both backends
- PyTorch: pass language= to generate_voice_clone()
- MLX: pass lang_code= to all 4 model.generate() call sites
- Frontend: auto-sync generation form language with selected voice profile

Closes #97
2026-03-13 04:04:04 -07:00
James Pine 1f770a157d fix: mismatched JSX closing tag in ModelManagement 2026-03-13 03:59:30 -07:00
Jamie PineandGitHub d64e24d422 Merge pull request #230 from haosenwang1018/docs/readme-grammar-profile-management
docs: fix minor README grammar in feature bullets
2026-03-13 03:56:55 -07:00
Jamie PineandGitHub 77d86ba835 Merge pull request #88 from Balneario-de-Cofrentes/fix/restrict-cors-origins
security: restrict CORS to known local origins
2026-03-13 03:56:15 -07:00
Jamie PineandGitHub 986a748420 Merge pull request #161 from ageofalgo/feat/docker-web-deployment
feat: add Docker + web deployment support
2026-03-13 03:55:04 -07:00
James Pine 50e01d17f8 fix: remove unused TTS_MODE env var from docker-compose
TTS_MODE is not read by any code in the backend — it only exists in
unimplemented planning docs. Remove it to avoid confusing users.
2026-03-13 03:53:15 -07:00
Jamie PineandGitHub 084c51b983 Merge pull request #215 from mikeswann/main
Update prerequisites in markdown with Tauri deps
2026-03-13 03:52:34 -07:00
Jamie PineandGitHub efbbbc7ec1 Merge branch 'main' into main 2026-03-13 03:52:22 -07:00
Jamie PineandGitHub 8e7f0cb9ad Merge pull request #133 from rayl15/feat/network-access-toggle
feat: add network access toggle to server settings
2026-03-13 03:47:35 -07:00
Jamie PineandGitHub 3357a06cba Merge pull request #263 from jamiepine/fix/atomic-save-error-handling
fix: atomic audio save with error handling and filesystem health endpoint
2026-03-13 03:45:26 -07:00
Jamie PineandGitHub f58c7c1cf3 Merge pull request #262 from jamiepine/feat/linux-rocm-whisper-turbo
feat: Linux support, AMD ROCm, Whisper Turbo, and spawn fix
2026-03-13 03:44:36 -07:00
James Pine ea41213123 fix: atomic audio save with errno-specific error handling and filesystem health endpoint
- save_audio() now writes to .tmp then os.replace() for atomic writes
- /generate endpoint catches OSError with specific messages for ENOENT, EACCES, ENOSPC, and BrokenPipeError
- New /health/filesystem endpoint checks directory existence, write permissions, and disk space
- New DirectoryCheck and FilesystemHealthResponse models

Cherry-picked and expanded from #178 (@Vaibhavee89)
2026-03-13 03:43:42 -07:00
James Pine b5801891b8 feat: Linux support, AMD ROCm, Whisper Turbo, and spawn fix
Cherry-picked and adapted from PR #89 and #214:

- Linux audio capture via PulseAudio/PipeWire monitor sources (cpal)
- AMD ROCm GPU support: HSA_OVERRIDE_GFX_VERSION env var, ROCm detection
- Whisper Turbo model (openai/whisper-large-v3-turbo) in all endpoints
- Cleaner Whisper language handling via generate_kwargs
- tauri::async_runtime::spawn fix to prevent panic on app shutdown
- Enable Linux (ubuntu-22.04) in release CI matrix
2026-03-13 03:35:18 -07:00
Jamie PineandGitHub 8f77c041f5 Merge pull request #152 from mpecanha/fix-offline-mode-crash
Fix: Prevent crashes when HuggingFace is unreachable
2026-03-13 03:31:23 -07:00
James Pine 5a3f3ba030 Merge remote-tracking branch 'origin/main' into feat/docker-web-deployment 2026-03-13 03:21:39 -07:00
Jamie PineandGitHub 3c25ee6e2c Merge pull request #243 from ways2read/a11y/screen-reader-and-keyboard-improvements
a11y: screen reader and keyboard improvements
2026-03-13 03:18:42 -07:00
James Pine b92b0dd508 merge: resolve conflicts with latest main 2026-03-13 03:16:56 -07:00
Jamie PineandGitHub 670900bf5a Merge pull request #258 from jamiepine/feat/chatterbox-turbo
feat: Chatterbox Turbo engine + per-engine language lists
2026-03-13 03:14:44 -07:00
James Pine 219cfb1605 docs: update PROJECT_STATUS.md to reflect multi-engine architecture
- Reflects merged PRs: #254 (LuxTTS/multi-engine), #257 (Chatterbox), #252 (CUDA swap), #238 (download UI)
- Updated architecture diagram to show all 4 TTS engines
- Added TTS engine comparison table and multi-engine architecture section
- Marked resolved bottlenecks (singleton backend, frontend Qwen assumptions)
- Updated PR triage: marked #194 and #33 as superseded
- Added 'Adding a New Engine' guide (now ~1 day effort)
- Updated recommended priorities to reflect current state
- Added new API endpoints (CUDA, cancel, active tasks)
2026-03-13 02:39:10 -07:00
James Pine bf728a780c feat: add Chatterbox Turbo engine and per-engine language lists
- New ChatterboxTurboTTSBackend wrapping ChatterboxTurboTTS (ResembleAI/chatterbox-turbo)
- English-only 350M model with paralinguistic tag support ([laugh], [cough], [chuckle])
- Bypasses upstream token=True bug by calling snapshot_download(token=None) + from_local()
- Same CPU-on-macOS forcing and torch.load monkey-patching as multilingual backend
- Full engine integration: generate, stream, model status/download/delete endpoints
- Language dropdown now shows only languages supported by the selected engine
- Per-engine language maps: Qwen (10), LuxTTS (en), Chatterbox (23), Turbo (en)
- Auto-switches to English when selecting English-only engines
- Backend language regex expanded to accept all 23 Chatterbox languages
2026-03-13 02:35:10 -07:00
Jamie PineandGitHub 3e6513c0fb Merge pull request #257 from jamiepine/feat/chatterbox
feat: Chatterbox TTS engine with multilingual voice cloning
2026-03-13 02:12:56 -07:00
James Pine c54ee14173 fix: model loaded icon uses accent-colored CircleCheck, show size for loaded models, fix generate box overlapping player on stories route 2026-03-13 02:09:32 -07:00
James Pine cc07d4d3c9 fix: download progress tracking for all engines and inline progress UI
- Add HFProgressTracker to LuxTTS and Chatterbox backends so tqdm-based
  file-level download progress reaches the frontend (previously only Qwen
  had this, LuxTTS/Chatterbox showed a static spinner)
- Add progress/current/total/filename fields to ActiveDownloadTask so the
  /tasks/active polling endpoint carries progress data
- Show inline progress bar + bytes in the model list and detail modal,
  poll at 1s during active downloads (5s otherwise)
- Fix GpuAcceleration crash: cudaStatusLoading was referenced before
  initialization in its own useQuery declaration
2026-03-13 02:09:32 -07:00
James Pine 9beb9d7fec fix: install chatterbox-tts with --no-deps to avoid numpy pin conflict
chatterbox-tts 0.1.6 pins numpy<1.26 and torch==2.6 which are
incompatible with Python 3.12+. Install with --no-deps and list
its sub-dependencies explicitly in requirements.txt.

Also removes HFProgressTracker from chatterbox backend to avoid
'generator didn't stop after throw()' errors from tqdm patching.
2026-03-13 02:09:32 -07:00
James Pine 76bb207b2b feat: add Chatterbox TTS engine for multilingual voice cloning
- New ChatterboxTTSBackend wrapping ChatterboxMultilingualTTS (ResembleAI/chatterbox)
- Supports 23 languages including Hebrew, forces CPU on macOS (MPS issue)
- Monkey-patches torch.load for CPU loading, forces eager attention for compatibility
- trim_tts_output utility cuts trailing silence/hallucination from Chatterbox output
- Full engine integration: /generate, /generate/stream, model status/download/delete
- Hebrew (he) added to supported languages in frontend and backend validation
- Single flat model dropdown extended with Chatterbox option in both generation UIs
- ModelManagement UI groups LuxTTS and Chatterbox under 'Other Voice Models' section
2026-03-13 02:09:32 -07:00
Jamie PineandGitHub 3576521d62 Merge pull request #254 from jamiepine/feat/luxtts
feat: LuxTTS integration — multi-engine TTS support
2026-03-13 02:04:46 -07:00
Jamie PineandGitHub 2df4ece388 Merge pull request #210 from ieguiguren/fix/linux-nvidia-gbm-buffer
fix: Linux NVIDIA GBM buffer crash + WebKitGTK microphone access
2026-03-13 01:55:58 -07:00
Jamie PineandGitHub cbb4979ed6 Merge pull request #175 from Vaibhavee89/fix/profile-duplicate-name-validation
Fix #134: Add validation for duplicate profile names
2026-03-13 01:55:30 -07:00
Jamie PineandGitHub 573f82a7e6 Merge pull request #250 from pandego/fix/docs-align-local-port-17493
docs: align local API port examples with current dev flow
2026-03-13 01:53:28 -07:00
pandego 3d2506767d docs: address review nits for API generator 2026-03-13 05:08:25 +01:00
pandego cdef2163c1 docs: align local API port examples with current dev flow 2026-03-12 12:17:50 +01:00
Richard Orme 9955e1dcb7 a11y: address PR feedback and polish docs
- HistoryTable: skip row key handler when focus is on Actions button (Enter/Space)
- StoryList: expose selected story (aria-pressed, 'Selected' in label)
- ProfileCard: skip card key handler when focus is on Export/Edit/Delete
- VoicesTab: keep table semantics; edit button in first cell instead of role=button on row
- PR-ACCESSIBILITY.md: 'Fine-tune' wording, 'focus on the text area' phrasing

Made-with: Cursor
2026-03-07 12:36:02 -08:00
Richard Orme 19a28bf6c5 a11y: screen reader and keyboard improvements
- Audio player: aria-labels for Play/Pause, Loop, Mute, Close; labelled playback and volume sliders
- Generation: aria-labels for Generate speech and Fine tune instructions buttons
- Voice cards: focusable, labelled, Enter/Space to select
- History rows: focusable, labelled, Enter/Space to play; transcript textarea labelled
- Voices tab: focusable rows, labelled, Enter/Space to edit; Actions button labelled
- Model management: focusable model rows and labelled Download/Delete buttons
- Server tab: regions with aria-label and tabIndex for Connection, Status, App Updates
- Stories: focusable story rows, labelled, Enter/Space to select; Actions and track editor buttons labelled
- Voice profile samples: Play/Pause/Stop and mini-player slider labelled

Tested with NVDA and Narrator on Windows. See docs/PR-ACCESSIBILITY.md for full description.

Made-with: Cursor
2026-03-07 12:02:33 -08:00
OpenClaw Bot 3f10a70d4c docs: fix minor grammar in feature bullets 2026-03-04 04:39:28 +00:00
mikeswannGitHubcoderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
d0dfe78701 Update CONTRIBUTING.md
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
2026-02-28 10:36:34 +01:00
mikeswannandGitHub 172addd918 Update README.md 2026-02-28 00:29:23 +01:00
mikeswannandGitHub ada309cfb9 Update CONTRIBUTING.md 2026-02-28 00:28:01 +01:00
IvanandClaude Opus 4.6 30ee07c2e3 fix: scope DMABUF workaround to Linux+NVIDIA, add origin validation
Address CodeRabbit review feedback:
- Makefile: only set WEBKIT_DISABLE_DMABUF_RENDERER=1 when running on
  Linux with an NVIDIA GPU detected via lspci
- main.rs: validate webview origin before auto-granting microphone
  permission — only allow for trusted local origins (tauri://, localhost,
  127.0.0.1)

Co-Authored-By: Claude Opus 4.6 <[email protected]>
2026-02-27 07:01:32 +01:00
IvanandClaude Opus 4.6 d21c63b52c fix: enable microphone access on Linux via WebKitGTK
WebKitGTK denies getUserMedia by default. This adds webkit2gtk as a
Linux dependency and configures the webview to enable media streams
and auto-grant UserMediaPermissionRequest for microphone access.

Co-Authored-By: Claude Opus 4.6 <[email protected]>
2026-02-27 06:49:08 +01:00
IvanandClaude Opus 4.6 5ad67d7ecb fix: disable DMABUF renderer for NVIDIA GPUs on Linux
WebKitGTK fails to create GBM buffers with NVIDIA proprietary drivers,
resulting in an empty/blank Tauri window. Set WEBKIT_DISABLE_DMABUF_RENDERER=1
in the dev target to work around this.

Co-Authored-By: Claude Opus 4.6 <[email protected]>
2026-02-27 06:32:02 +01:00
Vaibhavee Singh 6cc96c2614 Fix #134: Add validation for duplicate profile names
- Add validation in create_profile() to check for existing names before insert
- Add validation in update_profile() to prevent renaming to duplicate names
- Improve error handling in API endpoints with user-friendly messages
- Add comprehensive test suite for duplicate name validation
- Update CHANGELOG.md with fix details

This fix prevents database constraint violations and provides clear
error messages when users attempt to create or update profiles with
names that already exist in the database.
2026-02-24 10:17:39 +05:30
Claudio Casale edfc6e99fe feat: add Docker + web deployment support 2026-02-23 12:52:03 +01:00
Makinde d00e28ffda Fix: Prevent crashes when HuggingFace is unreachable
Implements offline mode patch for API stability issues:

- Add hf_offline_patch.py to monkey-patch huggingface_hub
- Force cache-only lookups before mlx_audio imports
- Create symlink from original Qwen repo to MLX community version
  when only MLX version is cached

This fixes:
- Issue #150: Internet required even with cached models
- Issue #151: API crashes when HF network fails

The patch ensures that if models are locally cached, no network
requests are made to HuggingFace during speech generation.
2026-02-22 01:57:02 -08:00
Rahul Sharma 28a4fd4824 feat: add network access toggle to server settings
Exposes the existing remote server mode through a checkbox in Server
Connection settings. When enabled, the server binds to 0.0.0.0 instead
of 127.0.0.1, making it accessible from other devices on the network.

The plumbing already existed (Rust sidecar passes --host 0.0.0.0 when
remote=true, serverStore has mode state, Python backend accepts --host),
but the UI hardcoded startServer(false). This wires it up.

Closes #104
2026-02-21 00:39:39 +05:30
David Gil 80c87c8e2c test: add CORS origin restriction tests
20 tests covering:
- All 6 default local origins are allowed
- Arbitrary external origins are blocked
- Preflight (OPTIONS) requests respect the allowlist
- VOICEBOX_CORS_ORIGINS env var extends the allowlist
- Edge cases: empty env, whitespace trimming, trailing commas

Tests use a minimal FastAPI app mirroring the real CORS config,
so they run without ML dependencies (torch, numpy, etc.).
2026-02-17 22:04:25 +01:00
David Gil 427d811954 security: restrict CORS to known local origins instead of wildcard
The wildcard `allow_origins=["*"]` allows any website the user visits to
make requests to the local voicebox backend, potentially triggering TTS
generation or reading voice profiles without consent.

Restrict to the known Tauri webview and Vite dev server origins by
default. Users running in remote server mode can set
VOICEBOX_CORS_ORIGINS to allow additional origins.
2026-02-17 21:58:08 +01:00
117 changed files with 11022 additions and 1728 deletions
+1 -1
View File
@@ -1,5 +1,5 @@
[bumpversion]
current_version = 0.1.13
current_version = 0.2.0
commit = True
tag = True
tag_name = v{new_version}
+46
View File
@@ -0,0 +1,46 @@
# Version control
.git
.github
.gitignore
# Desktop-only (not needed in web container)
tauri/
landing/
docs/
mlx-test/
scripts/
# Dependencies & build artifacts (rebuilt in Docker)
node_modules/
__pycache__/
*.pyc
*.pyo
*.egg-info/
dist/
build/
*.spec
# Data (will be bind-mounted)
data/
backend/data/
# IDE & OS
.vscode/
.idea/
*.swp
*.swo
.DS_Store
Thumbs.db
# Config files not needed in container
biome.json
.biomeignore
.bumpversion.cfg
.npmrc
Makefile
CHANGELOG.md
CONTRIBUTING.md
SECURITY.md
LICENSE
README.md
backend/README.md
-73
View File
@@ -1,73 +0,0 @@
name: Build CUDA Backend
on:
workflow_dispatch:
push:
tags:
- "v*"
jobs:
build-cuda-windows:
runs-on: windows-latest
permissions:
contents: write
steps:
- uses: actions/checkout@v4
- name: Setup Python
uses: actions/setup-python@v5
with:
python-version: "3.12"
cache: "pip"
- name: Install Python dependencies
run: |
python -m pip install --upgrade pip
pip install pyinstaller
pip install -r backend/requirements.txt
- name: Install PyTorch with CUDA 12.1
run: |
pip install torch --index-url https://download.pytorch.org/whl/cu121 --force-reinstall --no-deps
pip install torchaudio --index-url https://download.pytorch.org/whl/cu121
- name: Verify CUDA support in torch
run: |
python -c "import torch; print(f'CUDA available in build: {torch.cuda.is_available()}'); print(f'CUDA version: {torch.version.cuda}')"
- name: Build CUDA server binary
shell: bash
working-directory: backend
run: python build_binary.py --cuda
- name: Split binary for GitHub Releases
shell: bash
run: |
python scripts/split_binary.py \
backend/dist/voicebox-server-cuda.exe \
--output release-assets/
- name: Upload split parts to GitHub Release
if: startsWith(github.ref, 'refs/tags/')
uses: softprops/action-gh-release@v1
with:
files: |
release-assets/voicebox-server-cuda.part*.exe
release-assets/voicebox-server-cuda.sha256
release-assets/voicebox-server-cuda.manifest
draft: true
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: Upload binary as workflow artifact (for testing)
uses: actions/upload-artifact@v4
with:
name: voicebox-server-cuda-windows
path: backend/dist/voicebox-server-cuda.exe
retention-days: 7
# Linux CUDA build can be added later with:
# build-cuda-linux:
# runs-on: ubuntu-22.04
# ...
+68 -13
View File
@@ -22,10 +22,6 @@ jobs:
args: "--target x86_64-apple-darwin"
python-version: "3.12"
backend: "pytorch"
# - platform: 'ubuntu-22.04'
# args: ''
# python-version: '3.12'
# backend: 'pytorch'
- platform: "windows-latest"
args: ""
python-version: "3.12"
@@ -37,10 +33,10 @@ jobs:
- uses: actions/checkout@v4
- name: Install dependencies (ubuntu only)
if: matrix.platform == 'ubuntu-22.04'
if: contains(matrix.platform, 'ubuntu') || contains(matrix.platform, 'namespace')
run: |
sudo apt-get update
sudo apt-get install -y libwebkit2gtk-4.1-dev libappindicator3-dev librsvg2-dev patchelf llvm-dev
sudo apt-get install -y libwebkit2gtk-4.1-dev libappindicator3-dev librsvg2-dev patchelf llvm-dev libasound2-dev
- name: Install LLVM (macOS)
if: matrix.platform == 'macos-latest' || matrix.platform == 'macos-15-intel'
@@ -55,6 +51,11 @@ jobs:
python-version: ${{ matrix.python-version }}
cache: "pip"
- name: Install CPU-only PyTorch (Linux)
if: contains(matrix.platform, 'ubuntu') || contains(matrix.platform, 'namespace')
run: |
pip install torch torchaudio --index-url https://download.pytorch.org/whl/cpu
- name: Install Python dependencies
run: |
python -m pip install --upgrade pip
@@ -66,12 +67,6 @@ jobs:
run: |
pip install -r backend/requirements-mlx.txt
# - name: Install PyTorch with CUDA (Windows only)
# if: matrix.platform == 'windows-latest'
# run: |
# pip install torch --index-url https://download.pytorch.org/whl/cu121 --force-reinstall --no-deps
# pip install torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121
- name: Build Python server (Linux/macOS)
if: matrix.platform != 'windows-latest'
run: |
@@ -151,10 +146,70 @@ jobs:
- **macOS (Apple Silicon)**: Download the `aarch64.dmg` file - uses MLX for fast native inference
- **macOS (Intel)**: Download the `x64.dmg` file - uses PyTorch
- **Windows**: Download the `.msi` installer
- **Linux**: Download the `.AppImage` or `.deb` package
- **Linux**: Compile from source (see README)
The app includes automatic updates - future updates will be installed automatically.
releaseDraft: true
prerelease: false
args: ${{ matrix.args }}
includeUpdaterJson: true
build-cuda-windows:
runs-on: windows-latest
permissions:
contents: write
steps:
- uses: actions/checkout@v4
- name: Setup Python
uses: actions/setup-python@v5
with:
python-version: "3.12"
cache: "pip"
- name: Install Python dependencies
run: |
python -m pip install --upgrade pip
pip install pyinstaller
pip install -r backend/requirements.txt
- name: Install PyTorch with CUDA 12.1
run: |
pip install torch --index-url https://download.pytorch.org/whl/cu121 --force-reinstall --no-deps
pip install torchaudio --index-url https://download.pytorch.org/whl/cu121
- name: Verify CUDA support in torch
run: |
python -c "import torch; print(f'CUDA available in build: {torch.cuda.is_available()}'); print(f'CUDA version: {torch.version.cuda}')"
- name: Build CUDA server binary
shell: bash
working-directory: backend
run: python build_binary.py --cuda
- name: Split binary for GitHub Releases
shell: bash
run: |
python scripts/split_binary.py \
backend/dist/voicebox-server-cuda.exe \
--output release-assets/
- name: Upload split parts to GitHub Release
if: startsWith(github.ref, 'refs/tags/')
uses: softprops/action-gh-release@v1
with:
files: |
release-assets/voicebox-server-cuda.part*.exe
release-assets/voicebox-server-cuda.sha256
release-assets/voicebox-server-cuda.manifest
draft: true
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: Upload binary as workflow artifact
uses: actions/upload-artifact@v4
with:
name: voicebox-server-cuda-windows
path: backend/dist/voicebox-server-cuda.exe
retention-days: 7
+1 -4
View File
@@ -35,10 +35,7 @@ target/
Thumbs.db
# Data (user-generated)
data/profiles/*
data/generations/*
data/projects/*
data/voicebox.db
data/
!data/.gitkeep
# Logs
+9
View File
@@ -5,6 +5,14 @@ All notable changes to Voicebox will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [Unreleased]
### Fixed
- **Profile Name Validation** - Added proper validation to prevent duplicate profile names ([#134](https://github.com/jamiepine/voicebox/issues/134))
- Users now receive clear error messages when attempting to create or update profiles with duplicate names
- Improved error handling in create and update profile API endpoints
- Added comprehensive test suite for duplicate name validation
## [0.1.0] - 2026-01-25
### Added
@@ -55,6 +63,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Fixed
- Audio export failing when Tauri save dialog returns object instead of string path
- OpenAPI client generator script now documents the local backend port and avoids an unused loop variable warning
### Added
- **Makefile** - Comprehensive development workflow automation with commands for setup, development, building, testing, and code quality checks
+2 -1
View File
@@ -27,6 +27,7 @@ Thank you for your interest in contributing to Voicebox! This document provides
```bash
rustc --version # Check if installed
```
- **[Tauri Prerequisites](https://v2.tauri.app/start/prerequisites)** - Tauri-specific system dependencies (varies by OS).
- **Git** - Version control
@@ -426,7 +427,7 @@ See [docs/TROUBLESHOOTING.md](docs/TROUBLESHOOTING.md) for common issues and sol
- **Backend won't start:** Check Python version (3.11+), ensure venv is activated, install dependencies
- **Tauri build fails:** Ensure Rust is installed, clean build with `cd tauri/src-tauri && cargo clean`
- **OpenAPI client generation fails:** Ensure backend is running, check `curl http://localhost:8000/openapi.json`
- **OpenAPI client generation fails:** Ensure backend is running, check `curl http://localhost:17493/openapi.json`
## Questions?
+79
View File
@@ -0,0 +1,79 @@
# ============================================================
# Voicebox — Local TTS Server with Web UI (CPU)
# 3-stage build: Frontend → Python deps → Runtime
# ============================================================
# === Stage 1: Build frontend ===
FROM oven/bun:1 AS frontend
WORKDIR /build
# Copy workspace config and frontend source
COPY package.json bun.lock ./
COPY app/ ./app/
COPY web/ ./web/
# Strip workspaces not needed for web build, and fix trailing comma
RUN sed -i '/"tauri"/d; /"landing"/d' package.json && \
sed -i -z 's/,\n ]/\n ]/' package.json
RUN bun install --no-save
# Build frontend (skip tsc — upstream has pre-existing type errors)
RUN cd web && bunx --bun vite build
# === Stage 2: Build Python dependencies ===
FROM python:3.11-slim AS backend-builder
WORKDIR /build
RUN apt-get update && apt-get install -y --no-install-recommends \
git \
build-essential \
&& rm -rf /var/lib/apt/lists/*
COPY backend/requirements.txt .
RUN pip install --no-cache-dir --prefix=/install -r requirements.txt
RUN pip install --no-cache-dir --prefix=/install \
git+https://github.com/QwenLM/Qwen3-TTS.git
# === Stage 3: Runtime ===
FROM python:3.11-slim
# Create non-root user for security
RUN groupadd -r voicebox && \
useradd -r -g voicebox -m -s /bin/bash voicebox
WORKDIR /app
# Install only runtime system dependencies
RUN apt-get update && apt-get install -y --no-install-recommends \
ffmpeg \
curl \
&& rm -rf /var/lib/apt/lists/*
# Copy installed Python packages from builder stage
COPY --from=backend-builder /install /usr/local
# Copy backend application code
COPY --chown=voicebox:voicebox backend/ /app/backend/
# Copy built frontend from frontend stage
COPY --from=frontend --chown=voicebox:voicebox /build/web/dist /app/frontend/
# Create data directories owned by non-root user
RUN mkdir -p /app/data/generations /app/data/profiles /app/data/cache \
&& chown -R voicebox:voicebox /app/data
# Switch to non-root user
USER voicebox
# Expose the API port
EXPOSE 17493
# Health check — auto-restart if the server hangs
HEALTHCHECK --interval=30s --timeout=10s --retries=3 --start-period=60s \
CMD curl -f http://localhost:17493/health || exit 1
# Start the FastAPI server
CMD ["uvicorn", "backend.main:app", "--host", "0.0.0.0", "--port", "17493"]
+6 -1
View File
@@ -48,6 +48,7 @@ setup-python: $(VENV)/bin/activate ## Set up Python virtual environment and depe
@echo -e "$(BLUE)Installing Python dependencies...$(NC)"
$(PIP) install --upgrade pip
$(PIP) install -r $(BACKEND_DIR)/requirements.txt
$(PIP) install --no-deps chatterbox-tts
@if [ "$$(uname -m)" = "arm64" ] && [ "$$(uname)" = "Darwin" ]; then \
echo -e "$(BLUE)Detected Apple Silicon - installing MLX dependencies...$(NC)"; \
$(PIP) install -r $(BACKEND_DIR)/requirements-mlx.txt; \
@@ -79,7 +80,11 @@ dev: ## Start backend + desktop app (parallel)
@echo -e "$(YELLOW)Note: If Tauri fails, run 'make build-server' first or use separate terminals$(NC)"
@trap 'kill 0' EXIT; \
$(MAKE) dev-backend & \
sleep 2 && $(MAKE) dev-frontend & \
sleep 2 && if [ "$$(uname)" = "Linux" ] && lspci 2>/dev/null | grep -qi nvidia; then \
WEBKIT_DISABLE_DMABUF_RENDERER=1 $(MAKE) dev-frontend; \
else \
$(MAKE) dev-frontend; \
fi & \
wait
dev-backend: ## Start FastAPI backend server
+58
View File
@@ -0,0 +1,58 @@
# Voicebox Offline Mode Fix
## Problem
Voicebox crashes when generating speech if HuggingFace is unreachable, even when models are fully cached locally.
**Root Cause:**
- Voicebox downloads `mlx-community/Qwen3-TTS-12Hz-1.7B-Base-bf16` (MLX optimized version)
- But `mlx_audio.tts.load()` tries to fetch `config.json` from original repo `Qwen/Qwen3-TTS-12Hz-1.7B-Base`
- This network request fails → server crashes with `RemoteDisconnected`
**Related Issues:**
- Issue #150: "Internet connection required, even though models are downloaded?"
- Issue #151: "API Stability Issues: Model Loading Hangs and Server Crashes"
## Solution
Two-part fix:
### 1. Monkey-patch huggingface_hub (`backend/utils/hf_offline_patch.py`)
- Intercepts cache lookup functions
- Forces offline mode early (before mlx_audio imports)
- Adds debug logging for cache hits/misses
### 2. Symlink original repo to MLX version (`ensure_original_qwen_config_cached()`)
- When original `Qwen/Qwen3-TTS-12Hz-1.7B-Base` cache doesn't exist
- But MLX `mlx-community/Qwen3-TTS-12Hz-1.7B-Base-bf16` does exist
- Creates a symlink so cache lookups succeed
## Files Changed
- `backend/backends/mlx_backend.py` - Added patch imports at top
- `backend/utils/hf_offline_patch.py` - New patch module
## Testing
To test this fix:
1. Build Voicebox from source: `make build`
2. Disconnect from internet
3. Try generating speech
4. Should work without network requests
## Build Instructions
```bash
# Install dependencies
pip install -r requirements.txt
# Build the app
make build
# Or build just the server
make build-server
```
## Notes
- The patch is applied automatically when `mlx_backend.py` is imported
- Set `VOICEBOX_OFFLINE_PATCH=0` to disable the patch
- The symlink approach works because the config.json is compatible between versions
---
*Patch contributed by community*
+11 -8
View File
@@ -85,7 +85,7 @@ Voicebox is available now for macOS and Windows.
| Windows (MSI) | [Latest Windows MSI](https://github.com/jamiepine/voicebox/releases/latest) |
| Windows (Setup) | [Latest Windows Setup](https://github.com/jamiepine/voicebox/releases/latest) |
> **Linux builds coming soon** — Currently blocked by GitHub runner disk space limitations.
> **Linux** — Pre-built binaries are not yet available. Linux users can compile from source, see [Development](#development) below.
---
@@ -98,12 +98,12 @@ Powered by Alibaba's **Qwen3-TTS** — a breakthrough model that achieves near-p
- **Instant cloning** — Upload a sample, get a voice profile
- **High fidelity** — Natural prosody, emotion, and cadence
- **Multi-language** — English, Chinese, and more coming
- **Lightning fast on Mac** — MLX backend leverages Apple Silicon's Neural Engine for super fast generation
- **Lightning fast on Mac** — MLX backend leverages Apple Silicon's Neural Engine for super-fast generation
### Voice Profile Management
- **Create profiles** from audio files or record directly in-app
- **Import/Export** profiles to share or backup
- **Import/Export** profiles to share or back up
- **Multi-sample support** — combine multiple samples for higher quality cloning
- **Organize** with descriptions and language tags
@@ -147,17 +147,20 @@ Create multi-voice narratives, podcasts, and conversations with a timeline-based
Voicebox exposes a full REST API, so you can integrate voice synthesis into your own apps.
For the current local app and development workflow, the backend is typically available at `http://localhost:17493`.
If you launch the backend manually with a different host or port, use that address instead.
```bash
# Generate speech
curl -X POST http://localhost:8000/generate \
curl -X POST http://localhost:17493/generate \
-H "Content-Type: application/json" \
-d '{"text": "Hello world", "profile_id": "abc123", "language": "en"}'
# List voice profiles
curl http://localhost:8000/profiles
curl http://localhost:17493/profiles
# Create a profile
curl -X POST http://localhost:8000/profiles \
curl -X POST http://localhost:17493/profiles \
-H "Content-Type: application/json" \
-d '{"name": "My Voice", "language": "en"}'
```
@@ -170,7 +173,7 @@ curl -X POST http://localhost:8000/profiles \
- Voice assistants
- Content creation automation
Full API documentation available at `http://localhost:8000/docs` when running.
Full API documentation is available at `http://localhost:17493/docs` in the default local workflow, or at `/docs` on whatever server address you configured.
---
@@ -239,7 +242,7 @@ Install [just](https://github.com/casey/just): `brew install just` or `cargo ins
Also available via Makefile: `make setup && make dev` (run `make help` for all commands).
**Prerequisites:** [Bun](https://bun.sh), [Rust](https://rustup.rs), [Python 3.11+](https://python.org), [XCode on macOS](https://developer.apple.com/xcode/).
**Prerequisites:** [Bun](https://bun.sh), [Rust](https://rustup.rs), [Python 3.11+](https://python.org), [XCode on macOS](https://developer.apple.com/xcode/), [Tauri Prerequisites](https://v2.tauri.app/start/prerequisites/).
**Performance:**
- **Apple Silicon (M1/M2/M3)**: Uses MLX backend with native Metal acceleration for 4-5x faster inference
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@voicebox/app",
"version": "0.1.13",
"version": "0.2.0",
"private": true,
"type": "module",
"scripts": {
+4 -2
View File
@@ -93,10 +93,12 @@ function App() {
}
serverStartingRef.current = true;
console.log('Production mode: Starting bundled server...');
const isRemote = useServerStore.getState().mode === 'remote';
const customModelsDir = useServerStore.getState().customModelsDir;
console.log(`Production mode: Starting bundled server... (remote: ${isRemote})`);
platform.lifecycle
.startServer(false)
.startServer(isRemote, customModelsDir)
.then((serverUrl) => {
console.log('Server is ready at:', serverUrl);
// Update the server URL in the store with the dynamically assigned port
+40 -7
View File
@@ -1,17 +1,18 @@
import { useQuery } from '@tanstack/react-query';
import { Pause, Play, Repeat, Volume2, VolumeX, X } from 'lucide-react';
import { useEffect, useMemo, useRef, useState } from 'react';
import { useEffect, useId, useMemo, useRef, useState } from 'react';
import WaveSurfer from 'wavesurfer.js';
import { Button } from '@/components/ui/button';
import { Slider } from '@/components/ui/slider';
import { apiClient } from '@/lib/api/client';
import { formatAudioDuration } from '@/lib/utils/audio';
import { debug } from '@/lib/utils/debug';
import { usePlayerStore } from '@/stores/playerStore';
import { usePlatform } from '@/platform/PlatformContext';
import { usePlayerStore } from '@/stores/playerStore';
export function AudioPlayer() {
const platform = usePlatform();
const volumeLabelId = useId();
const {
audioUrl,
audioId,
@@ -156,8 +157,21 @@ export function AudioPlayer() {
const wavesurfer = wavesurferRef.current;
if (!wavesurfer) return;
// Update store when time changes
// Update store when time changes, stop if past duration
wavesurfer.on('timeupdate', (time) => {
const dur = usePlayerStore.getState().duration;
if (dur > 0 && time >= dur) {
setCurrentTime(dur);
const loop = usePlayerStore.getState().isLooping;
if (loop) {
wavesurfer.seekTo(0);
wavesurfer.play();
} else {
wavesurfer.pause();
setIsPlaying(false);
}
return;
}
setCurrentTime(time);
});
@@ -359,7 +373,7 @@ export function AudioPlayer() {
if (shouldAutoPlayNow) {
// Clear the flag first
usePlayerStore.getState().clearAutoPlayFlag();
// Use a small delay to ensure audio element is fully ready
setTimeout(() => {
wavesurfer.play().catch((error) => {
@@ -664,7 +678,7 @@ export function AudioPlayer() {
// Handle shouldAutoPlay flag - for story mode auto-advance
const shouldAutoPlay = usePlayerStore((state) => state.shouldAutoPlay);
const clearAutoPlayFlag = usePlayerStore((state) => state.clearAutoPlayFlag);
useEffect(() => {
const wavesurfer = wavesurferRef.current;
if (!wavesurfer || !shouldAutoPlay || duration === 0) {
@@ -831,6 +845,9 @@ export function AudioPlayer() {
disabled={isLoading || duration === 0}
className="shrink-0"
title={duration === 0 && !isLoading ? 'Audio not loaded' : ''}
aria-label={
duration === 0 && !isLoading ? 'Audio not loaded' : isPlaying ? 'Pause' : 'Play'
}
>
{isPlaying ? <Pause className="h-5 w-5" /> : <Play className="h-5 w-5" />}
</Button>
@@ -845,6 +862,8 @@ export function AudioPlayer() {
max={100}
step={0.1}
className="w-full"
aria-label="Playback position"
aria-valuetext={`${formatAudioDuration(currentTime)} of ${formatAudioDuration(duration)}`}
/>
)}
{isLoading && (
@@ -862,7 +881,9 @@ export function AudioPlayer() {
{/* Title */}
{title && (
<div className="text-sm font-medium truncate max-w-[200px] shrink-0">{title}</div>
<div className="text-sm font-medium truncate max-w-[200px] shrink-0 hidden lg:block">
{title}
</div>
)}
{/* Loop Button */}
@@ -872,26 +893,37 @@ export function AudioPlayer() {
onClick={toggleLoop}
className={isLooping ? 'text-primary' : ''}
title="Toggle loop"
aria-label={isLooping ? 'Stop looping' : 'Loop'}
>
<Repeat className="h-4 w-4" />
</Button>
{/* Volume Control */}
<div className="flex items-center gap-2 shrink-0 w-[120px]">
<div
className="flex items-center gap-2 shrink-0 w-[120px]"
role="group"
aria-label="Volume"
>
<Button
variant="ghost"
size="icon"
onClick={() => setVolume(volume > 0 ? 0 : 1)}
className="h-8 w-8"
aria-label={volume > 0 ? 'Mute' : 'Unmute'}
>
{volume > 0 ? <Volume2 className="h-4 w-4" /> : <VolumeX className="h-4 w-4" />}
</Button>
<span id={volumeLabelId} className="sr-only">
Volume level, {Math.round(volume * 100)}%
</span>
<Slider
value={[volume * 100]}
onValueChange={handleVolumeChange}
max={100}
step={1}
className="flex-1"
aria-labelledby={volumeLabelId}
aria-valuetext={`${Math.round(volume * 100)}%`}
/>
</div>
@@ -902,6 +934,7 @@ export function AudioPlayer() {
onClick={handleClose}
className="shrink-0"
title="Close player"
aria-label="Close player"
>
<X className="h-5 w-5" />
</Button>
+6 -4
View File
@@ -23,8 +23,8 @@ import {
import { apiClient } from '@/lib/api/client';
import { BOTTOM_SAFE_AREA_PADDING } from '@/lib/constants/ui';
import { cn } from '@/lib/utils/cn';
import { usePlayerStore } from '@/stores/playerStore';
import { usePlatform } from '@/platform/PlatformContext';
import { usePlayerStore } from '@/stores/playerStore';
interface AudioDevice {
id: string;
@@ -129,7 +129,7 @@ export function AudioTab() {
if (await confirm('Delete this channel?')) {
deleteChannel.mutate(channelId);
}
}
};
const allChannels = channels || [];
const allDevices = devices || [];
@@ -168,7 +168,7 @@ export function AudioTab() {
</Button>
</div>
) : (
<div className="space-y-3 p-2">
<div className="space-y-3">
{allChannels.map((channel) => {
const isSelected = selectedChannelId === channel.id;
return (
@@ -343,7 +343,9 @@ export function AudioTab() {
<div className="flex flex-col items-center justify-center py-12 border-2 border-dashed border-muted rounded-md">
<CheckCircle2 className="h-12 w-12 text-muted-foreground mb-4" />
<p className="text-muted-foreground text-center">
{platform.metadata.isTauri ? 'No audio devices found' : 'Audio device selection requires Tauri'}
{platform.metadata.isTauri
? 'No audio devices found'
: 'Audio device selection requires Tauri'}
</p>
</div>
)}
@@ -0,0 +1,377 @@
import {
closestCenter,
DndContext,
type DragEndEvent,
KeyboardSensor,
PointerSensor,
useSensor,
useSensors,
} from '@dnd-kit/core';
import {
arrayMove,
SortableContext,
sortableKeyboardCoordinates,
useSortable,
verticalListSortingStrategy,
} from '@dnd-kit/sortable';
import { CSS } from '@dnd-kit/utilities';
import { useQuery } from '@tanstack/react-query';
import { ChevronDown, ChevronRight, GripVertical, Plus, Power, Trash2 } from 'lucide-react';
import { useCallback, useMemo, useRef, useState } from 'react';
import { Button } from '@/components/ui/button';
import { Label } from '@/components/ui/label';
import {
Select,
SelectContent,
SelectItem,
SelectTrigger,
SelectValue,
} from '@/components/ui/select';
import { Slider } from '@/components/ui/slider';
import { apiClient } from '@/lib/api/client';
import type { AvailableEffect, EffectConfig, EffectPresetResponse } from '@/lib/api/types';
import { cn } from '@/lib/utils/cn';
// Each effect in the chain gets a stable ID for dnd-kit
interface EffectWithId extends EffectConfig {
_id: string;
}
let nextId = 0;
function makeId() {
return `fx-${++nextId}`;
}
interface EffectsChainEditorProps {
value: EffectConfig[];
onChange: (chain: EffectConfig[]) => void;
compact?: boolean;
showPresets?: boolean;
}
export function EffectsChainEditor({
value,
onChange,
compact = false,
showPresets = true,
}: EffectsChainEditorProps) {
const [expandedId, setExpandedId] = useState<string | null>(null);
// Maintain stable IDs for each effect across renders.
// We use a ref to map value items to IDs, rebuilding when length changes.
const idsRef = useRef<string[]>([]);
const items: EffectWithId[] = useMemo(() => {
// Grow ID array if effects were added
while (idsRef.current.length < value.length) {
idsRef.current.push(makeId());
}
// Shrink if effects were removed
if (idsRef.current.length > value.length) {
idsRef.current = idsRef.current.slice(0, value.length);
}
return value.map((e, i) => ({ ...e, _id: idsRef.current[i] }));
}, [value]);
const sensors = useSensors(
useSensor(PointerSensor, { activationConstraint: { distance: 5 } }),
useSensor(KeyboardSensor, { coordinateGetter: sortableKeyboardCoordinates }),
);
const { data: availableEffects } = useQuery({
queryKey: ['available-effects'],
queryFn: () => apiClient.getAvailableEffects(),
staleTime: Infinity,
});
const { data: presets } = useQuery({
queryKey: ['effect-presets'],
queryFn: () => apiClient.listEffectPresets(),
staleTime: 30_000,
});
const effectsMap = useMemo(() => {
const m = new Map<string, AvailableEffect>();
if (availableEffects) {
for (const e of availableEffects.effects) {
m.set(e.type, e);
}
}
return m;
}, [availableEffects]);
function addEffect(type: string) {
const def = effectsMap.get(type);
if (!def) return;
const params: Record<string, number> = {};
for (const [key, p] of Object.entries(def.params)) {
params[key] = p.default;
}
const newEffect: EffectConfig = { type, enabled: true, params };
const newId = makeId();
idsRef.current = [...idsRef.current, newId];
onChange([...value, newEffect]);
setExpandedId(newId);
}
const removeEffect = useCallback(
(index: number) => {
const removedId = idsRef.current[index];
idsRef.current = idsRef.current.filter((_, i) => i !== index);
onChange(value.filter((_, i) => i !== index));
if (expandedId === removedId) setExpandedId(null);
},
[value, onChange, expandedId],
);
const toggleEnabled = useCallback(
(index: number) => {
onChange(value.map((e, i) => (i === index ? { ...e, enabled: !e.enabled } : e)));
},
[value, onChange],
);
const updateParam = useCallback(
(index: number, paramName: string, paramValue: number) => {
onChange(
value.map((e, i) =>
i === index ? { ...e, params: { ...e.params, [paramName]: paramValue } } : e,
),
);
},
[value, onChange],
);
function loadPreset(preset: EffectPresetResponse) {
idsRef.current = preset.effects_chain.map(() => makeId());
onChange(preset.effects_chain);
setExpandedId(null);
}
function clearAll() {
idsRef.current = [];
onChange([]);
setExpandedId(null);
}
function handleDragEnd(event: DragEndEvent) {
const { active, over } = event;
if (!over || active.id === over.id) return;
const oldIndex = idsRef.current.indexOf(active.id as string);
const newIndex = idsRef.current.indexOf(over.id as string);
if (oldIndex === -1 || newIndex === -1) return;
idsRef.current = arrayMove(idsRef.current, oldIndex, newIndex);
onChange(arrayMove([...value], oldIndex, newIndex));
}
return (
<div className={cn('space-y-2', compact && 'text-sm')}>
{/* Preset selector row */}
{showPresets && (
<div className="flex items-center gap-2">
<Select
onValueChange={(id) => {
const preset = presets?.find((p) => p.id === id);
if (preset) loadPreset(preset);
}}
>
<SelectTrigger className="h-8 flex-1 text-xs focus:ring-0 focus:ring-offset-0">
<SelectValue placeholder="Load preset..." />
</SelectTrigger>
<SelectContent>
{presets?.map((p) => (
<SelectItem key={p.id} value={p.id}>
{p.name}
{p.description && (
<span className="ml-1 text-muted-foreground">- {p.description}</span>
)}
</SelectItem>
))}
</SelectContent>
</Select>
{value.length > 0 && (
<Button
variant="ghost"
size="sm"
className="h-8 px-2 text-xs text-muted-foreground"
onClick={clearAll}
>
Clear
</Button>
)}
</div>
)}
{/* Sortable effects chain */}
<DndContext sensors={sensors} collisionDetection={closestCenter} onDragEnd={handleDragEnd}>
<SortableContext items={items.map((i) => i._id)} strategy={verticalListSortingStrategy}>
{items.map((effect, index) => (
<SortableEffectItem
key={effect._id}
id={effect._id}
effect={effect}
index={index}
effectDef={effectsMap.get(effect.type)}
isExpanded={expandedId === effect._id}
onToggleExpand={() => setExpandedId(expandedId === effect._id ? null : effect._id)}
onRemove={() => removeEffect(index)}
onToggleEnabled={() => toggleEnabled(index)}
onUpdateParam={(paramName, paramValue) => updateParam(index, paramName, paramValue)}
/>
))}
</SortableContext>
</DndContext>
{/* Add effect */}
{availableEffects && (
<Select onValueChange={addEffect}>
<SelectTrigger className="h-8 border-dashed text-xs text-muted-foreground focus:ring-0 focus:ring-offset-0">
<Plus className="mr-1 h-3.5 w-3.5" />
<SelectValue placeholder="Add effect..." />
</SelectTrigger>
<SelectContent>
{availableEffects.effects.map((e) => (
<SelectItem key={e.type} value={e.type}>
{e.label}
</SelectItem>
))}
</SelectContent>
</Select>
)}
</div>
);
}
// ---------------------------------------------------------------------------
// Sortable effect item
// ---------------------------------------------------------------------------
interface SortableEffectItemProps {
id: string;
effect: EffectConfig;
index: number;
effectDef?: AvailableEffect;
isExpanded: boolean;
onToggleExpand: () => void;
onRemove: () => void;
onToggleEnabled: () => void;
onUpdateParam: (paramName: string, paramValue: number) => void;
}
function SortableEffectItem({
id,
effect,
effectDef,
isExpanded,
onToggleExpand,
onRemove,
onToggleEnabled,
onUpdateParam,
}: SortableEffectItemProps) {
const { attributes, listeners, setNodeRef, transform, transition, isDragging } = useSortable({
id,
});
const style = {
transform: CSS.Transform.toString(transform),
transition,
zIndex: isDragging ? 10 : undefined,
};
const label = effectDef?.label ?? effect.type;
return (
<div
ref={setNodeRef}
style={style}
className={cn(
'rounded-md border',
effect.enabled ? 'border-border bg-card' : 'border-border/50 bg-muted/30',
isDragging && 'opacity-80 shadow-lg',
)}
>
{/* Header */}
<div className="flex items-center gap-1 px-2 py-1.5">
<button
type="button"
className="p-0.5 text-muted-foreground hover:text-foreground"
onClick={onToggleExpand}
>
{isExpanded ? (
<ChevronDown className="h-3.5 w-3.5" />
) : (
<ChevronRight className="h-3.5 w-3.5" />
)}
</button>
<button
type="button"
className="p-0.5 text-muted-foreground/50 hover:text-muted-foreground cursor-grab active:cursor-grabbing touch-none"
{...attributes}
{...listeners}
>
<GripVertical className="h-3.5 w-3.5" />
</button>
<span
className={cn('flex-1 text-xs font-medium', !effect.enabled && 'text-muted-foreground')}
>
{label}
</span>
<button
type="button"
className={cn(
'p-0.5 transition-colors',
effect.enabled ? 'text-primary' : 'text-muted-foreground hover:text-foreground',
)}
onClick={onToggleEnabled}
title={effect.enabled ? 'Disable' : 'Enable'}
>
<Power className="h-3.5 w-3.5" />
</button>
<button
type="button"
className="p-0.5 text-muted-foreground hover:text-destructive"
onClick={onRemove}
title="Remove"
>
<Trash2 className="h-3.5 w-3.5" />
</button>
</div>
{/* Params */}
{isExpanded && effectDef && (
<div className="space-y-3 border-t px-3 py-2.5">
{Object.entries(effectDef.params).map(([paramName, paramDef]) => {
const currentValue = effect.params[paramName] ?? paramDef.default;
return (
<div key={paramName} className="space-y-1">
<div className="flex items-center justify-between">
<Label className="text-[11px] text-muted-foreground">
{paramDef.description}
</Label>
<span className="text-[11px] font-mono tabular-nums text-foreground">
{currentValue.toFixed(
paramDef.step < 1 ? Math.max(1, -Math.floor(Math.log10(paramDef.step))) : 0,
)}
</span>
</div>
<Slider
min={paramDef.min}
max={paramDef.max}
step={paramDef.step}
value={[currentValue]}
onValueChange={([v]) => onUpdateParam(paramName, v)}
/>
</div>
);
})}
</div>
)}
</div>
);
}
@@ -0,0 +1,103 @@
import { ChevronDown, Search } from 'lucide-react';
import { useMemo, useState } from 'react';
import { Button } from '@/components/ui/button';
import { Input } from '@/components/ui/input';
import { Popover, PopoverContent, PopoverTrigger } from '@/components/ui/popover';
import type { HistoryResponse } from '@/lib/api/types';
import { useHistory } from '@/lib/hooks/useHistory';
import { cn } from '@/lib/utils/cn';
interface GenerationPickerProps {
selectedId: string | null;
onSelect: (generation: HistoryResponse) => void;
className?: string;
}
export function GenerationPicker({ selectedId, onSelect, className }: GenerationPickerProps) {
const [open, setOpen] = useState(false);
const [searchQuery, setSearchQuery] = useState('');
const { data: historyData } = useHistory({ limit: 50 });
const completedGenerations = useMemo(() => {
if (!historyData?.items) return [];
return historyData.items.filter((gen) => gen.status === 'completed');
}, [historyData]);
const filtered = useMemo(() => {
if (!searchQuery) return completedGenerations;
const q = searchQuery.toLowerCase();
return completedGenerations.filter(
(gen) => gen.text.toLowerCase().includes(q) || gen.profile_name.toLowerCase().includes(q),
);
}, [completedGenerations, searchQuery]);
const selectedGeneration = completedGenerations.find((g) => g.id === selectedId);
return (
<Popover open={open} onOpenChange={setOpen}>
<PopoverTrigger asChild>
<Button
variant="outline"
size="sm"
className={cn('h-8 justify-between gap-2 text-xs font-normal', className)}
>
{selectedGeneration ? (
<span className="truncate">
<span className="font-medium">{selectedGeneration.profile_name}</span>
<span className="text-muted-foreground ml-1.5">
{selectedGeneration.text.length > 30
? `${selectedGeneration.text.substring(0, 30)}...`
: selectedGeneration.text}
</span>
</span>
) : (
<span className="text-muted-foreground">Select a generation...</span>
)}
<ChevronDown className="h-3.5 w-3.5 shrink-0 opacity-50" />
</Button>
</PopoverTrigger>
<PopoverContent className="w-80 p-0" align="start">
<div className="p-2 border-b">
<div className="relative">
<Search className="absolute left-2 top-1/2 -translate-y-1/2 h-3.5 w-3.5 text-muted-foreground" />
<Input
placeholder="Search by voice or text..."
value={searchQuery}
onChange={(e) => setSearchQuery(e.target.value)}
className="h-8 pl-7 text-xs"
/>
</div>
</div>
<div className="max-h-60 overflow-y-auto">
{filtered.length === 0 ? (
<div className="p-4 text-center text-xs text-muted-foreground">
No generations found
</div>
) : (
filtered.map((gen) => (
<button
key={gen.id}
type="button"
className={cn(
'w-full text-left px-3 py-2 hover:bg-muted/50 transition-colors border-b border-border/30 last:border-0',
gen.id === selectedId && 'bg-accent/10',
)}
onClick={() => {
onSelect(gen);
setOpen(false);
setSearchQuery('');
}}
>
<div className="font-medium text-sm">{gen.profile_name}</div>
<div className="text-xs text-muted-foreground truncate">
{gen.text.length > 60 ? `${gen.text.substring(0, 60)}...` : gen.text}
</div>
</button>
))
)}
</div>
</PopoverContent>
</Popover>
);
}
@@ -0,0 +1,332 @@
import { useQuery, useQueryClient } from '@tanstack/react-query';
import { Loader2, Play, Save, Trash2, Wand2 } from 'lucide-react';
import { useEffect, useRef, useState } from 'react';
import { EffectsChainEditor } from '@/components/Effects/EffectsChainEditor';
import { GenerationPicker } from '@/components/Effects/GenerationPicker';
import { Button } from '@/components/ui/button';
import { Input } from '@/components/ui/input';
import { Label } from '@/components/ui/label';
import { Separator } from '@/components/ui/separator';
import { Textarea } from '@/components/ui/textarea';
import { useToast } from '@/components/ui/use-toast';
import { apiClient } from '@/lib/api/client';
import type { HistoryResponse } from '@/lib/api/types';
import { useHistory } from '@/lib/hooks/useHistory';
import { useEffectsStore } from '@/stores/effectsStore';
import { usePlayerStore } from '@/stores/playerStore';
export function EffectsDetail() {
const selectedPresetId = useEffectsStore((s) => s.selectedPresetId);
const isCreatingNew = useEffectsStore((s) => s.isCreatingNew);
const workingChain = useEffectsStore((s) => s.workingChain);
const setWorkingChain = useEffectsStore((s) => s.setWorkingChain);
const setSelectedPresetId = useEffectsStore((s) => s.setSelectedPresetId);
const setIsCreatingNew = useEffectsStore((s) => s.setIsCreatingNew);
const [name, setName] = useState('');
const [description, setDescription] = useState('');
const [saving, setSaving] = useState(false);
const [deleting, setDeleting] = useState(false);
// Preview state
const [previewGenId, setPreviewGenId] = useState<string | null>(null);
const [previewLoading, setPreviewLoading] = useState(false);
const blobUrlRef = useRef<string | null>(null);
const setAudioWithAutoPlay = usePlayerStore((s) => s.setAudioWithAutoPlay);
const { toast } = useToast();
const queryClient = useQueryClient();
// Auto-select the most recent generation as preview source
const { data: historyData } = useHistory({ limit: 1 });
useEffect(() => {
if (!previewGenId && historyData?.items?.length) {
const first = historyData.items.find((g) => g.status === 'completed');
if (first) setPreviewGenId(first.id);
}
}, [historyData, previewGenId]);
const { data: preset } = useQuery({
queryKey: ['effect-preset', selectedPresetId],
queryFn: () =>
selectedPresetId
? apiClient
.listEffectPresets()
.then((all) => all.find((p) => p.id === selectedPresetId) ?? null)
: null,
enabled: !!selectedPresetId,
staleTime: 30_000,
});
// Sync name/description when selecting a preset
useEffect(() => {
if (preset) {
setName(preset.name);
setDescription(preset.description ?? '');
} else if (isCreatingNew) {
setName('');
setDescription('');
}
}, [preset, isCreatingNew]);
// Cleanup blob URL on unmount
useEffect(() => {
return () => {
if (blobUrlRef.current) {
URL.revokeObjectURL(blobUrlRef.current);
blobUrlRef.current = null;
}
};
}, []);
const isEditing = !!selectedPresetId || isCreatingNew;
const isBuiltIn = preset?.is_builtin ?? false;
async function handlePreview() {
if (!previewGenId || workingChain.length === 0) return;
setPreviewLoading(true);
try {
const blob = await apiClient.previewEffects(previewGenId, workingChain);
// Revoke old blob URL
if (blobUrlRef.current) {
URL.revokeObjectURL(blobUrlRef.current);
}
const url = URL.createObjectURL(blob);
blobUrlRef.current = url;
// Play through the main audio player
setAudioWithAutoPlay(url, `preview-${Date.now()}`, null, 'Effects Preview');
} catch (error) {
toast({
title: 'Preview failed',
description: error instanceof Error ? error.message : 'Unknown error',
variant: 'destructive',
});
} finally {
setPreviewLoading(false);
}
}
function handleSelectGeneration(gen: HistoryResponse) {
setPreviewGenId(gen.id);
}
async function handleSaveNew() {
if (!name.trim()) {
toast({ title: 'Name required', variant: 'destructive' });
return;
}
setSaving(true);
try {
const created = await apiClient.createEffectPreset({
name: name.trim(),
description: description.trim() || undefined,
effects_chain: workingChain,
});
queryClient.invalidateQueries({ queryKey: ['effect-presets'] });
setIsCreatingNew(false);
setSelectedPresetId(created.id);
toast({ title: 'Preset saved', description: `"${created.name}" has been created.` });
} catch (error) {
toast({
title: 'Failed to save',
description: error instanceof Error ? error.message : 'Unknown error',
variant: 'destructive',
});
} finally {
setSaving(false);
}
}
async function handleSaveExisting() {
if (!selectedPresetId || !name.trim()) return;
setSaving(true);
try {
await apiClient.updateEffectPreset(selectedPresetId, {
name: name.trim(),
description: description.trim() || undefined,
effects_chain: workingChain,
});
queryClient.invalidateQueries({ queryKey: ['effect-presets'] });
queryClient.invalidateQueries({ queryKey: ['effect-preset', selectedPresetId] });
toast({ title: 'Preset updated' });
} catch (error) {
toast({
title: 'Failed to save',
description: error instanceof Error ? error.message : 'Unknown error',
variant: 'destructive',
});
} finally {
setSaving(false);
}
}
async function handleSaveAsNew() {
await handleSaveNew();
}
async function handleDelete() {
if (!selectedPresetId) return;
setDeleting(true);
try {
await apiClient.deleteEffectPreset(selectedPresetId);
queryClient.invalidateQueries({ queryKey: ['effect-presets'] });
setSelectedPresetId(null);
setWorkingChain([]);
toast({ title: 'Preset deleted' });
} catch (error) {
toast({
title: 'Failed to delete',
description: error instanceof Error ? error.message : 'Unknown error',
variant: 'destructive',
});
} finally {
setDeleting(false);
}
}
if (!isEditing) {
return (
<div className="flex-1 flex items-center justify-center text-muted-foreground">
<div className="text-center space-y-2">
<Wand2 className="h-10 w-10 mx-auto opacity-30" />
<p className="text-sm">Select a preset or create a new one</p>
</div>
</div>
);
}
return (
<div className="flex flex-col h-full min-h-0">
{/* Header */}
<div className="flex items-center justify-between mb-4">
<h2 className="text-lg font-semibold">
{isCreatingNew ? 'New Preset' : isBuiltIn ? preset?.name : 'Edit Preset'}
</h2>
<div className="flex items-center gap-2">
{!isBuiltIn && !isCreatingNew && (
<>
<Button
variant="ghost"
size="sm"
className="h-8 text-destructive hover:text-destructive gap-1.5"
onClick={handleDelete}
disabled={deleting}
>
<Trash2 className="h-3.5 w-3.5" />
{deleting ? 'Deleting...' : 'Delete'}
</Button>
<Button
size="sm"
className="h-8 gap-1.5"
onClick={handleSaveExisting}
disabled={saving || workingChain.length === 0}
>
<Save className="h-3.5 w-3.5" />
{saving ? 'Saving...' : 'Save'}
</Button>
</>
)}
{isCreatingNew && (
<Button
size="sm"
className="h-8 gap-1.5"
onClick={handleSaveNew}
disabled={saving || workingChain.length === 0}
>
<Save className="h-3.5 w-3.5" />
{saving ? 'Saving...' : 'Save Preset'}
</Button>
)}
{isBuiltIn && (
<Button
size="sm"
variant="outline"
className="h-8 gap-1.5"
onClick={handleSaveAsNew}
disabled={saving}
>
<Save className="h-3.5 w-3.5" />
{saving ? 'Saving...' : 'Save as Custom'}
</Button>
)}
</div>
</div>
{/* Scrollable content */}
<div className="flex-1 min-h-0 overflow-y-auto space-y-5 pr-1">
{/* Name & description */}
{(isCreatingNew || !isBuiltIn) && (
<div className="space-y-3">
<div className="space-y-1.5">
<Label className="text-xs">Name</Label>
<Input
value={name}
onChange={(e) => setName(e.target.value)}
placeholder="My preset..."
className="h-9"
/>
</div>
<div className="space-y-1.5">
<Label className="text-xs">Description</Label>
<Textarea
value={description}
onChange={(e) => setDescription(e.target.value)}
placeholder="Describe what this preset does..."
className="min-h-[60px] resize-none"
/>
</div>
</div>
)}
{/* Built-in description (read-only) */}
{isBuiltIn && preset?.description && (
<p className="text-sm text-muted-foreground">{preset.description}</p>
)}
{/* Effects chain editor */}
<EffectsChainEditor value={workingChain} onChange={setWorkingChain} showPresets={false} />
<Separator />
{/* Preview section */}
<div className="space-y-3">
<Label className="text-xs">Preview</Label>
<div className="flex items-center gap-2">
<GenerationPicker
selectedId={previewGenId}
onSelect={handleSelectGeneration}
className="flex-1"
/>
<Button
variant="outline"
size="sm"
className="h-8 gap-1.5 shrink-0"
onClick={handlePreview}
disabled={!previewGenId || workingChain.length === 0 || previewLoading}
>
{previewLoading ? (
<>
<Loader2 className="h-3.5 w-3.5 animate-spin" />
Processing...
</>
) : (
<>
<Play className="h-3.5 w-3.5" />
Preview
</>
)}
</Button>
</div>
<p className="text-[11px] text-muted-foreground">
Preview applies effects to the clean version without saving.
</p>
</div>
</div>
</div>
);
}
@@ -0,0 +1,165 @@
import { useQuery } from '@tanstack/react-query';
import { Loader2, Plus, Sparkles, Wand2 } from 'lucide-react';
import { Button } from '@/components/ui/button';
import { apiClient } from '@/lib/api/client';
import type { EffectPresetResponse } from '@/lib/api/types';
import { cn } from '@/lib/utils/cn';
import { useEffectsStore } from '@/stores/effectsStore';
export function EffectsList() {
const selectedPresetId = useEffectsStore((s) => s.selectedPresetId);
const setSelectedPresetId = useEffectsStore((s) => s.setSelectedPresetId);
const setWorkingChain = useEffectsStore((s) => s.setWorkingChain);
const setIsCreatingNew = useEffectsStore((s) => s.setIsCreatingNew);
const isCreatingNew = useEffectsStore((s) => s.isCreatingNew);
const { data: presets, isLoading } = useQuery({
queryKey: ['effect-presets'],
queryFn: () => apiClient.listEffectPresets(),
staleTime: 30_000,
});
const builtIn = presets?.filter((p) => p.is_builtin) ?? [];
const userPresets = presets?.filter((p) => !p.is_builtin) ?? [];
function handleSelect(preset: EffectPresetResponse) {
setSelectedPresetId(preset.id);
setWorkingChain(preset.effects_chain);
}
function handleCreateNew() {
setIsCreatingNew(true);
setWorkingChain([]);
}
if (isLoading) {
return (
<div className="flex items-center justify-center h-full">
<Loader2 className="h-6 w-6 animate-spin text-muted-foreground" />
</div>
);
}
return (
<div className="flex flex-col h-full min-h-0">
{/* Header */}
<div className="flex items-center justify-between mb-4">
<h2 className="text-lg font-semibold">Effects</h2>
<Button variant="outline" size="sm" className="h-8 gap-1.5" onClick={handleCreateNew}>
<Plus className="h-3.5 w-3.5" />
New Preset
</Button>
</div>
{/* Scrollable list */}
<div className="flex-1 min-h-0 overflow-y-auto space-y-4">
{/* Built-in presets */}
{builtIn.length > 0 && (
<div>
<div className="text-[11px] text-muted-foreground font-medium uppercase tracking-wider mb-2 px-1">
Built-in
</div>
<div className="space-y-1.5">
{builtIn.map((preset) => (
<PresetCard
key={preset.id}
preset={preset}
isSelected={selectedPresetId === preset.id && !isCreatingNew}
onSelect={() => handleSelect(preset)}
/>
))}
</div>
</div>
)}
{/* User presets */}
{userPresets.length > 0 && (
<div>
<div className="text-[11px] text-muted-foreground font-medium uppercase tracking-wider mb-2 px-1">
Custom
</div>
<div className="space-y-1.5">
{userPresets.map((preset) => (
<PresetCard
key={preset.id}
preset={preset}
isSelected={selectedPresetId === preset.id && !isCreatingNew}
onSelect={() => handleSelect(preset)}
/>
))}
</div>
</div>
)}
{/* New preset placeholder */}
{isCreatingNew && (
<div>
<div className="text-[11px] text-muted-foreground font-medium uppercase tracking-wider mb-2 px-1">
New
</div>
<div className="rounded-xl border-2 border-accent/40 bg-accent/5 p-3">
<div className="flex items-center gap-2">
<Sparkles className="h-4 w-4 text-accent" />
<span className="text-sm font-medium">Unsaved Preset</span>
</div>
<p className="text-xs text-muted-foreground mt-1">
Configure effects in the panel on the right.
</p>
</div>
</div>
)}
</div>
</div>
);
}
function PresetCard({
preset,
isSelected,
onSelect,
}: {
preset: EffectPresetResponse;
isSelected: boolean;
onSelect: () => void;
}) {
const effectCount = preset.effects_chain.length;
return (
<button
type="button"
className={cn(
'w-full text-left rounded-xl border p-3 h-[88px] transition-all duration-150',
isSelected
? 'border-accent/50 bg-accent/10'
: 'border-border bg-card hover:bg-muted/50 hover:border-border',
)}
onClick={onSelect}
>
<div className="flex items-center gap-2">
<Wand2
className={cn('h-4 w-4 shrink-0', isSelected ? 'text-accent' : 'text-muted-foreground')}
/>
<span className="text-sm font-medium truncate">{preset.name}</span>
{preset.is_builtin && (
<span className="text-[10px] bg-muted text-muted-foreground px-1.5 py-0.5 rounded-full shrink-0">
built-in
</span>
)}
</div>
<p className="text-xs text-muted-foreground mt-1 line-clamp-1 pl-6">
{preset.description || 'No description'}
</p>
<div className="flex items-center gap-2 mt-1.5 pl-6">
<span className="text-[10px] text-muted-foreground">
{effectCount} effect{effectCount !== 1 ? 's' : ''}
</span>
<span className="text-[10px] text-muted-foreground/50">
{preset.effects_chain
.filter((e) => e.enabled)
.map((e) => e.type)
.join(' → ')}
</span>
</div>
</button>
);
}
@@ -0,0 +1,20 @@
import {EffectsDetail} from "./EffectsDetail";
import {EffectsList} from "./EffectsList";
export function EffectsTab() {
return (
<div className="flex flex-col h-full min-h-0 overflow-hidden">
<div className="flex-1 min-h-0 flex gap-6 overflow-hidden">
{/* Left - Presets list */}
<div className="w-full max-w-[360px] shrink-0 flex flex-col min-h-0">
<EffectsList />
</div>
{/* Right - Detail / editor */}
<div className="flex-1 min-h-0 flex flex-col">
<EffectsDetail />
</div>
</div>
</div>
);
}
@@ -2,6 +2,7 @@ import { useMatchRoute } from '@tanstack/react-router';
import { AnimatePresence, motion } from 'framer-motion';
import { Loader2, SlidersHorizontal, Sparkles } from 'lucide-react';
import { useEffect, useRef, useState } from 'react';
import { EffectsChainEditor } from '@/components/Effects/EffectsChainEditor';
import { Button } from '@/components/ui/button';
import { Form, FormControl, FormField, FormItem, FormMessage } from '@/components/ui/form';
import {
@@ -12,14 +13,16 @@ import {
SelectValue,
} from '@/components/ui/select';
import { Textarea } from '@/components/ui/textarea';
import { useToast } from '@/components/ui/use-toast';
import { LANGUAGE_OPTIONS } from '@/lib/constants/languages';
import type { EffectConfig } from '@/lib/api/types';
import { getLanguageOptionsForEngine, type LanguageCode } from '@/lib/constants/languages';
import { useGenerationForm } from '@/lib/hooks/useGenerationForm';
import { useProfile, useProfiles } from '@/lib/hooks/useProfiles';
import { useAddStoryItem, useStory } from '@/lib/hooks/useStories';
import { useStory } from '@/lib/hooks/useStories';
import { cn } from '@/lib/utils/cn';
import { useGenerationStore } from '@/stores/generationStore';
import { useStoryStore } from '@/stores/storyStore';
import { useUIStore } from '@/stores/uiStore';
import { ParalinguisticInput } from './ParalinguisticInput';
interface FloatingGenerateBoxProps {
isPlayerOpen?: boolean;
@@ -36,6 +39,7 @@ export function FloatingGenerateBox({
const { data: profiles } = useProfiles();
const [isExpanded, setIsExpanded] = useState(false);
const [isInstructMode, setIsInstructMode] = useState(false);
const [effectsChain, setEffectsChain] = useState<EffectConfig[]>([]);
const containerRef = useRef<HTMLDivElement>(null);
const textareaRef = useRef<HTMLTextAreaElement | null>(null);
const matchRoute = useMatchRoute();
@@ -43,8 +47,7 @@ export function FloatingGenerateBox({
const selectedStoryId = useStoryStore((state) => state.selectedStoryId);
const trackEditorHeight = useStoryStore((state) => state.trackEditorHeight);
const { data: currentStory } = useStory(selectedStoryId);
const addStoryItem = useAddStoryItem();
const { toast } = useToast();
const addPendingStoryAdd = useGenerationStore((s) => s.addPendingStoryAdd);
// Calculate if track editor is visible (on stories route with items)
const hasTrackEditor = isStoriesRoute && currentStory && currentStory.items.length > 0;
@@ -52,27 +55,12 @@ export function FloatingGenerateBox({
const { form, handleSubmit, isPending } = useGenerationForm({
onSuccess: async (generationId) => {
setIsExpanded(false);
// If on stories route and a story is selected, add generation to story
// Defer the story add until TTS completes — useGenerationProgress handles it
if (isStoriesRoute && selectedStoryId && generationId) {
try {
await addStoryItem.mutateAsync({
storyId: selectedStoryId,
data: { generation_id: generationId },
});
toast({
title: 'Added to story',
description: `Generation added to "${currentStory?.name || 'story'}"`,
});
} catch (error) {
toast({
title: 'Failed to add to story',
description:
error instanceof Error ? error.message : 'Could not add generation to story',
variant: 'destructive',
});
}
addPendingStoryAdd(generationId, selectedStoryId);
}
},
getEffectsChain: () => (effectsChain.length > 0 ? effectsChain : undefined),
});
// Click away handler to collapse the box
@@ -112,6 +100,13 @@ export function FloatingGenerateBox({
}
}, [selectedProfileId, profiles, setSelectedProfileId]);
// Sync generation form language with selected profile's language
useEffect(() => {
if (selectedProfile?.language) {
form.setValue('language', selectedProfile.language as LanguageCode);
}
}, [selectedProfile, form]);
// Auto-resize textarea based on content (only when expanded)
useEffect(() => {
if (!isExpanded) {
@@ -174,7 +169,7 @@ export function FloatingGenerateBox({
isStoriesRoute
? // Position aligned with story list: after sidebar + padding, width 360px
'left-[calc(5rem+2rem)] w-[360px]'
: 'left-[calc(5rem+2rem)] w-[calc((100%-5rem-4rem)/2-1rem)]',
: 'left-[calc(5rem+2rem)] right-8 lg:right-auto lg:w-[calc((100%-5rem-4rem)/2-1rem)]',
)}
style={{
// On stories route: offset by track editor height when visible
@@ -212,34 +207,57 @@ export function FloatingGenerateBox({
transition={{ duration: 0.15, ease: 'easeOut' }}
style={{ overflow: 'hidden' }}
>
<Textarea
{...field}
ref={(node: HTMLTextAreaElement | null) => {
// Store ref for auto-resize (only for active field)
if (!isInstructMode) {
textareaRef.current = node;
{form.watch('engine') === 'chatterbox_turbo' ? (
<ParalinguisticInput
value={field.value}
onChange={field.onChange}
placeholder={
isStoriesRoute && currentStory
? `Generate speech for "${currentStory.name}"... (type / for effects)`
: selectedProfile
? `Type / for effects like [laugh], [sigh]...`
: 'Select a voice profile above...'
}
// Forward ref to react-hook-form
if (typeof field.ref === 'function') {
field.ref(node);
className="px-3 py-2 resize-none bg-transparent border-none focus-visible:ring-0 focus-visible:ring-offset-0 focus:outline-none focus:ring-0 outline-none ring-0 rounded-2xl text-sm w-full"
style={{
minHeight: isExpanded ? '100px' : '32px',
maxHeight: '300px',
overflowY: 'auto',
}}
disabled={!selectedProfileId}
onClick={() => setIsExpanded(true)}
onFocus={() => setIsExpanded(true)}
/>
) : (
<Textarea
{...field}
ref={(node: HTMLTextAreaElement | null) => {
// Store ref for auto-resize (only for active field)
if (!isInstructMode) {
textareaRef.current = node;
}
// Forward ref to react-hook-form
if (typeof field.ref === 'function') {
field.ref(node);
}
}}
placeholder={
isStoriesRoute && currentStory
? `Generate speech for "${currentStory.name}"...`
: selectedProfile
? `Generate speech using ${selectedProfile.name}...`
: 'Select a voice profile above...'
}
}}
placeholder={
isStoriesRoute && currentStory
? `Generate speech for "${currentStory.name}"...`
: selectedProfile
? `Generate speech using ${selectedProfile.name}...`
: 'Select a voice profile above...'
}
className="resize-none bg-transparent border-none focus-visible:ring-0 focus-visible:ring-offset-0 focus:outline-none focus:ring-0 outline-none ring-0 rounded-2xl text-sm placeholder:text-muted-foreground/60 w-full"
style={{
minHeight: isExpanded ? '100px' : '32px',
maxHeight: '300px',
}}
disabled={!selectedProfileId}
onClick={() => setIsExpanded(true)}
onFocus={() => setIsExpanded(true)}
/>
className="resize-none bg-transparent border-none focus-visible:ring-0 focus-visible:ring-offset-0 focus:outline-none focus:ring-0 outline-none ring-0 rounded-2xl text-sm placeholder:text-muted-foreground/60 w-full"
style={{
minHeight: isExpanded ? '100px' : '32px',
maxHeight: '300px',
}}
disabled={!selectedProfileId}
onClick={() => setIsExpanded(true)}
onFocus={() => setIsExpanded(true)}
/>
)}
</motion.div>
</FormControl>
<FormMessage className="text-xs" />
@@ -300,6 +318,13 @@ export function FloatingGenerateBox({
disabled={isPending || !selectedProfileId}
className="h-10 w-10 rounded-full bg-accent hover:bg-accent/90 hover:scale-105 text-accent-foreground shadow-lg hover:shadow-accent/50 transition-all duration-200"
size="icon"
aria-label={
isPending
? 'Generating...'
: !selectedProfileId
? 'Select a voice profile first'
: 'Generate speech'
}
>
{isPending ? (
<Loader2 className="h-4 w-4 animate-spin" />
@@ -316,7 +341,7 @@ export function FloatingGenerateBox({
</span>
</div>
<AnimatePresence>
{isExpanded && form.watch('engine') !== 'luxtts' && (
{isExpanded && form.watch('engine') === 'qwen' && (
<motion.div
initial={{ opacity: 0, scale: 0.8 }}
animate={{ opacity: 1, scale: 1 }}
@@ -334,13 +359,18 @@ export function FloatingGenerateBox({
'h-10 w-10 rounded-full transition-all duration-200',
isInstructMode
? 'bg-accent text-accent-foreground border border-accent hover:bg-accent/90'
: 'bg-card border border-border hover:bg-background/50',
: effectsChain.length > 0
? 'bg-accent/50 text-accent-foreground border border-accent/50 hover:bg-accent/70'
: 'bg-card border border-border hover:bg-background/50',
)}
aria-label={
isInstructMode ? 'Fine tune instructions, on' : 'Fine tune instructions'
}
>
<SlidersHorizontal className="h-4 w-4" />
</Button>
<span className="pointer-events-none absolute bottom-full left-1/2 -translate-x-1/2 mb-2 whitespace-nowrap rounded-md bg-popover px-3 py-1.5 text-xs text-popover-foreground border border-border opacity-0 transition-opacity group-hover:opacity-100 z-[9999]">
Fine tune instructions
Fine tune instructions & effects
</span>
</div>
</motion.div>
@@ -349,6 +379,23 @@ export function FloatingGenerateBox({
</div>
</div>
{/* Effects chain editor panel - shown alongside instruct */}
<AnimatePresence>
{isExpanded && isInstructMode && (
<motion.div
initial={{ height: 0, opacity: 0 }}
animate={{ height: 'auto', opacity: 1 }}
exit={{ height: 0, opacity: 0 }}
transition={{ duration: 0.2 }}
className="overflow-hidden mt-2"
>
<div className="border-t border-border/50 pt-2 pb-1">
<EffectsChainEditor value={effectsChain} onChange={setEffectsChain} compact />
</div>
</motion.div>
)}
</AnimatePresence>
<AnimatePresence>
<motion.div
initial={{ height: 0, opacity: 0 }}
@@ -381,25 +428,30 @@ export function FloatingGenerateBox({
<FormField
control={form.control}
name="language"
render={({ field }) => (
<FormItem className="flex-1 space-y-0">
<Select onValueChange={field.onChange} defaultValue={field.value}>
<FormControl>
<SelectTrigger className="h-8 text-xs bg-card border-border rounded-full hover:bg-background/50 transition-all">
<SelectValue />
</SelectTrigger>
</FormControl>
<SelectContent>
{LANGUAGE_OPTIONS.map((lang) => (
<SelectItem key={lang.value} value={lang.value} className="text-xs">
{lang.label}
</SelectItem>
))}
</SelectContent>
</Select>
<FormMessage className="text-xs" />
</FormItem>
)}
render={({ field }) => {
const engineLangs = getLanguageOptionsForEngine(
form.watch('engine') || 'qwen',
);
return (
<FormItem className="flex-1 space-y-0">
<Select onValueChange={field.onChange} value={field.value}>
<FormControl>
<SelectTrigger className="h-8 text-xs bg-card border-border rounded-full hover:bg-background/50 transition-all">
<SelectValue />
</SelectTrigger>
</FormControl>
<SelectContent>
{engineLangs.map((lang) => (
<SelectItem key={lang.value} value={lang.value} className="text-xs">
{lang.label}
</SelectItem>
))}
</SelectContent>
</Select>
<FormMessage className="text-xs" />
</FormItem>
);
}}
/>
<FormItem className="flex-1 space-y-0">
@@ -407,11 +459,21 @@ export function FloatingGenerateBox({
value={
form.watch('engine') === 'luxtts'
? 'luxtts'
: `qwen:${form.watch('modelSize') || '1.7B'}`
: form.watch('engine') === 'chatterbox'
? 'chatterbox'
: form.watch('engine') === 'chatterbox_turbo'
? 'chatterbox_turbo'
: `qwen:${form.watch('modelSize') || '1.7B'}`
}
onValueChange={(value) => {
if (value === 'luxtts') {
form.setValue('engine', 'luxtts');
form.setValue('language', 'en');
} else if (value === 'chatterbox') {
form.setValue('engine', 'chatterbox');
} else if (value === 'chatterbox_turbo') {
form.setValue('engine', 'chatterbox_turbo');
form.setValue('language', 'en');
} else {
const [, modelSize] = value.split(':');
form.setValue('engine', 'qwen');
@@ -434,6 +496,15 @@ export function FloatingGenerateBox({
<SelectItem value="luxtts" className="text-xs text-muted-foreground">
LuxTTS
</SelectItem>
<SelectItem value="chatterbox" className="text-xs text-muted-foreground">
Chatterbox
</SelectItem>
<SelectItem
value="chatterbox_turbo"
className="text-xs text-muted-foreground"
>
Chatterbox Turbo
</SelectItem>
</SelectContent>
</Select>
</FormItem>
@@ -19,10 +19,11 @@ import {
SelectValue,
} from '@/components/ui/select';
import { Textarea } from '@/components/ui/textarea';
import { LANGUAGE_OPTIONS } from '@/lib/constants/languages';
import { getLanguageOptionsForEngine } from '@/lib/constants/languages';
import { useGenerationForm } from '@/lib/hooks/useGenerationForm';
import { useProfile } from '@/lib/hooks/useProfiles';
import { useUIStore } from '@/stores/uiStore';
import { ParalinguisticInput } from './ParalinguisticInput';
export function GenerationForm() {
const selectedProfileId = useUIStore((state) => state.selectedProfileId);
@@ -64,19 +65,32 @@ export function GenerationForm() {
<FormItem>
<FormLabel>Text to Speak</FormLabel>
<FormControl>
<Textarea
placeholder="Enter the text you want to generate..."
className="min-h-[150px]"
{...field}
/>
{form.watch('engine') === 'chatterbox_turbo' ? (
<ParalinguisticInput
value={field.value}
onChange={field.onChange}
placeholder="Enter text... type / for effects like [laugh], [sigh]"
className="min-h-[150px] rounded-md border border-input bg-background px-3 py-2"
/>
) : (
<Textarea
placeholder="Enter the text you want to generate..."
className="min-h-[150px]"
{...field}
/>
)}
</FormControl>
<FormDescription>Max 5000 characters</FormDescription>
<FormDescription>
{form.watch('engine') === 'chatterbox_turbo'
? 'Max 5000 characters. Type / to insert sound effects.'
: 'Max 5000 characters'}
</FormDescription>
<FormMessage />
</FormItem>
)}
/>
{form.watch('engine') !== 'luxtts' && (
{form.watch('engine') === 'qwen' && (
<FormField
control={form.control}
name="instruct"
@@ -107,11 +121,21 @@ export function GenerationForm() {
value={
form.watch('engine') === 'luxtts'
? 'luxtts'
: `qwen:${form.watch('modelSize') || '1.7B'}`
: form.watch('engine') === 'chatterbox'
? 'chatterbox'
: form.watch('engine') === 'chatterbox_turbo'
? 'chatterbox_turbo'
: `qwen:${form.watch('modelSize') || '1.7B'}`
}
onValueChange={(value) => {
if (value === 'luxtts') {
form.setValue('engine', 'luxtts');
form.setValue('language', 'en');
} else if (value === 'chatterbox') {
form.setValue('engine', 'chatterbox');
} else if (value === 'chatterbox_turbo') {
form.setValue('engine', 'chatterbox_turbo');
form.setValue('language', 'en');
} else {
const [, modelSize] = value.split(':');
form.setValue('engine', 'qwen');
@@ -128,38 +152,47 @@ export function GenerationForm() {
<SelectItem value="qwen:1.7B">Qwen3-TTS 1.7B</SelectItem>
<SelectItem value="qwen:0.6B">Qwen3-TTS 0.6B</SelectItem>
<SelectItem value="luxtts">LuxTTS</SelectItem>
<SelectItem value="chatterbox">Chatterbox</SelectItem>
<SelectItem value="chatterbox_turbo">Chatterbox Turbo</SelectItem>
</SelectContent>
</Select>
<FormDescription>
{form.watch('engine') === 'luxtts'
? 'Fast, English-focused'
: 'Multi-language, two sizes'}
: form.watch('engine') === 'chatterbox'
? '23 languages, incl. Hebrew'
: form.watch('engine') === 'chatterbox_turbo'
? 'English, [laugh] [cough] tags'
: 'Multi-language, two sizes'}
</FormDescription>
</FormItem>
<FormField
control={form.control}
name="language"
render={({ field }) => (
<FormItem>
<FormLabel>Language</FormLabel>
<Select onValueChange={field.onChange} defaultValue={field.value}>
<FormControl>
<SelectTrigger>
<SelectValue />
</SelectTrigger>
</FormControl>
<SelectContent>
{LANGUAGE_OPTIONS.map((lang) => (
<SelectItem key={lang.value} value={lang.value}>
{lang.label}
</SelectItem>
))}
</SelectContent>
</Select>
<FormMessage />
</FormItem>
)}
render={({ field }) => {
const engineLangs = getLanguageOptionsForEngine(form.watch('engine') || 'qwen');
return (
<FormItem>
<FormLabel>Language</FormLabel>
<Select onValueChange={field.onChange} value={field.value}>
<FormControl>
<SelectTrigger>
<SelectValue />
</SelectTrigger>
</FormControl>
<SelectContent>
{engineLangs.map((lang) => (
<SelectItem key={lang.value} value={lang.value}>
{lang.label}
</SelectItem>
))}
</SelectContent>
</Select>
<FormMessage />
</FormItem>
);
}}
/>
<FormField
@@ -0,0 +1,422 @@
/**
* ParalinguisticInput — a contentEditable rich text input that renders
* Chatterbox Turbo paralinguistic tags (e.g. [laugh]) as inline badges.
*
* Trigger: typing "/" opens an autocomplete dropdown.
* Paste: pasting text with [tag] patterns auto-converts to badges.
* Output: serializes badges back to plain [tag] text for the API.
*/
import { AnimatePresence, motion } from 'framer-motion';
import { forwardRef, useCallback, useEffect, useImperativeHandle, useRef, useState } from 'react';
import { createPortal } from 'react-dom';
import { cn } from '@/lib/utils/cn';
// ── Tag definitions ─────────────────────────────────────────────────
const PARALINGUISTIC_TAGS = [
{ tag: '[laugh]', label: 'laugh', emoji: '\u{1F602}' },
{ tag: '[chuckle]', label: 'chuckle', emoji: '\u{1F60F}' },
{ tag: '[gasp]', label: 'gasp', emoji: '\u{1F62E}' },
{ tag: '[cough]', label: 'cough', emoji: '\u{1F637}' },
{ tag: '[sigh]', label: 'sigh', emoji: '\u{1F614}' },
{ tag: '[groan]', label: 'groan', emoji: '\u{1F629}' },
{ tag: '[sniff]', label: 'sniff', emoji: '\u{1F443}' },
{ tag: '[shush]', label: 'shush', emoji: '\u{1F92B}' },
{ tag: '[clear throat]', label: 'clear throat', emoji: '\u{1F64A}' },
] as const;
const TAG_REGEX = /\[(laugh|chuckle|gasp|cough|sigh|groan|sniff|shush|clear throat)\]/gi;
// Data attribute used to identify badge spans in the DOM
const BADGE_ATTR = 'data-ptag';
// ── Helpers ─────────────────────────────────────────────────────────
/** Build an inline badge <span> for a tag. */
function makeBadgeHTML(tag: string): string {
const entry = PARALINGUISTIC_TAGS.find((t) => t.tag.toLowerCase() === tag.toLowerCase());
const label = entry?.label ?? tag.replace(/[[\]]/g, '');
const emoji = entry?.emoji ?? '';
// Non-editable inline badge. Zero-width spaces around it let the
// caret sit on either side so the user can type before/after.
return `\u200B<span ${BADGE_ATTR}="${tag}" contenteditable="false" class="ptag-badge">${emoji ? `${emoji}\u00A0` : ''}${label}</span>\u200B`;
}
/** Convert plain text with [tag] patterns into HTML with badge spans. */
function textToHTML(text: string): string {
// Escape HTML entities first
const escaped = text.replace(/&/g, '&amp;').replace(/</g, '&lt;').replace(/>/g, '&gt;');
// Replace tag patterns with badge HTML
return escaped.replace(TAG_REGEX, (match) => makeBadgeHTML(match));
}
/** Serialize the contentEditable innerHTML back to plain text with [tag] syntax. */
function htmlToText(container: HTMLElement): string {
let result = '';
for (const node of container.childNodes) {
if (node.nodeType === Node.TEXT_NODE) {
// Strip zero-width spaces we added around badges
result += (node.textContent ?? '').replace(/\u200B/g, '');
} else if (node.nodeType === Node.ELEMENT_NODE) {
const el = node as HTMLElement;
if (el.hasAttribute(BADGE_ATTR)) {
result += el.getAttribute(BADGE_ATTR) ?? '';
} else if (el.tagName === 'BR') {
result += '\n';
} else {
// Recurse for nested elements (e.g. spans from paste)
result += htmlToText(el);
}
}
}
return result;
}
/** Get the text content from the current caret position back to the last
* whitespace or start of container, to detect the "/" trigger. */
function getWordBeforeCaret(_container: HTMLElement): { word: string; range: Range | null } {
const sel = window.getSelection();
if (!sel || sel.rangeCount === 0) return { word: '', range: null };
const range = sel.getRangeAt(0).cloneRange();
range.collapse(true);
// Walk backwards from caret through the text node
const textNode = range.startContainer;
if (textNode.nodeType !== Node.TEXT_NODE) return { word: '', range: null };
const text = textNode.textContent ?? '';
const offset = range.startOffset;
let start = offset;
while (
start > 0 &&
text[start - 1] !== ' ' &&
text[start - 1] !== '\n' &&
text[start - 1] !== '\u00A0'
) {
start--;
}
const word = text.slice(start, offset);
const wordRange = document.createRange();
wordRange.setStart(textNode, start);
wordRange.setEnd(textNode, offset);
return { word, range: wordRange };
}
// ── Component ───────────────────────────────────────────────────────
export interface ParalinguisticInputProps {
value?: string;
onChange?: (value: string) => void;
placeholder?: string;
disabled?: boolean;
className?: string;
style?: React.CSSProperties;
onClick?: () => void;
onFocus?: () => void;
}
export interface ParalinguisticInputRef {
focus: () => void;
element: HTMLDivElement | null;
}
export const ParalinguisticInput = forwardRef<ParalinguisticInputRef, ParalinguisticInputProps>(
function ParalinguisticInput(
{ value, onChange, placeholder, disabled, className, style, onClick, onFocus },
ref,
) {
const editorRef = useRef<HTMLDivElement>(null);
const [showMenu, setShowMenu] = useState(false);
const [menuFilter, setMenuFilter] = useState('');
const [menuIndex, setMenuIndex] = useState(0);
const [menuPosition, setMenuPosition] = useState<{ bottom: number; left: number }>({
bottom: 0,
left: 0,
});
const triggerRangeRef = useRef<Range | null>(null);
const lastSerializedRef = useRef<string>('');
const isComposingRef = useRef(false);
useImperativeHandle(ref, () => ({
focus: () => editorRef.current?.focus(),
element: editorRef.current,
}));
// Filtered tag list for the autocomplete menu
const filteredTags = PARALINGUISTIC_TAGS.filter((t) =>
t.label.toLowerCase().includes(menuFilter.toLowerCase()),
);
// ── Sync external value → editor ──────────────────────────────
useEffect(() => {
const el = editorRef.current;
if (!el) return;
// Only update DOM if the external value differs from what we last emitted
if (value !== undefined && value !== lastSerializedRef.current) {
lastSerializedRef.current = value;
el.innerHTML = value ? textToHTML(value) : '';
}
}, [value]);
// ── Emit plain-text value on input ────────────────────────────
const emitChange = useCallback(() => {
const el = editorRef.current;
if (!el || !onChange) return;
const text = htmlToText(el);
lastSerializedRef.current = text;
onChange(text);
}, [onChange]);
// ── Insert a tag badge at the caret ───────────────────────────
const insertTag = useCallback(
(tag: string) => {
const el = editorRef.current;
if (!el) return;
// Delete the /filter text
const wordRange = triggerRangeRef.current;
if (wordRange) {
wordRange.deleteContents();
}
// Insert badge HTML
const temp = document.createElement('span');
temp.innerHTML = makeBadgeHTML(tag);
const frag = document.createDocumentFragment();
let lastNode: Node | null = null;
while (temp.firstChild) {
lastNode = frag.appendChild(temp.firstChild);
}
const sel = window.getSelection();
if (sel && sel.rangeCount > 0) {
const range = sel.getRangeAt(0);
range.deleteContents();
range.insertNode(frag);
// Move caret after the badge
if (lastNode) {
const newRange = document.createRange();
newRange.setStartAfter(lastNode);
newRange.collapse(true);
sel.removeAllRanges();
sel.addRange(newRange);
}
}
setShowMenu(false);
setMenuFilter('');
emitChange();
el.focus();
},
[emitChange],
);
// ── Handle keydown for autocomplete navigation ────────────────
const handleKeyDown = useCallback(
(e: React.KeyboardEvent) => {
if (showMenu) {
if (filteredTags.length === 0) {
if (e.key === 'Escape') {
e.preventDefault();
setShowMenu(false);
}
return;
}
if (e.key === 'ArrowDown') {
e.preventDefault();
setMenuIndex((i) => (i + 1) % filteredTags.length);
} else if (e.key === 'ArrowUp') {
e.preventDefault();
setMenuIndex((i) => (i - 1 + filteredTags.length) % filteredTags.length);
} else if (e.key === 'Enter' || e.key === 'Tab') {
e.preventDefault();
if (filteredTags[menuIndex]) {
insertTag(filteredTags[menuIndex].tag);
}
} else if (e.key === 'Escape') {
e.preventDefault();
setShowMenu(false);
}
} else {
// Prevent Enter from creating <div> blocks in contentEditable
if (e.key === 'Enter' && !e.shiftKey) {
// Let the form handle submit
}
}
},
[showMenu, filteredTags, menuIndex, insertTag],
);
// ── Handle input (check for / trigger) ────────────────────────
const handleInput = useCallback(() => {
if (isComposingRef.current) return;
const el = editorRef.current;
if (!el) return;
const { word, range } = getWordBeforeCaret(el);
if (word.startsWith('/')) {
const filter = word.slice(1); // strip the /
setMenuFilter(filter);
setMenuIndex(0);
triggerRangeRef.current = range;
// Position the menu above the caret using viewport coords (portalled)
const sel = window.getSelection();
if (sel && sel.rangeCount > 0) {
const rect = sel.getRangeAt(0).getBoundingClientRect();
setMenuPosition({
bottom: window.innerHeight - rect.top + 4,
left: rect.left,
});
}
setShowMenu(true);
} else {
setShowMenu(false);
}
emitChange();
}, [emitChange]);
// ── Handle paste — convert [tag] patterns to badges ───────────
const handlePaste = useCallback(
(e: React.ClipboardEvent) => {
e.preventDefault();
const text = e.clipboardData.getData('text/plain');
if (!text) return;
const el = editorRef.current;
if (!el) return;
const html = textToHTML(text);
// Insert at caret
const sel = window.getSelection();
if (sel && sel.rangeCount > 0) {
const range = sel.getRangeAt(0);
range.deleteContents();
const temp = document.createElement('div');
temp.innerHTML = html;
const frag = document.createDocumentFragment();
let lastNode: Node | null = null;
while (temp.firstChild) {
lastNode = frag.appendChild(temp.firstChild);
}
range.insertNode(frag);
if (lastNode) {
const newRange = document.createRange();
newRange.setStartAfter(lastNode);
newRange.collapse(true);
sel.removeAllRanges();
sel.addRange(newRange);
}
}
emitChange();
},
[emitChange],
);
// ── Show placeholder ──────────────────────────────────────────
const isEmpty = !value || value.trim() === '';
return (
<div className="relative">
{/* Placeholder */}
{isEmpty && placeholder && (
<div
className="pointer-events-none absolute inset-0 text-sm text-muted-foreground/60 px-3 py-2 select-none"
aria-hidden
>
{placeholder}
</div>
)}
{/* Editable area */}
<div
ref={editorRef}
contentEditable={!disabled}
suppressContentEditableWarning
role={disabled ? undefined : 'textbox'}
aria-multiline={disabled ? undefined : true}
aria-placeholder={placeholder}
aria-disabled={disabled}
tabIndex={disabled ? -1 : 0}
className={cn(
'min-h-[32px] text-sm whitespace-pre-wrap break-words outline-none',
'[&_.ptag-badge]:inline-flex [&_.ptag-badge]:items-center [&_.ptag-badge]:rounded-full',
'[&_.ptag-badge]:bg-accent/20 [&_.ptag-badge]:text-accent [&_.ptag-badge]:border [&_.ptag-badge]:border-accent/30',
'[&_.ptag-badge]:px-2 [&_.ptag-badge]:py-0 [&_.ptag-badge]:text-xs [&_.ptag-badge]:font-medium',
'[&_.ptag-badge]:mx-0.5 [&_.ptag-badge]:select-none [&_.ptag-badge]:cursor-default',
'[&_.ptag-badge]:align-baseline',
disabled && 'opacity-50 cursor-not-allowed',
className,
)}
style={style}
onInput={!disabled ? handleInput : undefined}
onKeyDown={!disabled ? handleKeyDown : undefined}
onPaste={!disabled ? handlePaste : undefined}
onClick={!disabled ? onClick : undefined}
onFocus={!disabled ? onFocus : undefined}
onBlur={() => {
setShowMenu(false);
triggerRangeRef.current = null;
}}
onCompositionStart={() => {
isComposingRef.current = true;
}}
onCompositionEnd={() => {
isComposingRef.current = false;
handleInput();
}}
/>
{/* Autocomplete dropdown — portalled to body, positioned above the caret */}
{showMenu &&
filteredTags.length > 0 &&
createPortal(
<AnimatePresence>
<motion.div
initial={{ opacity: 0, y: 4 }}
animate={{ opacity: 1, y: 0 }}
exit={{ opacity: 0, y: 4 }}
transition={{ duration: 0.12 }}
className="fixed z-[9999] min-w-[200px] max-h-[280px] overflow-y-auto rounded-lg border border-border bg-popover shadow-lg"
style={{
bottom: menuPosition.bottom,
left: menuPosition.left,
}}
>
{filteredTags.map((t, i) => (
<button
key={t.tag}
type="button"
className={cn(
'flex items-center gap-2 w-full px-3 py-1.5 text-sm text-left transition-colors',
i === menuIndex
? 'bg-accent/20 text-accent-foreground'
: 'text-popover-foreground hover:bg-muted/50',
)}
onMouseDown={(e) => {
e.preventDefault(); // Keep focus in editor
insertTag(t.tag);
}}
onMouseEnter={() => setMenuIndex(i)}
>
<span className="text-base leading-none">{t.emoji}</span>
<span>{t.label}</span>
<span className="ml-auto text-xs text-muted-foreground font-mono">{t.tag}</span>
</button>
))}
</motion.div>
</AnimatePresence>,
document.body,
)}
</div>
);
},
);
+444 -83
View File
@@ -1,13 +1,22 @@
import { useQueryClient } from '@tanstack/react-query';
import { AnimatePresence, motion } from 'framer-motion';
import {
AlignCenter,
AudioLines,
AudioWaveform,
Download,
FileArchive,
Loader2,
MoreHorizontal,
Play,
RotateCcw,
Star,
Trash2,
Wand2,
} from 'lucide-react';
import { useEffect, useRef, useState } from 'react';
import Loader from 'react-loaders';
import { EffectsChainEditor } from '@/components/Effects/EffectsChainEditor';
import { Button } from '@/components/ui/button';
import {
Dialog,
@@ -23,10 +32,17 @@ import {
DropdownMenuItem,
DropdownMenuTrigger,
} from '@/components/ui/dropdown-menu';
import {
Select,
SelectContent,
SelectItem,
SelectTrigger,
SelectValue,
} from '@/components/ui/select';
import { Textarea } from '@/components/ui/textarea';
import { useToast } from '@/components/ui/use-toast';
import { apiClient } from '@/lib/api/client';
import type { HistoryResponse } from '@/lib/api/types';
import type { EffectConfig, GenerationVersionResponse, HistoryResponse } from '@/lib/api/types';
import { BOTTOM_SAFE_AREA_PADDING } from '@/lib/constants/ui';
import {
useDeleteGeneration,
@@ -36,7 +52,8 @@ import {
useImportGeneration,
} from '@/lib/hooks/useHistory';
import { cn } from '@/lib/utils/cn';
import { formatDate, formatDuration } from '@/lib/utils/format';
import { formatDate, formatDuration, formatEngineName } from '@/lib/utils/format';
import { useGenerationStore } from '@/stores/generationStore';
import { usePlayerStore } from '@/stores/playerStore';
// OLD TABLE-BASED COMPONENT - REMOVED (can be found in git history)
@@ -54,9 +71,21 @@ export function HistoryTable() {
const [importDialogOpen, setImportDialogOpen] = useState(false);
const [selectedFile, setSelectedFile] = useState<File | null>(null);
const [deleteDialogOpen, setDeleteDialogOpen] = useState(false);
const [generationToDelete, setGenerationToDelete] = useState<{ id: string; name: string } | null>(null);
const [generationToDelete, setGenerationToDelete] = useState<{ id: string; name: string } | null>(
null,
);
const [effectsDialogOpen, setEffectsDialogOpen] = useState(false);
const [effectsTargetId, setEffectsTargetId] = useState<string | null>(null);
const [effectsTargetVersions, setEffectsTargetVersions] = useState<GenerationVersionResponse[]>(
[],
);
const [effectsSourceVersionId, setEffectsSourceVersionId] = useState<string | null>(null);
const [effectsChain, setEffectsChain] = useState<EffectConfig[]>([]);
const [applyingEffects, setApplyingEffects] = useState(false);
const [expandedVersionsId, setExpandedVersionsId] = useState<string | null>(null);
const limit = 20;
const { toast } = useToast();
const queryClient = useQueryClient();
const {
data: historyData,
@@ -71,6 +100,7 @@ export function HistoryTable() {
const exportGeneration = useExportGeneration();
const exportGenerationAudio = useExportGenerationAudio();
const importGeneration = useImportGeneration();
const addPendingGeneration = useGenerationStore((state) => state.addPendingGeneration);
const setAudioWithAutoPlay = usePlayerStore((state) => state.setAudioWithAutoPlay);
const restartCurrentAudio = usePlayerStore((state) => state.restartCurrentAudio);
const currentAudioId = usePlayerStore((state) => state.audioId);
@@ -194,6 +224,120 @@ export function HistoryTable() {
}
};
const handleRetry = async (generationId: string) => {
try {
const result = await apiClient.retryGeneration(generationId);
addPendingGeneration(result.id);
queryClient.invalidateQueries({ queryKey: ['history'] });
} catch (error) {
toast({
title: 'Retry failed',
description: error instanceof Error ? error.message : 'Could not retry generation',
variant: 'destructive',
});
}
};
const handleRegenerate = async (generationId: string) => {
try {
await apiClient.regenerateGeneration(generationId);
addPendingGeneration(generationId);
queryClient.invalidateQueries({ queryKey: ['history'] });
} catch (error) {
toast({
title: 'Regenerate failed',
description: error instanceof Error ? error.message : 'Could not regenerate',
variant: 'destructive',
});
}
};
const handleToggleFavorite = async (generationId: string) => {
try {
await apiClient.toggleFavorite(generationId);
queryClient.invalidateQueries({ queryKey: ['history'] });
} catch (error) {
toast({
title: 'Failed to update favorite',
description: error instanceof Error ? error.message : 'Unknown error',
variant: 'destructive',
});
}
};
const handleApplyEffects = (generationId: string) => {
const gen = allHistory.find((g) => g.id === generationId);
const versions = gen?.versions ?? [];
setEffectsTargetId(generationId);
setEffectsTargetVersions(versions);
// Default to clean/original version (no effects chain)
const cleanVersion = versions.find((v) => !v.effects_chain || v.effects_chain.length === 0);
setEffectsSourceVersionId(cleanVersion?.id ?? null);
setEffectsChain([]);
setEffectsDialogOpen(true);
};
const handleApplyEffectsConfirm = async () => {
if (!effectsTargetId || effectsChain.length === 0) return;
setApplyingEffects(true);
try {
const newVersion = await apiClient.applyEffectsToGeneration(effectsTargetId, {
effects_chain: effectsChain,
source_version_id: effectsSourceVersionId ?? undefined,
set_as_default: true,
});
queryClient.invalidateQueries({ queryKey: ['history'] });
// If the player is currently on this generation, reload with the new version audio
if (currentAudioId === effectsTargetId) {
const gen = allHistory.find((g) => g.id === effectsTargetId);
if (gen) {
const versionUrl = apiClient.getVersionAudioUrl(newVersion.id);
setAudioWithAutoPlay(
versionUrl,
effectsTargetId,
gen.profile_id,
gen.text.substring(0, 50),
);
}
}
setEffectsDialogOpen(false);
toast({ title: 'Effects applied', description: 'A new version has been created.' });
} catch (error) {
toast({
title: 'Failed to apply effects',
description: error instanceof Error ? error.message : 'Unknown error',
variant: 'destructive',
});
} finally {
setApplyingEffects(false);
}
};
const handleSwitchVersion = async (generationId: string, versionId: string) => {
try {
await apiClient.setDefaultVersion(generationId, versionId);
queryClient.invalidateQueries({ queryKey: ['history'] });
} catch (error) {
toast({
title: 'Failed to switch version',
description: error instanceof Error ? error.message : 'Unknown error',
variant: 'destructive',
});
}
};
const handlePlayVersion = (
generationId: string,
versionId: string,
text: string,
profileId: string,
) => {
const audioUrl = apiClient.getVersionAudioUrl(versionId);
setAudioWithAutoPlay(audioUrl, generationId, profileId, text.substring(0, 50));
};
const handleImportConfirm = () => {
if (selectedFile) {
importGeneration.mutate(selectedFile, {
@@ -250,101 +394,266 @@ export function HistoryTable() {
>
{history.map((gen) => {
const isCurrentlyPlaying = currentAudioId === gen.id && isPlaying;
const isGenerating = gen.status === 'generating';
const isFailed = gen.status === 'failed';
const isPlayable = !isGenerating && !isFailed;
const hasVersions = gen.versions && gen.versions.length > 1;
const isVersionsExpanded = expandedVersionsId === gen.id;
return (
<div
key={gen.id}
className={cn(
'flex items-stretch gap-4 h-26 border rounded-md p-3 bg-card hover:bg-muted/70 transition-colors text-left w-full',
'border rounded-md bg-card transition-colors text-left w-full',
isCurrentlyPlaying && 'bg-muted/70',
)}
onMouseDown={(e) => {
// Don't trigger play if clicking on textarea or if text is selected
const target = e.target as HTMLElement;
if (target.closest('textarea') || window.getSelection()?.toString()) {
return;
}
handlePlay(gen.id, gen.text, gen.profile_id);
}}
>
{/* Waveform icon */}
<div className="flex items-center shrink-0">
<AudioWaveform className="h-5 w-5 text-muted-foreground" />
</div>
{/* Left side - Meta information */}
<div className="flex flex-col gap-1.5 w-48 shrink-0 justify-center">
<div className="font-medium text-sm truncate" title={gen.profile_name}>
{gen.profile_name}
</div>
<div className="flex items-center gap-2">
<span className="text-xs text-muted-foreground">{gen.language}</span>
<span className="text-xs text-muted-foreground">
{formatDuration(gen.duration)}
</span>
</div>
<div className="text-xs text-muted-foreground">
{formatDate(gen.created_at)}
</div>
</div>
{/* Right side - Transcript textarea */}
<div className="flex-1 min-w-0 flex">
<Textarea
value={gen.text}
className="flex-1 resize-none text-sm text-muted-foreground select-text"
readOnly
/>
</div>
{/* Far right - Ellipsis actions */}
{/* Main row */}
<div
className="w-10 shrink-0 flex justify-end"
onMouseDown={(e) => e.stopPropagation()}
onClick={(e) => e.stopPropagation()}
role={isPlayable ? 'button' : undefined}
tabIndex={isPlayable ? 0 : undefined}
className={cn(
'flex items-stretch gap-4 h-26 p-3',
isPlayable && 'hover:bg-muted/70 cursor-pointer rounded-md',
isVersionsExpanded && 'rounded-b-none',
)}
aria-label={
isGenerating
? `Generating speech for ${gen.profile_name}...`
: isFailed
? `Generation failed for ${gen.profile_name}`
: isCurrentlyPlaying
? `Sample from ${gen.profile_name}, ${formatDuration(gen.duration ?? 0)}, ${formatDate(gen.created_at)}. Playing. Press Enter to restart.`
: `Sample from ${gen.profile_name}, ${formatDuration(gen.duration ?? 0)}, ${formatDate(gen.created_at)}. Press Enter to play.`
}
onMouseDown={(e) => {
if (!isPlayable) return;
const target = e.target as HTMLElement;
if (target.closest('textarea') || window.getSelection()?.toString()) {
return;
}
handlePlay(gen.id, gen.text, gen.profile_id);
}}
onKeyDown={(e) => {
if (!isPlayable) return;
const target = e.target as HTMLElement;
if (target.closest('textarea') || target.closest('button')) return;
if (e.key === 'Enter' || e.key === ' ') {
e.preventDefault();
handlePlay(gen.id, gen.text, gen.profile_id);
}
}}
>
<DropdownMenu>
<DropdownMenuTrigger asChild>
{/* Status icon */}
<div className="flex items-center shrink-0 w-10 justify-center overflow-hidden">
<div className="scale-50">
<Loader
type={isGenerating ? 'line-scale' : 'line-scale-pulse-out-rapid'}
active={isGenerating || isCurrentlyPlaying}
/>
</div>
</div>
{/* Left side - Meta information */}
<div className="flex flex-col gap-1.5 w-48 shrink-0 justify-center">
<div className="font-medium text-sm truncate" title={gen.profile_name}>
{gen.profile_name}
</div>
<div className="flex items-center gap-2">
<span className="text-xs text-muted-foreground">{gen.language}</span>
<span className="text-xs text-muted-foreground">
{formatEngineName(gen.engine, gen.model_size)}
</span>
{isFailed ? (
<span className="text-xs text-destructive">Failed</span>
) : !isGenerating ? (
<span className="text-xs text-muted-foreground">
{formatDuration(gen.duration ?? 0)}
</span>
) : null}
</div>
<div className="text-xs text-muted-foreground">
{isGenerating ? (
<span className="text-accent">Generating...</span>
) : (
formatDate(gen.created_at)
)}
</div>
</div>
{/* Right side - Transcript textarea */}
<div className="flex-1 min-w-0 flex">
<Textarea
value={gen.text}
className="flex-1 resize-none text-sm text-muted-foreground select-text"
readOnly
aria-label={`Transcript for sample from ${gen.profile_name}, ${formatDuration(gen.duration ?? 0)}`}
/>
</div>
{/* Far right - Actions */}
<div
className="shrink-0 flex flex-col justify-center items-center gap-0.5"
onMouseDown={(e) => e.stopPropagation()}
onClick={(e) => e.stopPropagation()}
>
<Button
variant="ghost"
size="icon"
className={cn(
'h-6 w-6 text-muted-foreground/50 hover:bg-muted-foreground/20 hover:text-muted-foreground',
gen.is_favorited && 'text-accent hover:text-accent',
)}
aria-label={gen.is_favorited ? 'Unfavorite' : 'Favorite'}
onClick={() => handleToggleFavorite(gen.id)}
>
<Star
className="h-2 w-2"
fill={gen.is_favorited ? 'currentColor' : 'none'}
/>
</Button>
{hasVersions && (
<Button
variant="ghost"
size="icon"
className="h-8 w-8"
aria-label="Actions"
className={cn(
'h-6 w-6 text-muted-foreground/50 hover:bg-muted-foreground/20 hover:text-muted-foreground',
isVersionsExpanded && 'text-accent hover:text-accent',
)}
aria-label="Toggle versions"
onClick={() => setExpandedVersionsId(isVersionsExpanded ? null : gen.id)}
>
<MoreHorizontal className="h-4 w-4" />
<AudioLines className="h-2 w-2" />
</Button>
</DropdownMenuTrigger>
<DropdownMenuContent align="end">
<DropdownMenuItem
onClick={() => handlePlay(gen.id, gen.text, gen.profile_id)}
)}
{isFailed ? (
<Button
variant="ghost"
size="icon"
className="h-6 w-6 text-muted-foreground/50 hover:bg-muted-foreground/20 hover:text-muted-foreground"
aria-label="Retry generation"
onClick={() => handleRetry(gen.id)}
>
<Play className="mr-2 h-4 w-4" />
Play
</DropdownMenuItem>
<DropdownMenuItem
onClick={() => handleDownloadAudio(gen.id, gen.text)}
disabled={exportGenerationAudio.isPending}
>
<Download className="mr-2 h-4 w-4" />
Export Audio
</DropdownMenuItem>
<DropdownMenuItem
onClick={() => handleExportPackage(gen.id, gen.text)}
disabled={exportGeneration.isPending}
>
<FileArchive className="mr-2 h-4 w-4" />
Export Package
</DropdownMenuItem>
<DropdownMenuItem
onClick={() => handleDeleteClick(gen.id, gen.profile_name)}
disabled={deleteGeneration.isPending}
className="text-destructive focus:text-destructive"
>
<Trash2 className="mr-2 h-4 w-4" />
Delete
</DropdownMenuItem>
</DropdownMenuContent>
</DropdownMenu>
<RotateCcw className="h-2 w-2" />
</Button>
) : (
<>
<DropdownMenu>
<DropdownMenuTrigger asChild>
<Button
variant="ghost"
size="icon"
className="h-6 w-6 text-muted-foreground/50 hover:bg-muted-foreground/20 hover:text-muted-foreground"
aria-label="Actions"
disabled={isGenerating}
>
<MoreHorizontal className="h-2 w-2" />
</Button>
</DropdownMenuTrigger>
<DropdownMenuContent align="end">
<DropdownMenuItem
onClick={() => handlePlay(gen.id, gen.text, gen.profile_id)}
>
<Play className="mr-2 h-4 w-4" />
Play
</DropdownMenuItem>
<DropdownMenuItem
onClick={() => handleDownloadAudio(gen.id, gen.text)}
disabled={exportGenerationAudio.isPending}
>
<Download className="mr-2 h-4 w-4" />
Export Audio
</DropdownMenuItem>
<DropdownMenuItem
onClick={() => handleExportPackage(gen.id, gen.text)}
disabled={exportGeneration.isPending}
>
<FileArchive className="mr-2 h-4 w-4" />
Export Package
</DropdownMenuItem>
<DropdownMenuItem onClick={() => handleApplyEffects(gen.id)}>
<Wand2 className="mr-2 h-4 w-4" />
Apply Effects
</DropdownMenuItem>
<DropdownMenuItem onClick={() => handleRegenerate(gen.id)}>
<RotateCcw className="mr-2 h-4 w-4" />
Regenerate
</DropdownMenuItem>
<DropdownMenuItem
onClick={() => handleDeleteClick(gen.id, gen.profile_name)}
disabled={deleteGeneration.isPending}
// className="text-destructive focus:text-destructive"
>
<Trash2 className="mr-2 h-4 w-4" />
Delete
</DropdownMenuItem>
</DropdownMenuContent>
</DropdownMenu>
</>
)}
</div>
</div>
{/* Expandable versions panel */}
<AnimatePresence>
{isVersionsExpanded && gen.versions && (
<motion.div
initial={{ height: 0, opacity: 0 }}
animate={{ height: 'auto', opacity: 1 }}
exit={{ height: 0, opacity: 0 }}
transition={{ duration: 0.2, ease: 'easeOut' }}
className="overflow-hidden"
>
<div className="border-t border-border/50">
<div className="divide-y divide-border/40">
{gen.versions.map((v) => {
// Show source provenance when effects were applied to a non-clean version
const sourceVersion = v.source_version_id
? gen.versions?.find((sv) => sv.id === v.source_version_id)
: null;
const showSource =
sourceVersion &&
sourceVersion.effects_chain &&
sourceVersion.effects_chain.length > 0;
return (
<button
key={v.id}
type="button"
className="flex items-center gap-2 w-full h-9 px-3 text-left hover:bg-muted/50 transition-colors"
onClick={() => {
handlePlayVersion(gen.id, v.id, gen.text, gen.profile_id);
if (!v.is_default) {
handleSwitchVersion(gen.id, v.id);
}
}}
>
<AudioLines className="h-3 w-3 shrink-0 text-muted-foreground" />
<span className="truncate text-xs font-medium">{v.label}</span>
{v.effects_chain && v.effects_chain.length > 0 && (
<span className="text-[10px] text-muted-foreground truncate">
{v.effects_chain.map((e) => e.type).join(' → ')}
</span>
)}
{showSource && (
<span className="text-[10px] text-muted-foreground/60 truncate">
from {sourceVersion.label}
</span>
)}
<span className="flex-1" />
{v.is_default && (
<span className="text-[10px] bg-accent/15 text-accent px-1.5 py-0.5 rounded-full">
active
</span>
)}
</button>
);
})}
</div>
</div>
</motion.div>
)}
</AnimatePresence>
</div>
);
})}
@@ -371,7 +680,8 @@ export function HistoryTable() {
<DialogHeader>
<DialogTitle>Delete Generation</DialogTitle>
<DialogDescription>
Are you sure you want to delete this generation from "{generationToDelete?.name}"? This action cannot be undone.
Are you sure you want to delete this generation from "{generationToDelete?.name}"?
This action cannot be undone.
</DialogDescription>
</DialogHeader>
<DialogFooter>
@@ -425,6 +735,57 @@ export function HistoryTable() {
</DialogFooter>
</DialogContent>
</Dialog>
<Dialog open={effectsDialogOpen} onOpenChange={setEffectsDialogOpen}>
<DialogContent className="max-w-md">
<DialogHeader>
<DialogTitle>Apply Effects</DialogTitle>
<DialogDescription>
Configure post-processing effects to apply to this generation. A new version will be
created.
</DialogDescription>
</DialogHeader>
{effectsTargetVersions.length > 1 && (
<div className="space-y-1.5">
<label className="text-xs font-medium text-muted-foreground">Source</label>
<Select
value={effectsSourceVersionId ?? ''}
onValueChange={(val) => setEffectsSourceVersionId(val || null)}
>
<SelectTrigger className="h-8 text-xs">
<SelectValue placeholder="Select source version" />
</SelectTrigger>
<SelectContent>
{effectsTargetVersions.map((v) => (
<SelectItem key={v.id} value={v.id} className="text-xs">
{v.label}
{v.effects_chain && v.effects_chain.length > 0 && (
<span className="text-muted-foreground ml-1.5">
({v.effects_chain.map((e) => e.type).join(' + ')})
</span>
)}
</SelectItem>
))}
</SelectContent>
</Select>
</div>
)}
<div className="py-2 max-h-80 overflow-y-auto">
<EffectsChainEditor value={effectsChain} onChange={setEffectsChain} />
</div>
<DialogFooter>
<Button variant="outline" onClick={() => setEffectsDialogOpen(false)}>
Cancel
</Button>
<Button
onClick={handleApplyEffectsConfirm}
disabled={applyingEffects || effectsChain.length === 0}
>
{applyingEffects ? 'Applying...' : 'Apply'}
</Button>
</DialogFooter>
</DialogContent>
</Dialog>
</div>
);
}
+7 -7
View File
@@ -13,7 +13,7 @@ import {
} from '@/components/ui/dialog';
import { useToast } from '@/components/ui/use-toast';
import { ProfileList } from '@/components/VoiceProfiles/ProfileList';
import { BOTTOM_SAFE_AREA_PADDING } from '@/lib/constants/ui';
import { useImportProfile } from '@/lib/hooks/useProfiles';
import { cn } from '@/lib/utils/cn';
import { usePlayerStore } from '@/stores/playerStore';
@@ -77,9 +77,9 @@ export function MainEditor() {
return (
// Main view: Profiles top left, Generator bottom left, History right
<div className="grid grid-cols-1 lg:grid-cols-2 gap-6 h-full min-h-0 overflow-hidden relative">
<div className="grid grid-cols-1 lg:grid-cols-2 lg:gap-6 h-full min-h-0 overflow-hidden relative">
{/* Left Column */}
<div className="flex flex-col min-h-0 overflow-hidden relative">
<div className="flex flex-col min-h-0 overflow-hidden relative lg:overflow-hidden">
{/* Scroll Mask - Always visible, behind content */}
<div className="absolute top-0 left-0 right-0 h-16 bg-gradient-to-b from-background to-transparent z-0 pointer-events-none" />
@@ -110,10 +110,7 @@ export function MainEditor() {
{/* Scrollable Content */}
<div
ref={scrollRef}
className={cn(
'flex-1 min-h-0 overflow-y-auto pt-14',
isPlayerVisible ? BOTTOM_SAFE_AREA_PADDING : 'pb-4',
)}
className={cn('flex-1 min-h-0 overflow-y-auto pt-14 pb-4', isPlayerVisible && 'lg:pb-32')}
>
<div className="flex flex-col gap-6">
<div className="shrink-0 flex flex-col">
@@ -123,6 +120,9 @@ export function MainEditor() {
</div>
</div>
{/* Divider - single column only */}
{/* <div className="border-t border-border -my-3 lg:hidden" /> */}
{/* Right Column - History */}
<div className="flex flex-col min-h-0 overflow-hidden">
<HistoryTable />
+1 -1
View File
@@ -2,7 +2,7 @@ import { ModelManagement } from '@/components/ServerSettings/ModelManagement';
export function ModelsTab() {
return (
<div className="space-y-4 overflow-y-auto flex flex-col">
<div className="h-full flex flex-col">
<ModelManagement />
</div>
);
@@ -1,9 +1,12 @@
import { zodResolver } from '@hookform/resolvers/zod';
import { Loader2, XCircle } from 'lucide-react';
import { useEffect } from 'react';
import { useForm } from 'react-hook-form';
import * as z from 'zod';
import { Badge } from '@/components/ui/badge';
import { Button } from '@/components/ui/button';
import { Card, CardContent, CardHeader, CardTitle } from '@/components/ui/card';
import { Checkbox } from '@/components/ui/checkbox';
import {
Form,
FormControl,
@@ -14,10 +17,10 @@ import {
FormMessage,
} from '@/components/ui/form';
import { Input } from '@/components/ui/input';
import { Checkbox } from '@/components/ui/checkbox';
import { useToast } from '@/components/ui/use-toast';
import { useServerStore } from '@/stores/serverStore';
import { useServerHealth } from '@/lib/hooks/useServer';
import { usePlatform } from '@/platform/PlatformContext';
import { useServerStore } from '@/stores/serverStore';
const connectionSchema = z.object({
serverUrl: z.string().url('Please enter a valid URL'),
@@ -31,7 +34,10 @@ export function ConnectionForm() {
const setServerUrl = useServerStore((state) => state.setServerUrl);
const keepServerRunningOnClose = useServerStore((state) => state.keepServerRunningOnClose);
const setKeepServerRunningOnClose = useServerStore((state) => state.setKeepServerRunningOnClose);
const mode = useServerStore((state) => state.mode);
const setMode = useServerStore((state) => state.setMode);
const { toast } = useToast();
const { data: health, isLoading, error: healthError } = useServerHealth();
const form = useForm<ConnectionFormValues>({
resolver: zodResolver(connectionSchema),
@@ -49,7 +55,7 @@ export function ConnectionForm() {
function onSubmit(data: ConnectionFormValues) {
setServerUrl(data.serverUrl);
form.reset(data); // Reset form state after successful submission
form.reset(data);
toast({
title: 'Server URL updated',
description: `Connected to ${data.serverUrl}`,
@@ -57,7 +63,7 @@ export function ConnectionForm() {
}
return (
<Card>
<Card role="region" aria-label="Server Connection" tabIndex={0}>
<CardHeader>
<CardTitle>Server Connection</CardTitle>
</CardHeader>
@@ -83,10 +89,42 @@ export function ConnectionForm() {
</form>
</Form>
{/* Connection status */}
<div className="mt-4">
{isLoading ? (
<div className="flex items-center gap-2">
<Loader2 className="h-4 w-4 animate-spin" />
<span className="text-sm text-muted-foreground">Checking connection...</span>
</div>
) : healthError ? (
<div className="flex items-center gap-2">
<XCircle className="h-4 w-4 text-destructive" />
<span className="text-sm text-destructive">
Connection failed: {healthError.message}
</span>
</div>
) : health ? (
<div className="flex flex-wrap gap-2">
<Badge
variant={health.model_loaded || health.model_downloaded ? 'default' : 'secondary'}
>
{health.model_loaded || health.model_downloaded ? 'Model Ready' : 'No Model'}
</Badge>
<Badge variant={health.gpu_available ? 'default' : 'secondary'}>
GPU: {health.gpu_available ? 'Available' : 'Not Available'}
</Badge>
{health.vram_used_mb != null && health.vram_used_mb > 0 && (
<Badge variant="outline">VRAM: {health.vram_used_mb.toFixed(0)} MB</Badge>
)}
</div>
) : null}
</div>
<div className="mt-6 pt-6 border-t">
<div className="flex items-start space-x-3">
<Checkbox
id="keepServerRunning"
className="mt-[6px]"
checked={keepServerRunningOnClose}
onCheckedChange={(checked: boolean) => {
setKeepServerRunningOnClose(checked);
@@ -115,6 +153,39 @@ export function ConnectionForm() {
</div>
</div>
</div>
{platform.metadata.isTauri && (
<div className="mt-6 pt-6 border-t">
<div className="flex items-start space-x-3">
<Checkbox
id="allowNetworkAccess"
className="mt-[6px]"
checked={mode === 'remote'}
onCheckedChange={(checked: boolean) => {
setMode(checked ? 'remote' : 'local');
toast({
title: 'Setting updated',
description: checked
? 'Network access enabled. Restart the app to apply.'
: 'Network access disabled. Restart the app to apply.',
});
}}
/>
<div className="space-y-1">
<label
htmlFor="allowNetworkAccess"
className="text-sm font-medium leading-none peer-disabled:cursor-not-allowed peer-disabled:opacity-70 cursor-pointer"
>
Allow network access
</label>
<p className="text-sm text-muted-foreground">
Makes the server accessible from other devices on your network. Restart the app
after changing this setting.
</p>
</div>
</div>
</div>
)}
</CardContent>
</Card>
);
@@ -0,0 +1,116 @@
import { Card, CardContent, CardDescription, CardHeader, CardTitle } from '@/components/ui/card';
import { Checkbox } from '@/components/ui/checkbox';
import { Slider } from '@/components/ui/slider';
import { useServerStore } from '@/stores/serverStore';
export function GenerationSettings() {
const maxChunkChars = useServerStore((state) => state.maxChunkChars);
const setMaxChunkChars = useServerStore((state) => state.setMaxChunkChars);
const crossfadeMs = useServerStore((state) => state.crossfadeMs);
const setCrossfadeMs = useServerStore((state) => state.setCrossfadeMs);
const normalizeAudio = useServerStore((state) => state.normalizeAudio);
const setNormalizeAudio = useServerStore((state) => state.setNormalizeAudio);
const autoplayOnGenerate = useServerStore((state) => state.autoplayOnGenerate);
const setAutoplayOnGenerate = useServerStore((state) => state.setAutoplayOnGenerate);
return (
<Card role="region" aria-label="Generation Settings" tabIndex={0}>
<CardHeader>
<CardTitle>Generation Settings</CardTitle>
<CardDescription>
Controls for long text generation. These settings apply to all engines.
</CardDescription>
</CardHeader>
<CardContent>
<div className="space-y-6">
<div className="space-y-3">
<div className="flex items-center justify-between">
<label htmlFor="maxChunkChars" className="text-sm font-medium leading-none">
Auto-chunking limit
</label>
<span className="text-sm tabular-nums text-muted-foreground">
{maxChunkChars} chars
</span>
</div>
<Slider
id="maxChunkChars"
value={[maxChunkChars]}
onValueChange={([value]) => setMaxChunkChars(value)}
min={100}
max={5000}
step={50}
aria-label="Auto-chunking character limit"
/>
<p className="text-sm text-muted-foreground">
Long text is split into chunks at sentence boundaries before generating. Lower values
can improve quality for long outputs.
</p>
</div>
<div className="space-y-3">
<div className="flex items-center justify-between">
<label htmlFor="crossfadeMs" className="text-sm font-medium leading-none">
Chunk crossfade
</label>
<span className="text-sm tabular-nums text-muted-foreground">
{crossfadeMs === 0 ? 'Cut' : `${crossfadeMs}ms`}
</span>
</div>
<Slider
id="crossfadeMs"
value={[crossfadeMs]}
onValueChange={([value]) => setCrossfadeMs(value)}
min={0}
max={200}
step={10}
aria-label="Chunk crossfade duration"
/>
<p className="text-sm text-muted-foreground">
Blends audio between chunks to smooth transitions. Set to 0 for a hard cut.
</p>
</div>
<div className="flex items-start gap-3">
<Checkbox
id="normalizeAudio"
checked={normalizeAudio}
onCheckedChange={setNormalizeAudio}
className="mt-[6px]"
/>
<div className="space-y-1">
<label
htmlFor="normalizeAudio"
className="text-sm font-medium leading-none cursor-pointer"
>
Normalize audio
</label>
<p className="text-sm text-muted-foreground">
Adjusts output volume to a consistent level across generations.
</p>
</div>
</div>
<div className="flex items-start gap-3">
<Checkbox
id="autoplayOnGenerate"
checked={autoplayOnGenerate}
onCheckedChange={setAutoplayOnGenerate}
className="mt-[6px]"
/>
<div className="space-y-1">
<label
htmlFor="autoplayOnGenerate"
className="text-sm font-medium leading-none cursor-pointer"
>
Autoplay on generate
</label>
<p className="text-sm text-muted-foreground">
Automatically play audio when a generation completes.
</p>
</div>
</div>
</div>
</CardContent>
</Card>
);
}
@@ -1,7 +1,6 @@
import { useQuery, useQueryClient } from '@tanstack/react-query';
import { AlertCircle, Cpu, Download, Loader2, RotateCw, Trash2, Zap } from 'lucide-react';
import { AlertCircle, Download, Loader2, RotateCw, Trash2 } from 'lucide-react';
import { useCallback, useEffect, useRef, useState } from 'react';
import { Badge } from '@/components/ui/badge';
import { Button } from '@/components/ui/button';
import { Card, CardContent, CardHeader, CardTitle } from '@/components/ui/card';
import { Progress } from '@/components/ui/progress';
@@ -27,12 +26,12 @@ export function GpuAcceleration() {
// Query CUDA backend status
const {
data: cudaStatus,
isLoading: cudaStatusLoading,
isLoading: _cudaStatusLoading,
refetch: refetchCudaStatus,
} = useQuery({
queryKey: ['cuda-status', serverUrl],
queryFn: () => apiClient.getCudaStatus(),
refetchInterval: cudaStatusLoading ? false : 10000,
refetchInterval: (query) => (query.state.status === 'pending' ? false : 10000),
retry: 1,
enabled: !!health, // Only fetch when backend is reachable
});
@@ -216,58 +215,36 @@ export function GpuAcceleration() {
return (
<Card>
<CardHeader>
<CardTitle className="flex items-center gap-2">
<Zap className="h-4 w-4" />
GPU Acceleration
</CardTitle>
<CardTitle>GPU Acceleration</CardTitle>
</CardHeader>
<CardContent className="space-y-4">
{/* Current status */}
<div className="flex items-center justify-between">
<div className="space-y-1">
<div className="text-sm font-medium">Backend</div>
<div className="text-sm text-muted-foreground">
{isCurrentlyCuda ? 'CUDA (GPU accelerated)' : 'CPU'}
</div>
</div>
<Badge variant={isCurrentlyCuda ? 'default' : 'secondary'}>
{isCurrentlyCuda ? (
<>
<Zap className="h-3 w-3 mr-1" /> CUDA
</>
) : (
<>
<Cpu className="h-3 w-3 mr-1" /> CPU
</>
)}
</Badge>
{/* GPU status */}
<div className="space-y-1">
{health.gpu_available && health.gpu_type ? (
<>
<div className="text-sm font-medium">
{health.gpu_type.replace(/^(CUDA|ROCm|MPS|Metal|XPU|DirectML)\s*\((.+)\)$/, '$2') ||
health.gpu_type}
</div>
<div className="text-sm text-muted-foreground">
{health.gpu_type.replace(/\s*\(.+\)$/, '')}
{health.vram_used_mb != null && health.vram_used_mb > 0
? ` \u00b7 ${health.vram_used_mb.toFixed(0)} MB VRAM used`
: ''}
</div>
</>
) : (
<>
<div className="text-sm font-medium">CPU</div>
<div className="text-sm text-muted-foreground">No GPU acceleration available</div>
</>
)}
</div>
{/* GPU info from health */}
{health.gpu_type && (
<div className="space-y-1">
<div className="text-sm font-medium">GPU</div>
<div className="text-sm text-muted-foreground">{health.gpu_type}</div>
{health.vram_used_mb != null && (
<div className="text-xs text-muted-foreground">
VRAM: {health.vram_used_mb.toFixed(0)} MB used
</div>
)}
</div>
)}
{/* Native GPU detected - no CUDA download needed */}
{hasNativeGpu && (
<div className="p-3 rounded-lg bg-accent/10 border border-accent/20">
<div className="text-sm">
Your system uses <strong>{health.gpu_type}</strong> for acceleration. No additional
downloads needed.
</div>
</div>
)}
{/* CUDA download section - only show when native GPU is NOT detected (i.e., Windows/Linux NVIDIA users) */}
{!hasNativeGpu && (
{/* CUDA download section - only show when no GPU is active (native or CUDA) */}
{!hasNativeGpu && !isCurrentlyCuda && (
<>
{/* Download progress */}
{cudaDownloading && downloadProgress && (
File diff suppressed because it is too large Load Diff
@@ -3,14 +3,13 @@ import { Badge } from '@/components/ui/badge';
import { Card, CardContent, CardHeader, CardTitle } from '@/components/ui/card';
import { useServerHealth } from '@/lib/hooks/useServer';
import { useServerStore } from '@/stores/serverStore';
import { ModelProgress } from './ModelProgress';
export function ServerStatus() {
const { data: health, isLoading, error } = useServerHealth();
const serverUrl = useServerStore((state) => state.serverUrl);
return (
<Card>
<Card role="region" aria-label="Server Status" tabIndex={0}>
<CardHeader>
<CardTitle>Server Status</CardTitle>
</CardHeader>
@@ -20,16 +19,6 @@ export function ServerStatus() {
<div className="font-mono text-sm">{serverUrl}</div>
</div>
{/* Model download progress */}
<div className="space-y-2">
<ModelProgress modelName="qwen-tts-1.7B" displayName="Qwen TTS 1.7B" />
<ModelProgress modelName="qwen-tts-0.6B" displayName="Qwen TTS 0.6B" />
<ModelProgress modelName="whisper-base" displayName="Whisper Base" />
<ModelProgress modelName="whisper-small" displayName="Whisper Small" />
<ModelProgress modelName="whisper-medium" displayName="Whisper Medium" />
<ModelProgress modelName="whisper-large" displayName="Whisper Large" />
</div>
{isLoading ? (
<div className="flex items-center gap-2">
<Loader2 className="h-4 w-4 animate-spin" />
@@ -11,6 +11,7 @@ export function UpdateStatus() {
const platform = usePlatform();
const { status, checkForUpdates, downloadAndInstall, restartAndInstall } = useAutoUpdater(false);
const [currentVersion, setCurrentVersion] = useState<string>('');
const isDev = !import.meta.env?.PROD;
useEffect(() => {
platform.metadata
@@ -20,7 +21,7 @@ export function UpdateStatus() {
}, [platform]);
return (
<Card>
<Card role="region" aria-label="App Updates" tabIndex={0}>
<CardHeader>
<CardTitle>App Updates</CardTitle>
</CardHeader>
@@ -28,97 +29,110 @@ export function UpdateStatus() {
<div className="flex items-center justify-between">
<div className="space-y-1">
<div className="text-sm font-medium">Current Version</div>
<div className="text-sm text-muted-foreground">v{currentVersion}</div>
<div className="text-sm text-muted-foreground">
v{currentVersion}
{isDev ? ' (dev)' : ''}
</div>
</div>
<Button
onClick={checkForUpdates}
disabled={status.checking || status.downloading || status.readyToInstall}
variant="outline"
size="sm"
>
<RefreshCw className={`h-4 w-4 mr-2 ${status.checking ? 'animate-spin' : ''}`} />
Check for Updates
</Button>
{!isDev && (
<Button
onClick={checkForUpdates}
disabled={status.checking || status.downloading || status.readyToInstall}
variant="outline"
size="sm"
>
<RefreshCw className={`h-4 w-4 mr-2 ${status.checking ? 'animate-spin' : ''}`} />
Check for Updates
</Button>
)}
</div>
{status.checking && (
<div className="flex items-center gap-2 text-sm text-muted-foreground">
<RefreshCw className="h-4 w-4 animate-spin" />
Checking for updates...
{isDev ? (
<div className="text-sm text-muted-foreground">
Auto-updates are disabled in development mode.
</div>
)}
{status.error && (
<div className="flex items-center gap-2 text-sm text-destructive">
<AlertCircle className="h-4 w-4" />
{status.error}
</div>
)}
{status.available && !status.downloading && !status.readyToInstall && (
<div className="space-y-3 p-4 border rounded-lg bg-primary/5">
<div className="flex items-center justify-between">
<div>
<div className="font-semibold">Update Available</div>
<div className="text-sm text-muted-foreground">Version {status.version}</div>
) : (
<>
{status.checking && (
<div className="flex items-center gap-2 text-sm text-muted-foreground">
<RefreshCw className="h-4 w-4 animate-spin" />
Checking for updates...
</div>
<Badge>New</Badge>
</div>
<Button onClick={downloadAndInstall} className="w-full" size="sm">
<Download className="h-4 w-4 mr-2" />
Download Update
</Button>
</div>
)}
)}
{status.downloading && (
<div className="space-y-2">
<div className="flex items-center justify-between text-sm">
<div className="flex items-center gap-2">
<Download className="h-4 w-4" />
Downloading update...
{status.error && (
<div className="flex items-center gap-2 text-sm text-destructive">
<AlertCircle className="h-4 w-4" />
{status.error}
</div>
{status.downloadProgress !== undefined && (
<span className="text-muted-foreground">{status.downloadProgress}%</span>
)}
</div>
<Progress value={status.downloadProgress} />
{status.downloadedBytes !== undefined &&
status.totalBytes !== undefined &&
status.totalBytes > 0 && (
<div className="text-xs text-muted-foreground">
{(status.downloadedBytes / 1024 / 1024).toFixed(1)} MB /{' '}
{(status.totalBytes / 1024 / 1024).toFixed(1)} MB
)}
{status.available && !status.downloading && !status.readyToInstall && (
<div className="space-y-3 p-4 border rounded-lg bg-primary/5">
<div className="flex items-center justify-between">
<div>
<div className="font-semibold">Update Available</div>
<div className="text-sm text-muted-foreground">Version {status.version}</div>
</div>
<Badge>New</Badge>
</div>
)}
</div>
)}
<Button onClick={downloadAndInstall} className="w-full" size="sm">
<Download className="h-4 w-4 mr-2" />
Download Update
</Button>
</div>
)}
{status.readyToInstall && (
<div className="space-y-3 p-4 border rounded-lg bg-accent/30 border-accent/50">
<div className="flex items-center gap-2">
<div>
<div className="font-semibold">Update Ready to Install</div>
{status.downloading && (
<div className="space-y-2">
<div className="flex items-center justify-between text-sm">
<div className="flex items-center gap-2">
<Download className="h-4 w-4" />
Downloading update...
</div>
{status.downloadProgress !== undefined && (
<span className="text-muted-foreground">{status.downloadProgress}%</span>
)}
</div>
<Progress value={status.downloadProgress} />
{status.downloadedBytes !== undefined &&
status.totalBytes !== undefined &&
status.totalBytes > 0 && (
<div className="text-xs text-muted-foreground">
{(status.downloadedBytes / 1024 / 1024).toFixed(1)} MB /{' '}
{(status.totalBytes / 1024 / 1024).toFixed(1)} MB
</div>
)}
</div>
)}
{status.readyToInstall && (
<div className="space-y-3 p-4 border rounded-lg bg-accent/30 border-accent/50">
<div className="flex items-center gap-2">
<div>
<div className="font-semibold">Update Ready to Install</div>
<div className="text-sm text-muted-foreground">
Version {status.version} has been downloaded
</div>
</div>
</div>
<div className="text-sm text-muted-foreground">
Version {status.version} has been downloaded
The app needs to restart to complete the installation. You can do this now or
later at your convenience.
</div>
<Button onClick={restartAndInstall} className="w-full" size="sm">
<RefreshCw className="h-4 w-4 mr-2" />
Restart Now
</Button>
</div>
</div>
<div className="text-sm text-muted-foreground">
The app needs to restart to complete the installation. You can do this now or later at
your convenience.
</div>
<Button onClick={restartAndInstall} className="w-full" size="sm">
<RefreshCw className="h-4 w-4 mr-2" />
Restart Now
</Button>
</div>
)}
)}
{!status.available && !status.checking && !status.error && status.checking === false && (
<div className="flex items-center gap-2 text-sm text-muted-foreground">
You're up to date
</div>
{!status.available && !status.checking && !status.error && (
<div className="flex items-center gap-2 text-sm text-muted-foreground">
You're up to date
</div>
)}
</>
)}
</CardContent>
</Card>
+11 -5
View File
@@ -1,19 +1,25 @@
import { ConnectionForm } from '@/components/ServerSettings/ConnectionForm';
import { GenerationSettings } from '@/components/ServerSettings/GenerationSettings';
import { GpuAcceleration } from '@/components/ServerSettings/GpuAcceleration';
import { ServerStatus } from '@/components/ServerSettings/ServerStatus';
import { UpdateStatus } from '@/components/ServerSettings/UpdateStatus';
import { BOTTOM_SAFE_AREA_PADDING } from '@/lib/constants/ui';
import { cn } from '@/lib/utils/cn';
import { usePlatform } from '@/platform/PlatformContext';
import { usePlayerStore } from '@/stores/playerStore';
export function ServerTab() {
const platform = usePlatform();
const isPlayerVisible = !!usePlayerStore((state) => state.audioUrl);
return (
<div className="space-y-4 overflow-y-auto flex flex-col">
<div
className={cn('overflow-y-auto flex flex-col', isPlayerVisible && BOTTOM_SAFE_AREA_PADDING)}
>
<div className="grid gap-4 md:grid-cols-2">
<ConnectionForm />
<ServerStatus />
<GenerationSettings />
{platform.metadata.isTauri && <GpuAcceleration />}
{platform.metadata.isTauri && <UpdateStatus />}
</div>
{platform.metadata.isTauri && <GpuAcceleration />}
{platform.metadata.isTauri && <UpdateStatus />}
<div className="py-8 text-center text-sm text-muted-foreground">
Created by{' '}
<a
+37 -28
View File
@@ -1,9 +1,9 @@
import { Link, useMatchRoute } from '@tanstack/react-router';
import { Box, BookOpen, Loader2, Mic, Server, Speaker, Volume2 } from 'lucide-react';
import { AudioLines, Box, Mic, Server, Speaker, Volume2, Wand2 } from 'lucide-react';
import voiceboxLogo from '@/assets/voicebox-logo.png';
import { cn } from '@/lib/utils/cn';
import { useGenerationStore } from '@/stores/generationStore';
import { usePlayerStore } from '@/stores/playerStore';
import { version } from '../../package.json';
interface SidebarProps {
isMacOS?: boolean;
@@ -11,18 +11,17 @@ interface SidebarProps {
const tabs = [
{ id: 'main', path: '/', icon: Volume2, label: 'Generate' },
{ id: 'stories', path: '/stories', icon: BookOpen, label: 'Stories' },
{ id: 'stories', path: '/stories', icon: AudioLines, label: 'Stories' },
{ id: 'voices', path: '/voices', icon: Mic, label: 'Voices' },
{ id: 'effects', path: '/effects', icon: Wand2, label: 'Effects' },
{ id: 'audio', path: '/audio', icon: Speaker, label: 'Audio' },
{ id: 'models', path: '/models', icon: Box, label: 'Models' },
{ id: 'server', path: '/server', icon: Server, label: 'Server' },
];
export function Sidebar({ isMacOS }: SidebarProps) {
const isGenerating = useGenerationStore((state) => state.isGenerating);
const audioUrl = usePlayerStore((state) => state.audioUrl);
const isPlayerVisible = !!audioUrl;
const matchRoute = useMatchRoute();
const isPlayerOpen = !!usePlayerStore((s) => s.audioUrl);
return (
<div
@@ -33,7 +32,15 @@ export function Sidebar({ isMacOS }: SidebarProps) {
>
{/* Logo */}
<div className="mb-2">
<img src={voiceboxLogo} alt="Voicebox" className="w-12 h-12 object-contain" />
<img
src={voiceboxLogo}
alt="Voicebox"
className="w-12 h-12 object-contain"
style={{
filter:
'drop-shadow(0 0 6px hsl(var(--accent) / 0.5)) drop-shadow(0 0 14px hsl(var(--accent) / 0.35)) drop-shadow(0 0 28px hsl(var(--accent) / 0.2))',
}}
/>
</div>
{/* Navigation Buttons */}
@@ -42,42 +49,44 @@ export function Sidebar({ isMacOS }: SidebarProps) {
const Icon = tab.icon;
// For index route, use exact match; for others, use default matching
const isActive =
tab.path === '/'
? matchRoute({ to: '/', exact: true })
: matchRoute({ to: tab.path });
tab.path === '/' ? matchRoute({ to: '/', exact: true }) : matchRoute({ to: tab.path });
return (
<Link
key={tab.id}
to={tab.path}
className={cn(
'w-12 h-12 rounded-full flex items-center justify-center transition-all duration-200',
'hover:bg-muted/50',
isActive ? 'bg-muted/50 text-foreground shadow-lg' : 'text-muted-foreground',
'relative w-12 h-12 rounded-full flex items-center justify-center transition-all duration-200 overflow-hidden',
isActive
? 'bg-white/[0.07] text-foreground shadow-lg backdrop-blur-sm border border-white/[0.08]'
: 'text-muted-foreground hover:bg-muted/50',
)}
title={tab.label}
aria-label={tab.label}
>
<Icon className="h-5 w-5" />
{isActive && (
<div
className="absolute inset-0 rounded-full pointer-events-none"
style={{
maskImage: 'linear-gradient(to bottom, black, transparent 60%)',
WebkitMaskImage: 'linear-gradient(to bottom, black, transparent 60%)',
border: '1px solid hsl(var(--accent) / 0.5)',
}}
/>
)}
<Icon className="h-5 w-5 relative z-10" />
</Link>
);
})}
</div>
{/* Spacer to push loader to bottom */}
<div className="flex-1" />
{/* Generation Loader */}
{isGenerating && (
<div
className={cn(
'w-full flex items-center justify-center transition-all duration-200',
isPlayerVisible ? 'mb-[120px]' : 'mb-0',
)}
>
<Loader2 className="h-6 w-6 text-accent animate-spin" />
</div>
)}
{/* Version */}
<div
className="mt-auto text-[10px] text-muted-foreground/50 transition-all duration-300"
style={{ paddingBottom: isPlayerOpen ? '7rem' : undefined }}
>
v{version}
</div>
</div>
);
}
+4 -1
View File
@@ -1,8 +1,11 @@
import { FloatingGenerateBox } from '@/components/Generation/FloatingGenerateBox';
import { usePlayerStore } from '@/stores/playerStore';
import { StoryContent } from './StoryContent';
import { StoryList } from './StoryList';
export function StoriesTab() {
const audioUrl = usePlayerStore((state) => state.audioUrl);
return (
<div className="flex flex-col h-full min-h-0 overflow-hidden">
{/* Main content area */}
@@ -18,7 +21,7 @@ export function StoriesTab() {
</div>
{/* Floating Generate Box - position is managed via storyStore.trackEditorHeight */}
<FloatingGenerateBox showVoiceSelector />
<FloatingGenerateBox showVoiceSelector isPlayerOpen={!!audioUrl} />
</div>
</div>
);
+33 -6
View File
@@ -13,8 +13,11 @@ import {
sortableKeyboardCoordinates,
verticalListSortingStrategy,
} from '@dnd-kit/sortable';
import { Link } from '@tanstack/react-router';
import { AnimatePresence, motion } from 'framer-motion';
import { Download, Plus } from 'lucide-react';
import { useEffect, useMemo, useRef, useState } from 'react';
import Loader from 'react-loaders';
import { Button } from '@/components/ui/button';
import { Input } from '@/components/ui/input';
import { Popover, PopoverContent, PopoverTrigger } from '@/components/ui/popover';
@@ -28,6 +31,7 @@ import {
useStory,
} from '@/lib/hooks/useStories';
import { useStoryPlayback } from '@/lib/hooks/useStoryPlayback';
import { useGenerationStore } from '@/stores/generationStore';
import { useStoryStore } from '@/stores/storyStore';
import { SortableStoryChatItem } from './StoryChatItem';
@@ -40,6 +44,7 @@ export function StoryContent() {
const addStoryItem = useAddStoryItem();
const { toast } = useToast();
const scrollRef = useRef<HTMLDivElement>(null);
const pendingCount = useGenerationStore((s) => s.pendingGenerationIds.size);
// Add generation popover state
const [searchQuery, setSearchQuery] = useState('');
@@ -53,9 +58,9 @@ export function StoryContent() {
const query = searchQuery.toLowerCase();
return historyData.items.filter(
(gen) =>
gen.status === 'completed' &&
!storyGenerationIds.has(gen.id) &&
(gen.text.toLowerCase().includes(query) ||
gen.profile_name.toLowerCase().includes(query)),
(gen.text.toLowerCase().includes(query) || gen.profile_name.toLowerCase().includes(query)),
);
}, [historyData, story, searchQuery]);
@@ -267,7 +272,31 @@ export function StoryContent() {
<p className="text-sm text-muted-foreground mt-1">{story.description}</p>
)}
</div>
<div className="flex gap-2">
<div className="flex gap-2 items-center">
<AnimatePresence>
{pendingCount > 0 && (
<motion.div
initial={{ opacity: 0, scale: 0.9, width: 0 }}
animate={{ opacity: 1, scale: 1, width: 'auto' }}
exit={{ opacity: 0, scale: 0.9, width: 0 }}
transition={{ duration: 0.2 }}
>
<Link
to="/"
className="flex items-center gap-2 h-8 pl-1.5 pr-3 rounded-full bg-card border border-border hover:bg-muted/50 transition-all duration-200 cursor-pointer"
>
<div className="shrink-0 w-10 h-5 overflow-hidden flex items-center justify-center">
<div className="scale-[0.45]">
<Loader type="line-scale" active />
</div>
</div>
<span className="text-xs text-muted-foreground whitespace-nowrap">
Generating {pendingCount} {pendingCount === 1 ? 'audio' : 'audios'}
</span>
</Link>
</motion.div>
)}
</AnimatePresence>
<Popover open={isAddOpen} onOpenChange={setIsAddOpen}>
<PopoverTrigger asChild>
<Button variant="outline" size="sm">
@@ -287,9 +316,7 @@ export function StoryContent() {
<div className="max-h-60 overflow-y-auto">
{availableGenerations.length === 0 ? (
<div className="p-4 text-center text-sm text-muted-foreground">
{searchQuery
? 'No matching generations found'
: 'No available generations'}
{searchQuery ? 'No matching generations found' : 'No available generations'}
</div>
) : (
availableGenerations.map((gen) => (
+97 -67
View File
@@ -1,5 +1,5 @@
import { Plus, BookOpen, MoreHorizontal, Pencil, Trash2 } from 'lucide-react';
import { useState } from 'react';
import { BookOpen, MoreHorizontal, Pencil, Plus, Trash2 } from 'lucide-react';
import { useEffect, useState } from 'react';
import {
AlertDialog,
AlertDialogAction,
@@ -29,7 +29,13 @@ import { Input } from '@/components/ui/input';
import { Label } from '@/components/ui/label';
import { Textarea } from '@/components/ui/textarea';
import { useToast } from '@/components/ui/use-toast';
import { useStories, useCreateStory, useUpdateStory, useDeleteStory } from '@/lib/hooks/useStories';
import {
useCreateStory,
useDeleteStory,
useStories,
useStory,
useUpdateStory,
} from '@/lib/hooks/useStories';
import { cn } from '@/lib/utils/cn';
import { formatDate } from '@/lib/utils/format';
import { useStoryStore } from '@/stores/storyStore';
@@ -38,6 +44,8 @@ export function StoryList() {
const { data: stories, isLoading } = useStories();
const selectedStoryId = useStoryStore((state) => state.selectedStoryId);
const setSelectedStoryId = useStoryStore((state) => state.setSelectedStoryId);
const trackEditorHeight = useStoryStore((state) => state.trackEditorHeight);
const { data: selectedStory } = useStory(selectedStoryId);
const createStory = useCreateStory();
const updateStory = useUpdateStory();
const deleteStory = useDeleteStory();
@@ -54,6 +62,13 @@ export function StoryList() {
const [newStoryDescription, setNewStoryDescription] = useState('');
const { toast } = useToast();
// Auto-select the first story when the list loads with no selection
useEffect(() => {
if (!selectedStoryId && stories && stories.length > 0) {
setSelectedStoryId(stories[0].id);
}
}, [selectedStoryId, stories, setSelectedStoryId]);
const handleCreateStory = () => {
if (!newStoryName.trim()) {
toast({
@@ -170,20 +185,29 @@ export function StoryList() {
}
const storyList = stories || [];
const hasTrackEditor = selectedStoryId && selectedStory && selectedStory.items.length > 0;
return (
<div className="flex flex-col h-full min-h-0">
{/* Header */}
<div className="flex items-center justify-between mb-4 px-1">
<h2 className="text-2xl font-bold">Stories</h2>
<Button onClick={() => setCreateDialogOpen(true)} size="sm">
<Plus className="mr-2 h-4 w-4" />
New Story
</Button>
<div className="h-full flex flex-col relative overflow-hidden">
{/* Scroll Mask */}
<div className="absolute top-0 left-0 right-0 h-16 bg-gradient-to-b from-background to-transparent z-10 pointer-events-none" />
{/* Fixed Header */}
<div className="absolute top-0 left-0 right-0 z-20">
<div className="flex items-center justify-between mb-4 px-1">
<h2 className="text-2xl font-bold">Stories</h2>
<Button onClick={() => setCreateDialogOpen(true)} size="sm">
<Plus className="mr-2 h-4 w-4" />
New Story
</Button>
</div>
</div>
{/* Story List */}
<div className="flex-1 min-h-0 overflow-y-auto space-y-2">
{/* Scrollable Story List */}
<div
className="flex-1 overflow-y-auto pt-14 relative z-0"
style={{ paddingBottom: hasTrackEditor ? `${trackEditorHeight + 140}px` : '170px' }}
>
{storyList.length === 0 ? (
<div className="text-center py-12 px-5 border-2 border-dashed border-muted rounded-2xl text-muted-foreground">
<BookOpen className="h-12 w-12 mx-auto mb-4 opacity-50" />
@@ -191,62 +215,68 @@ export function StoryList() {
<p className="text-xs mt-2">Create your first story to get started</p>
</div>
) : (
storyList.map((story) => (
<div
key={story.id}
className={cn(
'h-24 p-4 border rounded-2xl transition-colors group flex items-center',
selectedStoryId === story.id && 'bg-muted border-primary',
)}
>
<div className="flex items-start justify-between gap-2 w-full min-w-0">
<button
type="button"
className="flex-1 min-w-0 text-left cursor-pointer overflow-hidden"
onClick={() => setSelectedStoryId(story.id)}
>
<h3 className="font-medium truncate">{story.name}</h3>
{story.description && (
<p className="text-sm text-muted-foreground mt-1 truncate">
{story.description}
</p>
)}
<div className="flex items-center gap-3 mt-2 text-xs text-muted-foreground">
<span>
{story.item_count} {story.item_count === 1 ? 'item' : 'items'}
</span>
<span>•</span>
<span>{formatDate(story.updated_at)}</span>
<div className="space-y-0.5">
{storyList.map((story) => (
<div
key={story.id}
role="button"
tabIndex={0}
className={cn(
'px-5 py-3 rounded-lg transition-colors group flex items-center cursor-pointer',
selectedStoryId === story.id ? 'bg-muted' : 'hover:bg-muted/50',
)}
aria-label={`Story ${story.name}, ${story.item_count} ${story.item_count === 1 ? 'item' : 'items'}, ${formatDate(story.updated_at)}`}
aria-pressed={selectedStoryId === story.id}
onClick={() => setSelectedStoryId(story.id)}
onKeyDown={(e) => {
if (e.target !== e.currentTarget) return;
if (e.key === 'Enter' || e.key === ' ') {
e.preventDefault();
setSelectedStoryId(story.id);
}
}}
>
<div className="flex items-start justify-between gap-2 w-full min-w-0">
<div className="flex-1 min-w-0 text-left overflow-hidden">
<h3 className="text-sm font-medium truncate">{story.name}</h3>
<div className="flex items-center gap-2 mt-1 text-xs text-muted-foreground">
<span>
{story.item_count} {story.item_count === 1 ? 'item' : 'items'}
</span>
<span>·</span>
<span>{formatDate(story.updated_at)}</span>
</div>
</div>
</button>
<DropdownMenu>
<DropdownMenuTrigger asChild>
<Button
variant="ghost"
size="icon"
className="h-8 w-8 opacity-0 group-hover:opacity-100 transition-opacity"
onClick={(e) => e.stopPropagation()}
>
<MoreHorizontal className="h-4 w-4" />
</Button>
</DropdownMenuTrigger>
<DropdownMenuContent align="end">
<DropdownMenuItem onClick={() => handleEditClick(story)}>
<Pencil className="mr-2 h-4 w-4" />
Edit
</DropdownMenuItem>
<DropdownMenuItem
onClick={() => handleDeleteClick(story.id)}
className="text-destructive focus:text-destructive"
>
<Trash2 className="mr-2 h-4 w-4" />
Delete
</DropdownMenuItem>
</DropdownMenuContent>
</DropdownMenu>
<DropdownMenu>
<DropdownMenuTrigger asChild>
<Button
variant="ghost"
size="icon"
className="h-7 w-7 opacity-0 group-hover:opacity-100 transition-opacity"
onClick={(e) => e.stopPropagation()}
aria-label={`Actions for ${story.name}`}
>
<MoreHorizontal className="h-3.5 w-3.5" />
</Button>
</DropdownMenuTrigger>
<DropdownMenuContent align="end">
<DropdownMenuItem onClick={() => handleEditClick(story)}>
<Pencil className="mr-2 h-4 w-4" />
Edit
</DropdownMenuItem>
<DropdownMenuItem
onClick={() => handleDeleteClick(story.id)}
className="text-destructive focus:text-destructive"
>
<Trash2 className="mr-2 h-4 w-4" />
Delete
</DropdownMenuItem>
</DropdownMenuContent>
</DropdownMenu>
</div>
</div>
</div>
))
))}
</div>
)}
</div>
@@ -1,5 +1,7 @@
import {
Check,
Copy,
GalleryVerticalEnd,
GripHorizontal,
Minus,
Pause,
@@ -12,6 +14,12 @@ import {
import { useCallback, useEffect, useMemo, useRef, useState } from 'react';
import WaveSurfer from 'wavesurfer.js';
import { Button } from '@/components/ui/button';
import {
DropdownMenu,
DropdownMenuContent,
DropdownMenuItem,
DropdownMenuTrigger,
} from '@/components/ui/dropdown-menu';
import { useToast } from '@/components/ui/use-toast';
import { apiClient } from '@/lib/api/client';
import type { StoryItemDetail } from '@/lib/api/types';
@@ -19,6 +27,7 @@ import {
useDuplicateStoryItem,
useMoveStoryItem,
useRemoveStoryItem,
useSetStoryItemVersion,
useSplitStoryItem,
useTrimStoryItem,
} from '@/lib/hooks/useStories';
@@ -28,12 +37,14 @@ import { useStoryStore } from '@/stores/storyStore';
// Clip waveform component with trim support
function ClipWaveform({
generationId,
versionId,
width,
trimStartMs,
trimEndMs,
duration,
}: {
generationId: string;
versionId?: string;
width: number;
trimStartMs: number;
trimEndMs: number;
@@ -79,7 +90,9 @@ function ClipWaveform({
wavesurferRef.current = wavesurfer;
const audioUrl = apiClient.getAudioUrl(generationId);
const audioUrl = versionId
? apiClient.getVersionAudioUrl(versionId)
: apiClient.getAudioUrl(generationId);
wavesurfer.load(audioUrl).catch(() => {
// Ignore load errors
});
@@ -88,7 +101,7 @@ function ClipWaveform({
wavesurfer.destroy();
wavesurferRef.current = null;
};
}, [generationId, fullWaveformWidth]);
}, [generationId, versionId, fullWaveformWidth]);
return (
<div className="w-full h-full opacity-60 overflow-hidden">
@@ -135,12 +148,57 @@ export function StoryTrackEditor({ storyId, items }: StoryTrackEditorProps) {
const splitItem = useSplitStoryItem();
const duplicateItem = useDuplicateStoryItem();
const removeItem = useRemoveStoryItem();
const setItemVersion = useSetStoryItemVersion();
const { toast } = useToast();
// Selection state
const selectedClipId = useStoryStore((state) => state.selectedClipId);
const setSelectedClipId = useStoryStore((state) => state.setSelectedClipId);
// Selected clip item (for version picker)
const selectedItem = useMemo(
() => (selectedClipId ? items.find((i) => i.id === selectedClipId) : undefined),
[selectedClipId, items],
);
const selectedItemVersions = selectedItem?.versions;
const hasMultipleVersions = selectedItemVersions && selectedItemVersions.length > 1;
// Determine which version label is active for the selected clip
const activeVersionLabel = useMemo(() => {
if (!selectedItem || !selectedItemVersions) return null;
// If the item has a pinned version_id, find its label
if (selectedItem.version_id) {
const pinned = selectedItemVersions.find((v) => v.id === selectedItem.version_id);
return pinned?.label ?? null;
}
// Otherwise use the generation's default version
const defaultVersion = selectedItemVersions.find((v) => v.is_default);
return defaultVersion?.label ?? null;
}, [selectedItem, selectedItemVersions]);
const handleSetVersion = useCallback(
(versionId: string | null) => {
if (!selectedClipId) return;
setItemVersion.mutate(
{
storyId,
itemId: selectedClipId,
data: { version_id: versionId },
},
{
onError: (error) => {
toast({
title: 'Failed to set version',
description: error instanceof Error ? error.message : String(error),
variant: 'destructive',
});
},
},
);
},
[selectedClipId, storyId, setItemVersion, toast],
);
// Trim state
const [trimmingItem, setTrimmingItem] = useState<string | null>(null);
const [trimSide, setTrimSide] = useState<'start' | 'end' | null>(null);
@@ -736,6 +794,7 @@ export function StoryTrackEditor({ storyId, items }: StoryTrackEditorProps) {
className="h-7 w-7"
onClick={handlePlayPause}
title="Play/Pause (Space)"
aria-label={isCurrentlyPlaying ? 'Pause' : 'Play'}
>
{isCurrentlyPlaying ? <Pause className="h-4 w-4" /> : <Play className="h-4 w-4" />}
</Button>
@@ -745,6 +804,7 @@ export function StoryTrackEditor({ storyId, items }: StoryTrackEditorProps) {
className="h-7 w-7"
onClick={handleStop}
disabled={!isCurrentlyPlaying}
aria-label="Stop"
>
<Square className="h-3 w-3" />
</Button>
@@ -762,6 +822,7 @@ export function StoryTrackEditor({ storyId, items }: StoryTrackEditorProps) {
className="h-7 w-7"
onClick={handleSplit}
title="Split at playhead (S)"
aria-label="Split at playhead"
>
<Scissors className="h-4 w-4" />
</Button>
@@ -771,6 +832,7 @@ export function StoryTrackEditor({ storyId, items }: StoryTrackEditorProps) {
className="h-7 w-7"
onClick={handleDuplicate}
title="Duplicate (Cmd/Ctrl+D)"
aria-label="Duplicate clip"
>
<Copy className="h-4 w-4" />
</Button>
@@ -780,19 +842,75 @@ export function StoryTrackEditor({ storyId, items }: StoryTrackEditorProps) {
className="h-7 w-7"
onClick={handleDelete}
title="Delete (Delete/Backspace)"
aria-label="Delete clip"
>
<Trash2 className="h-4 w-4" />
</Button>
{hasMultipleVersions && (
<>
<div className="w-px h-4 bg-border mx-1" />
<DropdownMenu>
<DropdownMenuTrigger asChild>
<Button
variant="ghost"
className="h-7 gap-1.5 px-2 text-xs"
title="Change version/take"
>
<GalleryVerticalEnd className="h-3.5 w-3.5" />
<span className="max-w-[80px] truncate">
{activeVersionLabel ?? 'default'}
</span>
</Button>
</DropdownMenuTrigger>
<DropdownMenuContent align="center" className="min-w-[160px]">
{selectedItemVersions.map((version) => {
const isActive = selectedItem?.version_id
? version.id === selectedItem.version_id
: version.is_default;
return (
<DropdownMenuItem
key={version.id}
onClick={() => handleSetVersion(version.id)}
className="gap-2 text-xs"
>
<Check
className={cn('h-3 w-3', isActive ? 'opacity-100' : 'opacity-0')}
/>
<span className="truncate">{version.label}</span>
{version.effects_chain && version.effects_chain.length > 0 && (
<span className="text-muted-foreground ml-auto text-[10px]">
{version.effects_chain.length} fx
</span>
)}
</DropdownMenuItem>
);
})}
</DropdownMenuContent>
</DropdownMenu>
</>
)}
</div>
)}
{/* Zoom controls - right side */}
<div className="flex items-center gap-2">
<span className="text-xs text-muted-foreground">Zoom:</span>
<Button variant="ghost" size="icon" className="h-6 w-6" onClick={handleZoomOut}>
<Button
variant="ghost"
size="icon"
className="h-6 w-6"
onClick={handleZoomOut}
aria-label="Zoom out"
>
<Minus className="h-3 w-3" />
</Button>
<Button variant="ghost" size="icon" className="h-6 w-6" onClick={handleZoomIn}>
<Button
variant="ghost"
size="icon"
className="h-6 w-6"
onClick={handleZoomIn}
aria-label="Zoom in"
>
<Plus className="h-3 w-3" />
</Button>
</div>
@@ -941,6 +1059,7 @@ export function StoryTrackEditor({ storyId, items }: StoryTrackEditorProps) {
<div className="absolute inset-0 top-3">
<ClipWaveform
generationId={item.generation_id}
versionId={item.version_id}
width={clipWidth}
trimStartMs={displayTrimStart}
trimEndMs={displayTrimEnd}
+5 -6
View File
@@ -1,8 +1,7 @@
const isWindows = navigator.userAgent.includes('Windows');
export function TitleBarDragRegion() {
return (
<div
data-tauri-drag-region
className="fixed top-0 left-0 right-0 h-12 z-[9999]"
/>
);
if (isWindows) return null;
return <div data-tauri-drag-region className="fixed top-0 left-0 right-0 h-12 z-[9999]" />;
}
@@ -140,7 +140,13 @@ export function AudioSampleRecording({
</div>
<p className="text-sm text-muted-foreground text-center">File: {file.name}</p>
<div className="flex gap-2">
<Button type="button" size="icon" variant="outline" onClick={onPlayPause}>
<Button
type="button"
size="icon"
variant="outline"
onClick={onPlayPause}
aria-label={isPlaying ? 'Pause' : 'Play'}
>
{isPlaying ? <Pause className="h-4 w-4" /> : <Play className="h-4 w-4" />}
</Button>
<Button
@@ -77,7 +77,13 @@ export function AudioSampleSystem({
</div>
<p className="text-sm text-muted-foreground text-center">File: {file.name}</p>
<div className="flex gap-2">
<Button type="button" size="icon" variant="outline" onClick={onPlayPause}>
<Button
type="button"
size="icon"
variant="outline"
onClick={onPlayPause}
aria-label={isPlaying ? 'Pause' : 'Play'}
>
{isPlaying ? <Pause className="h-4 w-4" /> : <Play className="h-4 w-4" />}
</Button>
<Button
@@ -110,6 +110,7 @@ export function AudioSampleUpload({
variant="outline"
onClick={onPlayPause}
disabled={isValidating}
aria-label={isPlaying ? 'Pause' : 'Play'}
>
{isPlaying ? <Pause className="h-4 w-4" /> : <Play className="h-4 w-4" />}
</Button>
@@ -1,4 +1,4 @@
import { Download, Edit, Mic, Trash2 } from 'lucide-react';
import { Download, Edit, Sparkles, Trash2 } from 'lucide-react';
import { useState } from 'react';
import { Badge } from '@/components/ui/badge';
import { Button } from '@/components/ui/button';
@@ -15,7 +15,6 @@ import {
import type { VoiceProfileResponse } from '@/lib/api/types';
import { useDeleteProfile, useExportProfile } from '@/lib/hooks/useProfiles';
import { cn } from '@/lib/utils/cn';
import { useServerStore } from '@/stores/serverStore';
import { useUIStore } from '@/stores/uiStore';
interface ProfileCardProps {
@@ -24,19 +23,16 @@ interface ProfileCardProps {
export function ProfileCard({ profile }: ProfileCardProps) {
const [deleteDialogOpen, setDeleteDialogOpen] = useState(false);
const [avatarError, setAvatarError] = useState(false);
const deleteProfile = useDeleteProfile();
const exportProfile = useExportProfile();
const setEditingProfileId = useUIStore((state) => state.setEditingProfileId);
const setProfileDialogOpen = useUIStore((state) => state.setProfileDialogOpen);
const selectedProfileId = useUIStore((state) => state.selectedProfileId);
const setSelectedProfileId = useUIStore((state) => state.setSelectedProfileId);
const serverUrl = useServerStore((state) => state.serverUrl);
const isSelected = selectedProfileId === profile.id;
const avatarUrl = profile.avatar_path ? `${serverUrl}/profiles/${profile.id}/avatar` : null;
const handleSelect = () => {
setSelectedProfileId(isSelected ? null : profile.id);
};
@@ -61,32 +57,35 @@ export function ProfileCard({ profile }: ProfileCardProps) {
exportProfile.mutate(profile.id);
};
const handleKeyDown = (e: React.KeyboardEvent) => {
const target = e.target as HTMLElement;
if (target.closest('button')) return;
if (e.key === 'Enter' || e.key === ' ') {
e.preventDefault();
handleSelect();
}
};
const selectLabel = isSelected
? `${profile.name}, ${profile.language}. Selected as voice for generation.`
: `${profile.name}, ${profile.language}. Select as voice for generation.`;
return (
<>
<Card
className={cn(
'cursor-pointer hover:shadow-md transition-all flex flex-col',
isSelected && 'ring-2 ring-primary shadow-md',
'cursor-pointer hover:shadow-md transition-all flex flex-col h-[162px]',
isSelected && 'ring-2 ring-accent shadow-md',
)}
onClick={handleSelect}
tabIndex={0}
role="button"
aria-label={selectLabel}
aria-pressed={isSelected}
onKeyDown={handleKeyDown}
>
<CardHeader className="p-3 pb-2">
<CardTitle className="flex items-center gap-1.5 text-base font-medium">
<div className="h-6 w-6 rounded-full bg-muted flex items-center justify-center shrink-0 overflow-hidden">
{avatarUrl && !avatarError ? (
<img
src={avatarUrl}
alt={`${profile.name} avatar`}
className={cn(
'h-full w-full object-cover transition-all duration-200',
!isSelected && 'grayscale',
)}
onError={() => setAvatarError(true)}
/>
) : (
<Mic className="h-3.5 w-3.5 text-muted-foreground" />
)}
</div>
<CardTitle className="text-base font-medium">
<span className="break-words">{profile.name}</span>
</CardTitle>
</CardHeader>
@@ -94,10 +93,13 @@ export function ProfileCard({ profile }: ProfileCardProps) {
<p className="text-xs text-muted-foreground mb-1.5 line-clamp-2 leading-relaxed">
{profile.description || 'No description'}
</p>
<div className="mb-2">
<div className="mb-2 flex items-center gap-1.5">
<Badge variant="outline" className="text-xs h-5 px-1.5 text-muted-foreground">
{profile.language}
</Badge>
{profile.effects_chain && profile.effects_chain.length > 0 && (
<Sparkles className="h-3.5 w-3.5 text-accent fill-accent" />
)}
</div>
<div className="flex gap-0.5 justify-end items-end mt-auto">
<CircleButton
@@ -3,6 +3,7 @@ import { Edit2, Mic, Monitor, Upload, X } from 'lucide-react';
import { useEffect, useRef, useState } from 'react';
import { useForm } from 'react-hook-form';
import * as z from 'zod';
import { EffectsChainEditor } from '@/components/Effects/EffectsChainEditor';
import { Button } from '@/components/ui/button';
import {
Dialog,
@@ -30,6 +31,8 @@ import {
import { Tabs, TabsContent, TabsList, TabsTrigger } from '@/components/ui/tabs';
import { Textarea } from '@/components/ui/textarea';
import { useToast } from '@/components/ui/use-toast';
import { apiClient } from '@/lib/api/client';
import type { EffectConfig } from '@/lib/api/types';
import { LANGUAGE_CODES, LANGUAGE_OPTIONS, type LanguageCode } from '@/lib/constants/languages';
import { useAudioPlayer } from '@/lib/hooks/useAudioPlayer';
import { useAudioRecording } from '@/lib/hooks/useAudioRecording';
@@ -125,6 +128,8 @@ export function ProfileForm() {
const { isPlaying, playPause, cleanup: cleanupAudio } = useAudioPlayer();
const isCreating = !editingProfileId;
const serverUrl = useServerStore((state) => state.serverUrl);
const [profileEffectsChain, setProfileEffectsChain] = useState<EffectConfig[]>([]);
const [effectsDirty, setEffectsDirty] = useState(false);
const form = useForm<ProfileFormValues>({
resolver: zodResolver(profileSchema),
@@ -280,6 +285,8 @@ export function ProfileForm() {
referenceText: undefined,
avatarFile: undefined,
});
setProfileEffectsChain(editingProfile.effects_chain ?? []);
setEffectsDirty(false);
} else if (profileFormDraft && open) {
// Restore from draft when opening in create mode
form.reset({
@@ -435,6 +442,24 @@ export function ProfileForm() {
}
}
// Save effects chain if changed
if (effectsDirty) {
try {
await apiClient.updateProfileEffects(
editingProfileId,
profileEffectsChain.length > 0 ? profileEffectsChain : null,
);
} catch (fxError) {
toast({
title: 'Effects update failed',
description:
fxError instanceof Error ? fxError.message : 'Failed to save effects chain',
variant: 'destructive',
});
return;
}
}
toast({
title: 'Voice updated',
description: `"${data.name}" has been updated successfully.`,
@@ -898,6 +923,23 @@ export function ProfileForm() {
</FormItem>
)}
/>
{editingProfileId && (
<div className="space-y-2">
<FormLabel>Default Effects</FormLabel>
<p className="text-xs text-muted-foreground">
Effects applied automatically to all new generations with this voice.
</p>
<EffectsChainEditor
value={profileEffectsChain}
onChange={(chain) => {
setProfileEffectsChain(chain);
setEffectsDirty(true);
}}
compact
/>
</div>
)}
</div>
</div>
@@ -41,9 +41,11 @@ export function ProfileList() {
</CardContent>
</Card>
) : (
<div className="grid gap-4 grid-cols-3 auto-rows-auto p-1 pb-[150px]">
<div className="flex gap-4 overflow-x-auto p-1 pb-1 lg:grid lg:grid-cols-3 lg:auto-rows-auto lg:overflow-x-visible lg:pb-[150px]">
{allProfiles.map((profile) => (
<ProfileCard key={profile.id} profile={profile} />
<div key={profile.id} className="shrink-0 w-[200px] lg:w-auto lg:shrink">
<ProfileCard profile={profile} />
</div>
))}
</div>
)}
@@ -102,6 +102,7 @@ function MiniSamplePlayer({ audioUrl }: MiniSamplePlayerProps) {
className="h-7 w-7 shrink-0"
onClick={handlePlayPause}
disabled={isLoading}
aria-label={isPlaying ? 'Pause sample' : 'Play sample'}
>
{isPlaying ? <Pause className="h-3.5 w-3.5" /> : <Play className="h-3.5 w-3.5 ml-0.5" />}
</Button>
@@ -113,6 +114,8 @@ function MiniSamplePlayer({ audioUrl }: MiniSamplePlayerProps) {
max={100}
step={0.1}
className="flex-1"
aria-label="Sample playback position"
aria-valuetext={`${formatAudioDuration(currentTime)} of ${formatAudioDuration(duration)}`}
/>
<div className="flex items-center gap-1 text-xs text-muted-foreground shrink-0 min-w-[70px]">
<span className="font-mono">{formatAudioDuration(currentTime)}</span>
@@ -128,6 +131,7 @@ function MiniSamplePlayer({ audioUrl }: MiniSamplePlayerProps) {
className="h-7 w-7 shrink-0"
onClick={handleStop}
title="Stop"
aria-label="Stop playback"
>
<X className="h-3.5 w-3.5" />
</Button>
@@ -0,0 +1,340 @@
import { zodResolver } from '@hookform/resolvers/zod';
import { Edit2, Mic, X } from 'lucide-react';
import { useEffect, useRef, useState } from 'react';
import { useForm } from 'react-hook-form';
import * as z from 'zod';
import { EffectsChainEditor } from '@/components/Effects/EffectsChainEditor';
import { Button } from '@/components/ui/button';
import {
Form,
FormControl,
FormField,
FormItem,
FormLabel,
FormMessage,
} from '@/components/ui/form';
import { Input } from '@/components/ui/input';
import {
Select,
SelectContent,
SelectItem,
SelectTrigger,
SelectValue,
} from '@/components/ui/select';
import { Textarea } from '@/components/ui/textarea';
import { useToast } from '@/components/ui/use-toast';
import { SampleList } from '@/components/VoiceProfiles/SampleList';
import { apiClient } from '@/lib/api/client';
import type { EffectConfig } from '@/lib/api/types';
import { LANGUAGE_CODES, LANGUAGE_OPTIONS, type LanguageCode } from '@/lib/constants/languages';
import { BOTTOM_SAFE_AREA_PADDING } from '@/lib/constants/ui';
import {
useDeleteAvatar,
useProfile,
useUpdateProfile,
useUploadAvatar,
} from '@/lib/hooks/useProfiles';
import { cn } from '@/lib/utils/cn';
import { usePlayerStore } from '@/stores/playerStore';
import { useServerStore } from '@/stores/serverStore';
const profileSchema = z.object({
name: z.string().min(1, 'Name is required').max(100),
description: z.string().max(500).optional(),
language: z.enum(LANGUAGE_CODES as [LanguageCode, ...LanguageCode[]]),
});
type ProfileFormValues = z.infer<typeof profileSchema>;
interface VoiceInspectorProps {
profileId: string;
}
export function VoiceInspector({ profileId }: VoiceInspectorProps) {
const { data: profile } = useProfile(profileId);
const audioUrl = usePlayerStore((state) => state.audioUrl);
const isPlayerVisible = !!audioUrl;
const updateProfile = useUpdateProfile();
const uploadAvatar = useUploadAvatar();
const deleteAvatar = useDeleteAvatar();
const serverUrl = useServerStore((state) => state.serverUrl);
const { toast } = useToast();
const [avatarPreview, setAvatarPreview] = useState<string | null>(null);
const [avatarError, setAvatarError] = useState(false);
const avatarInputRef = useRef<HTMLInputElement>(null);
const [effectsChain, setEffectsChain] = useState<EffectConfig[]>([]);
const [effectsDirty, setEffectsDirty] = useState(false);
const form = useForm<ProfileFormValues>({
resolver: zodResolver(profileSchema),
defaultValues: {
name: '',
description: '',
language: 'en',
},
});
// Populate form when profile loads
useEffect(() => {
if (profile) {
form.reset({
name: profile.name,
description: profile.description || '',
language: profile.language as LanguageCode,
});
setEffectsChain(profile.effects_chain ?? []);
setEffectsDirty(false);
}
}, [profile, form]);
// Avatar preview
useEffect(() => {
if (profile?.avatar_path) {
setAvatarPreview(`${serverUrl}/profiles/${profile.id}/avatar`);
} else {
setAvatarPreview(null);
}
setAvatarError(false);
}, [profile, serverUrl]);
function handleAvatarFileChange(e: React.ChangeEvent<HTMLInputElement>) {
const file = e.target.files?.[0];
if (!file) return;
if (!file.type.startsWith('image/')) {
toast({
title: 'Invalid file type',
description: 'Please select PNG, JPG, or WebP',
variant: 'destructive',
});
return;
}
if (file.size > 5 * 1024 * 1024) {
toast({
title: 'File too large',
description: 'Image must be less than 5MB',
variant: 'destructive',
});
return;
}
// Upload immediately
uploadAvatar.mutate(
{ profileId, file },
{
onSuccess: () => {
setAvatarPreview(URL.createObjectURL(file));
toast({ title: 'Avatar updated' });
},
onError: (err) => {
toast({
title: 'Avatar upload failed',
description: err instanceof Error ? err.message : 'Unknown error',
variant: 'destructive',
});
},
},
);
}
async function handleRemoveAvatar() {
if (profile?.avatar_path) {
try {
await deleteAvatar.mutateAsync(profileId);
toast({ title: 'Avatar removed' });
} catch (err) {
toast({
title: 'Failed to remove avatar',
description: err instanceof Error ? err.message : 'Unknown error',
variant: 'destructive',
});
}
}
setAvatarPreview(null);
if (avatarInputRef.current) avatarInputRef.current.value = '';
}
async function onSubmit(data: ProfileFormValues) {
try {
await updateProfile.mutateAsync({
profileId,
data: {
name: data.name,
description: data.description,
language: data.language,
},
});
if (effectsDirty) {
try {
await apiClient.updateProfileEffects(
profileId,
effectsChain.length > 0 ? effectsChain : null,
);
setEffectsDirty(false);
} catch (fxError) {
toast({
title: 'Effects update failed',
description: fxError instanceof Error ? fxError.message : 'Failed to save effects',
variant: 'destructive',
});
return;
}
}
toast({ title: 'Voice updated', description: `"${data.name}" saved.` });
} catch (error) {
toast({
title: 'Error',
description: error instanceof Error ? error.message : 'Failed to save profile',
variant: 'destructive',
});
}
}
if (!profile) {
return (
<div className="flex items-center justify-center h-full text-muted-foreground text-sm">
Loading...
</div>
);
}
const isDirty = form.formState.isDirty || effectsDirty;
return (
<div className="h-full flex flex-col overflow-hidden">
<div className={cn('flex-1 overflow-y-auto', isPlayerVisible && BOTTOM_SAFE_AREA_PADDING)}>
<Form {...form}>
<form onSubmit={form.handleSubmit(onSubmit)} className="space-y-0">
{/* Avatar */}
<div className="flex justify-center pt-5 pb-3">
<div className="relative group">
<div className="h-20 w-20 rounded-full bg-muted flex items-center justify-center shrink-0 overflow-hidden border-2 border-border">
{avatarPreview && !avatarError ? (
<img
src={avatarPreview}
alt={profile.name}
className="h-full w-full object-cover"
onError={() => setAvatarError(true)}
/>
) : (
<Mic className="h-8 w-8 text-muted-foreground" />
)}
</div>
<button
type="button"
onClick={() => avatarInputRef.current?.click()}
className="absolute inset-0 rounded-full bg-accent/60 opacity-0 group-hover:opacity-100 transition-opacity flex items-center justify-center cursor-pointer"
>
<Edit2 className="h-5 w-5 text-accent-foreground" />
</button>
{avatarPreview && (
<button
type="button"
onClick={handleRemoveAvatar}
disabled={deleteAvatar.isPending}
className="absolute bottom-0 right-0 h-5 w-5 rounded-full bg-background/60 backdrop-blur-sm text-muted-foreground flex items-center justify-center hover:bg-background/80 hover:text-foreground transition-colors shadow-sm border border-border/50"
>
<X className="h-3 w-3" />
</button>
)}
</div>
<input
ref={avatarInputRef}
type="file"
accept="image/png,image/jpeg,image/webp"
onChange={handleAvatarFileChange}
className="hidden"
/>
</div>
{/* Fields */}
<div className="space-y-3 px-5">
<FormField
control={form.control}
name="name"
render={({ field }) => (
<FormItem>
<FormLabel>Name</FormLabel>
<FormControl>
<Input placeholder="My Voice" {...field} />
</FormControl>
<FormMessage />
</FormItem>
)}
/>
<FormField
control={form.control}
name="description"
render={({ field }) => (
<FormItem>
<FormLabel>Description</FormLabel>
<FormControl>
<Textarea placeholder="Describe this voice..." rows={2} {...field} />
</FormControl>
<FormMessage />
</FormItem>
)}
/>
<FormField
control={form.control}
name="language"
render={({ field }) => (
<FormItem>
<FormLabel>Language</FormLabel>
<Select onValueChange={field.onChange} value={field.value}>
<FormControl>
<SelectTrigger>
<SelectValue />
</SelectTrigger>
</FormControl>
<SelectContent>
{LANGUAGE_OPTIONS.map((lang) => (
<SelectItem key={lang.value} value={lang.value}>
{lang.label}
</SelectItem>
))}
</SelectContent>
</Select>
<FormMessage />
</FormItem>
)}
/>
{/* Effects */}
<div className="space-y-2">
<FormLabel>Default Effects</FormLabel>
<p className="text-xs text-muted-foreground">
Applied automatically to new generations with this voice.
</p>
<EffectsChainEditor
value={effectsChain}
onChange={(chain) => {
setEffectsChain(chain);
setEffectsDirty(true);
}}
compact
/>
</div>
{/* Save */}
{isDirty && (
<Button type="submit" className="w-full" disabled={updateProfile.isPending}>
{updateProfile.isPending ? 'Saving...' : 'Save Changes'}
</Button>
)}
</div>
{/* Samples */}
<div className="px-5 pb-5">
<SampleList profileId={profileId} />
</div>
</form>
</Form>
</div>
</div>
);
}
+144 -116
View File
@@ -1,13 +1,9 @@
import { useQuery, useQueryClient } from '@tanstack/react-query';
import { Edit, MoreHorizontal, Plus, Trash2, Mic } from 'lucide-react';
import { useMemo, useRef } from 'react';
import { Mic, Plus, Search, Sparkles } from 'lucide-react';
import { useEffect, useMemo, useRef, useState } from 'react';
import { Button } from '@/components/ui/button';
import {
DropdownMenu,
DropdownMenuContent,
DropdownMenuItem,
DropdownMenuTrigger,
} from '@/components/ui/dropdown-menu';
import { Input } from '@/components/ui/input';
import { MultiSelect } from '@/components/ui/multi-select';
import {
Table,
@@ -21,33 +17,46 @@ import { ProfileForm } from '@/components/VoiceProfiles/ProfileForm';
import { apiClient } from '@/lib/api/client';
import type { VoiceProfileResponse } from '@/lib/api/types';
import { BOTTOM_SAFE_AREA_PADDING } from '@/lib/constants/ui';
import { useHistory } from '@/lib/hooks/useHistory';
import { useDeleteProfile, useProfileSamples, useProfiles } from '@/lib/hooks/useProfiles';
import { useProfiles } from '@/lib/hooks/useProfiles';
import { cn } from '@/lib/utils/cn';
import { usePlayerStore } from '@/stores/playerStore';
import { useServerStore } from '@/stores/serverStore';
import { useUIStore } from '@/stores/uiStore';
import { VoiceInspector } from './VoiceInspector';
export function VoicesTab() {
const { data: profiles, isLoading } = useProfiles();
const { data: historyData } = useHistory({ limit: 1000 });
const queryClient = useQueryClient();
const setDialogOpen = useUIStore((state) => state.setProfileDialogOpen);
const setEditingProfileId = useUIStore((state) => state.setEditingProfileId);
const deleteProfile = useDeleteProfile();
const selectedVoiceId = useUIStore((state) => state.selectedVoiceId);
const setSelectedVoiceId = useUIStore((state) => state.setSelectedVoiceId);
const scrollRef = useRef<HTMLDivElement>(null);
const audioUrl = usePlayerStore((state) => state.audioUrl);
const isPlayerVisible = !!audioUrl;
const [search, setSearch] = useState('');
// Get generation counts per profile
const generationCounts = useMemo(() => {
const counts: Record<string, number> = {};
if (historyData?.items) {
historyData.items.forEach((item) => {
counts[item.profile_id] = (counts[item.profile_id] || 0) + 1;
});
const filteredProfiles = useMemo(() => {
if (!profiles) return [];
if (!search.trim()) return profiles;
const q = search.toLowerCase();
return profiles.filter(
(p) =>
p.name.toLowerCase().includes(q) ||
p.description?.toLowerCase().includes(q) ||
p.language.toLowerCase().includes(q),
);
}, [profiles, search]);
// Auto-select first profile if none selected
useEffect(() => {
if (!selectedVoiceId && profiles && profiles.length > 0) {
setSelectedVoiceId(profiles[0].id);
}
return counts;
}, [historyData]);
// Clear selection if selected profile was deleted
if (selectedVoiceId && profiles && !profiles.find((p) => p.id === selectedVoiceId)) {
setSelectedVoiceId(profiles.length > 0 ? profiles[0].id : null);
}
}, [profiles, selectedVoiceId, setSelectedVoiceId]);
// Get channel assignments for each profile
const { data: channelAssignments } = useQuery({
@@ -74,17 +83,6 @@ export function VoicesTab() {
queryFn: () => apiClient.listChannels(),
});
const handleEdit = (profileId: string) => {
setEditingProfileId(profileId);
setDialogOpen(true);
};
const handleProfileDelete = async (profileId: string) => {
if (await confirm('Are you sure you want to delete this profile?')) {
deleteProfile.mutate(profileId);
}
};
const handleChannelChange = async (profileId: string, channelIds: string[]) => {
try {
await apiClient.setProfileChannels(profileId, channelIds);
@@ -103,56 +101,76 @@ export function VoicesTab() {
}
return (
<div className="h-full flex flex-col relative overflow-hidden">
{/* Scroll Mask - Always visible, behind content */}
<div className="absolute top-0 left-0 right-0 h-16 bg-gradient-to-b from-background to-transparent z-10 pointer-events-none" />
<div className="h-full flex gap-0 overflow-hidden -mx-8">
{/* Left: Table */}
<div className="flex-1 min-w-0 flex flex-col relative overflow-hidden">
{/* Scroll Mask */}
<div className="absolute top-0 left-0 right-0 h-16 bg-gradient-to-b from-background to-transparent z-10 pointer-events-none" />
{/* Fixed Header */}
<div className="absolute top-0 left-0 right-0 z-20">
<div className="flex items-center justify-between mb-6">
<h1 className="text-2xl font-bold">Voices</h1>
<Button onClick={() => setDialogOpen(true)}>
<Plus className="h-4 w-4 mr-2" />
New Voice
</Button>
{/* Fixed Header */}
<div className="absolute top-0 left-0 right-0 z-20 pl-8 pr-8">
<div className="flex items-center gap-3 mb-6">
<h1 className="text-2xl font-bold">Voices</h1>
<div className="flex-1" />
<div className="relative w-[240px]">
<Search className="absolute left-2.5 top-1/2 -translate-y-1/2 h-3.5 w-3.5 text-muted-foreground" />
<Input
placeholder="Search voices..."
value={search}
onChange={(e) => setSearch(e.target.value)}
className="h-10 pl-8 text-sm rounded-full focus-visible:ring-0 focus-visible:ring-offset-0"
/>
</div>
<Button onClick={() => setDialogOpen(true)}>
<Plus className="h-4 w-4 mr-2" />
New Voice
</Button>
</div>
</div>
{/* Scrollable Content */}
<div
ref={scrollRef}
className={cn(
'flex-1 overflow-y-auto overflow-x-hidden pt-16 relative z-0',
isPlayerVisible && BOTTOM_SAFE_AREA_PADDING,
)}
>
<Table className="table-fixed [&_td:first-child]:pl-8 [&_th:first-child]:pl-8">
<TableHeader>
<TableRow>
<TableHead className="w-[30%]">Name</TableHead>
<TableHead className="w-[10%]">Language</TableHead>
<TableHead className="w-[10%]">Generations</TableHead>
<TableHead className="w-[8%]">Samples</TableHead>
<TableHead className="w-[8%]">Effects</TableHead>
<TableHead className="w-[24%]">Channels</TableHead>
<TableHead className="w-6"></TableHead>
</TableRow>
</TableHeader>
<TableBody>
{filteredProfiles.map((profile) => (
<VoiceRow
key={profile.id}
profile={profile}
isSelected={selectedVoiceId === profile.id}
onSelect={() => setSelectedVoiceId(profile.id)}
channelIds={channelAssignments?.[profile.id] || []}
channels={channels || []}
onChannelChange={(channelIds) => handleChannelChange(profile.id, channelIds)}
/>
))}
</TableBody>
</Table>
</div>
</div>
{/* Scrollable Content */}
<div
ref={scrollRef}
className={cn(
'flex-1 overflow-y-auto pt-16 relative z-0',
isPlayerVisible && BOTTOM_SAFE_AREA_PADDING,
)}
>
<Table>
<TableHeader>
<TableRow>
<TableHead>Name</TableHead>
<TableHead>Language</TableHead>
<TableHead>Generations</TableHead>
<TableHead>Samples</TableHead>
<TableHead>Channels</TableHead>
<TableHead className="w-[50px]"></TableHead>
</TableRow>
</TableHeader>
<TableBody>
{profiles?.map((profile) => (
<VoiceRow
key={profile.id}
profile={profile}
generationCount={generationCounts[profile.id] || 0}
channelIds={channelAssignments?.[profile.id] || []}
channels={channels || []}
onChannelChange={(channelIds) => handleChannelChange(profile.id, channelIds)}
onEdit={() => handleEdit(profile.id)}
onDelete={() => handleProfileDelete(profile.id)}
/>
))}
</TableBody>
</Table>
</div>
{/* Right: Inspector */}
{selectedVoiceId && (
<div className="w-[340px] shrink-0 border-l border-t rounded-tl-xl bg-muted/30">
<VoiceInspector key={selectedVoiceId} profileId={selectedVoiceId} />
</div>
)}
<ProfileForm />
</div>
@@ -161,43 +179,71 @@ export function VoicesTab() {
interface VoiceRowProps {
profile: VoiceProfileResponse;
generationCount: number;
isSelected: boolean;
onSelect: () => void;
channelIds: string[];
channels: Array<{ id: string; name: string; is_default: boolean }>;
onChannelChange: (channelIds: string[]) => void;
onEdit: () => void;
onDelete: () => void;
}
function VoiceRow({
profile,
generationCount,
isSelected,
onSelect,
channelIds,
channels,
onChannelChange,
onEdit,
onDelete,
}: VoiceRowProps) {
const { data: samples } = useProfileSamples(profile.id);
const serverUrl = useServerStore((state) => state.serverUrl);
const [avatarError, setAvatarError] = useState(false);
const avatarUrl = profile.avatar_path ? `${serverUrl}/profiles/${profile.id}/avatar` : null;
const enabledEffects = profile.effects_chain?.filter((e) => e.enabled) ?? [];
const effectsSummary = enabledEffects.map((e) => e.type).join(' → ');
return (
<TableRow className="cursor-pointer" onClick={onEdit}>
<TableRow
className={cn('cursor-pointer', isSelected ? 'bg-muted/50' : 'hover:bg-muted/50')}
onClick={onSelect}
>
<TableCell>
<div className="flex items-center gap-2">
<div className="h-8 w-8 rounded-lg bg-muted flex items-center justify-center shrink-0">
<Mic className="h-4 w-4 text-muted-foreground" />
<div className="flex w-full min-w-0 items-center gap-2">
<div className="h-8 w-8 rounded-full bg-muted flex items-center justify-center shrink-0 overflow-hidden">
{avatarUrl && !avatarError ? (
<img
src={avatarUrl}
alt={`${profile.name} avatar`}
className="h-full w-full object-cover"
onError={() => setAvatarError(true)}
/>
) : (
<Mic className="h-4 w-4 text-muted-foreground" />
)}
</div>
<div>
<div className="font-medium">{profile.name}</div>
<div className="min-w-0">
<div className="font-medium truncate">{profile.name}</div>
{profile.description && (
<div className="text-sm text-muted-foreground">{profile.description}</div>
<div className="text-sm text-muted-foreground truncate">{profile.description}</div>
)}
</div>
</div>
</TableCell>
<TableCell onClick={(e) => e.stopPropagation()}>{profile.language}</TableCell>
<TableCell onClick={(e) => e.stopPropagation()}>{generationCount}</TableCell>
<TableCell onClick={(e) => e.stopPropagation()}>{samples?.length || 0}</TableCell>
<TableCell>{profile.language}</TableCell>
<TableCell>{profile.generation_count}</TableCell>
<TableCell>{profile.sample_count}</TableCell>
<TableCell>
{enabledEffects.length > 0 ? (
<span
className="inline-flex items-center gap-1 text-xs text-accent"
title={effectsSummary}
>
<Sparkles className="h-3 w-3 fill-accent" />
{enabledEffects.length}
</span>
) : (
<span className="text-xs text-muted-foreground">—</span>
)}
</TableCell>
<TableCell onClick={(e) => e.stopPropagation()}>
<MultiSelect
options={channels.map((ch) => ({
@@ -207,28 +253,10 @@ function VoiceRow({
value={channelIds}
onChange={onChannelChange}
placeholder="Select channels..."
className="min-w-[200px]"
className="w-full"
/>
</TableCell>
<TableCell onClick={(e) => e.stopPropagation()}>
<DropdownMenu>
<DropdownMenuTrigger asChild>
<Button variant="ghost" size="icon">
<MoreHorizontal className="h-4 w-4" />
</Button>
</DropdownMenuTrigger>
<DropdownMenuContent>
<DropdownMenuItem onClick={onEdit}>
<Edit className="h-4 w-4 mr-2" />
Edit
</DropdownMenuItem>
<DropdownMenuItem onClick={onDelete} className="text-destructive">
<Trash2 className="h-4 w-4 mr-2" />
Delete
</DropdownMenuItem>
</DropdownMenuContent>
</DropdownMenu>
</TableCell>
<TableCell />
</TableRow>
);
}
+1 -1
View File
@@ -1,5 +1,5 @@
import * as React from 'react';
import { Check } from 'lucide-react';
import * as React from 'react';
import { cn } from '@/lib/utils/cn';
export interface CheckboxProps {
+5 -3
View File
@@ -1,6 +1,6 @@
import * as React from 'react';
import * as DropdownMenuPrimitive from '@radix-ui/react-dropdown-menu';
import { MoreHorizontal } from 'lucide-react';
import * as React from 'react';
import { cn } from '@/lib/utils/cn';
const DropdownMenu = DropdownMenuPrimitive.Root;
@@ -73,7 +73,7 @@ const DropdownMenuItem = React.forwardRef<
<DropdownMenuPrimitive.Item
ref={ref}
className={cn(
'relative flex cursor-default select-none items-center rounded-sm px-2 py-1.5 text-sm outline-none transition-colors focus:bg-accent focus:text-accent-foreground data-[disabled]:pointer-events-none data-[disabled]:opacity-50',
'relative flex cursor-default select-none items-center rounded-sm px-2 py-1.5 text-sm outline-none focus:bg-accent focus:text-accent-foreground data-[disabled]:pointer-events-none data-[disabled]:opacity-50',
inset && 'pl-8',
className,
)}
@@ -154,7 +154,9 @@ const DropdownMenuSeparator = React.forwardRef<
DropdownMenuSeparator.displayName = DropdownMenuPrimitive.Separator.displayName;
const DropdownMenuShortcut = ({ className, ...props }: React.HTMLAttributes<HTMLSpanElement>) => {
return <span className={cn('ml-auto text-xs tracking-widest opacity-60', className)} {...props} />;
return (
<span className={cn('ml-auto text-xs tracking-widest opacity-60', className)} {...props} />
);
};
DropdownMenuShortcut.displayName = 'DropdownMenuShortcut';
+1 -1
View File
@@ -108,7 +108,7 @@ const SelectItem = React.forwardRef<
<SelectPrimitive.Item
ref={ref}
className={cn(
'relative flex w-full cursor-default select-none items-center rounded-sm py-1.5 pl-8 pr-2 text-sm outline-none focus:bg-accent focus:text-accent-foreground data-[disabled]:pointer-events-none data-[disabled]:opacity-50',
'relative flex w-full cursor-default select-none items-center rounded-sm py-1.5 pl-8 pr-2 text-sm outline-none focus:bg-accent focus:text-accent-foreground focus:[&_*]:text-accent-foreground data-[disabled]:pointer-events-none data-[disabled]:opacity-50',
className,
)}
{...props}
+6 -5
View File
@@ -14,7 +14,11 @@ const TableHeader = React.forwardRef<
HTMLTableSectionElement,
React.HTMLAttributes<HTMLTableSectionElement>
>(({ className, ...props }, ref) => (
<thead ref={ref} className={cn('[&_tr]:border-b', className)} {...props} />
<thead
ref={ref}
className={cn('[&_tr]:border-b [&_tr]:hover:bg-transparent', className)}
{...props}
/>
));
TableHeader.displayName = 'TableHeader';
@@ -42,10 +46,7 @@ const TableRow = React.forwardRef<HTMLTableRowElement, React.HTMLAttributes<HTML
({ className, ...props }, ref) => (
<tr
ref={ref}
className={cn(
'border-b transition-colors hover:bg-muted/50 data-[state=selected]:bg-muted',
className,
)}
className={cn('border-b hover:bg-muted/50 data-[state=selected]:bg-muted', className)}
{...props}
/>
),
+16
View File
@@ -1,4 +1,5 @@
@import "tailwindcss" source(".");
@import "loaders.css/loaders.min.css";
@theme {
--radius-sm: calc(var(--radius) - 4px);
@@ -155,3 +156,18 @@
animation: fadeIn 0.5s ease-out 0.15s forwards;
opacity: 0;
}
/* react-loaders */
.line-scale-pulse-out-rapid > div,
.line-scale > div {
background-color: hsl(var(--accent)) !important;
}
.loader-hidden {
display: block;
}
.loader-hidden > div > div {
animation-play-state: paused !important;
background-color: hsl(var(--muted-foreground)) !important;
}
+159
View File
@@ -2,9 +2,15 @@ import type { LanguageCode } from '@/lib/constants/languages';
import { useServerStore } from '@/stores/serverStore';
import type {
ActiveTasksResponse,
ApplyEffectsRequest,
AvailableEffectsResponse,
CudaStatus,
EffectConfig,
EffectPresetCreate,
EffectPresetResponse,
GenerationRequest,
GenerationResponse,
GenerationVersionResponse,
HealthResponse,
HistoryListResponse,
HistoryQuery,
@@ -21,6 +27,7 @@ import type {
StoryItemReorder,
StoryItemSplit,
StoryItemTrim,
StoryItemVersionUpdate,
StoryResponse,
TranscriptionResponse,
VoiceProfileCreate,
@@ -200,6 +207,24 @@ class ApiClient {
});
}
async retryGeneration(generationId: string): Promise<GenerationResponse> {
return this.request<GenerationResponse>(`/generate/${generationId}/retry`, {
method: 'POST',
});
}
async regenerateGeneration(generationId: string): Promise<GenerationResponse> {
return this.request<GenerationResponse>(`/generate/${generationId}/regenerate`, {
method: 'POST',
});
}
async toggleFavorite(generationId: string): Promise<{ is_favorited: boolean }> {
return this.request<{ is_favorited: boolean }>(`/history/${generationId}/favorite`, {
method: 'POST',
});
}
// History
async listHistory(query?: HistoryQuery): Promise<HistoryListResponse> {
const params = new URLSearchParams();
@@ -278,6 +303,11 @@ class ApiClient {
return response.json();
}
// Generation status SSE
getGenerationStatusUrl(generationId: string): string {
return `${this.getBaseUrl()}/generate/${generationId}/status`;
}
// Audio
getAudioUrl(audioId: string): string {
return `${this.getBaseUrl()}/audio/${audioId}`;
@@ -316,6 +346,21 @@ class ApiClient {
return this.request<ModelStatusListResponse>('/models/status');
}
async getModelsCacheDir(): Promise<{ path: string }> {
return this.request<{ path: string }>('/models/cache-dir');
}
async migrateModels(destination: string): Promise<{ source: string; destination: string }> {
return this.request('/models/migrate', {
method: 'POST',
body: JSON.stringify({ destination }),
});
}
getMigrationProgressUrl(): string {
return `${this.getBaseUrl()}/models/migrate/progress`;
}
async triggerModelDownload(modelName: string): Promise<{ message: string }> {
console.log(
'[API] triggerModelDownload called for:',
@@ -337,6 +382,12 @@ class ApiClient {
});
}
async unloadModel(modelName: string): Promise<{ message: string }> {
return this.request<{ message: string }>(`/models/${modelName}/unload`, {
method: 'POST',
});
}
async cancelDownload(modelName: string): Promise<{ message: string }> {
return this.request<{ message: string }>('/models/download/cancel', {
method: 'POST',
@@ -538,6 +589,17 @@ class ApiClient {
});
}
async setStoryItemVersion(
storyId: string,
itemId: string,
data: StoryItemVersionUpdate,
): Promise<StoryItemDetail> {
return this.request<StoryItemDetail>(`/stories/${storyId}/items/${itemId}/version`, {
method: 'PUT',
body: JSON.stringify(data),
});
}
async exportStoryAudio(storyId: string): Promise<Blob> {
const url = `${this.getBaseUrl()}/stories/${storyId}/export-audio`;
const response = await fetch(url);
@@ -551,6 +613,103 @@ class ApiClient {
return response.blob();
}
// Effects & Versions
async getAvailableEffects(): Promise<AvailableEffectsResponse> {
return this.request<AvailableEffectsResponse>('/effects/available');
}
async listEffectPresets(): Promise<EffectPresetResponse[]> {
return this.request<EffectPresetResponse[]>('/effects/presets');
}
async createEffectPreset(data: EffectPresetCreate): Promise<EffectPresetResponse> {
return this.request<EffectPresetResponse>('/effects/presets', {
method: 'POST',
body: JSON.stringify(data),
});
}
async updateEffectPreset(
presetId: string,
data: { name?: string; description?: string; effects_chain?: EffectConfig[] },
): Promise<EffectPresetResponse> {
return this.request<EffectPresetResponse>(`/effects/presets/${presetId}`, {
method: 'PUT',
body: JSON.stringify(data),
});
}
async deleteEffectPreset(presetId: string): Promise<void> {
await this.request<void>(`/effects/presets/${presetId}`, {
method: 'DELETE',
});
}
async listGenerationVersions(generationId: string): Promise<GenerationVersionResponse[]> {
return this.request<GenerationVersionResponse[]>(`/generations/${generationId}/versions`);
}
async applyEffectsToGeneration(
generationId: string,
data: ApplyEffectsRequest,
): Promise<GenerationVersionResponse> {
return this.request<GenerationVersionResponse>(
`/generations/${generationId}/versions/apply-effects`,
{
method: 'POST',
body: JSON.stringify(data),
},
);
}
async setDefaultVersion(
generationId: string,
versionId: string,
): Promise<GenerationVersionResponse> {
return this.request<GenerationVersionResponse>(
`/generations/${generationId}/versions/${versionId}/set-default`,
{ method: 'PUT' },
);
}
async deleteGenerationVersion(generationId: string, versionId: string): Promise<void> {
await this.request<void>(`/generations/${generationId}/versions/${versionId}`, {
method: 'DELETE',
});
}
getVersionAudioUrl(versionId: string): string {
return `${this.getBaseUrl()}/audio/version/${versionId}`;
}
async updateProfileEffects(
profileId: string,
effectsChain: EffectConfig[] | null,
): Promise<VoiceProfileResponse> {
return this.request<VoiceProfileResponse>(`/profiles/${profileId}/effects`, {
method: 'PUT',
body: JSON.stringify({ effects_chain: effectsChain }),
});
}
async previewEffects(generationId: string, effectsChain: EffectConfig[]): Promise<Blob> {
const url = `${this.getBaseUrl()}/effects/preview/${generationId}`;
const response = await fetch(url, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ effects_chain: effectsChain }),
});
if (!response.ok) {
const error = await response.json().catch(() => ({
detail: response.statusText,
}));
throw new Error(error.detail || `HTTP error! status: ${response.status}`);
}
return response.blob();
}
}
export const apiClient = new ApiClient();
+114 -3
View File
@@ -13,6 +13,9 @@ export interface VoiceProfileResponse {
description?: string;
language: string;
avatar_path?: string;
effects_chain?: EffectConfig[];
generation_count: number;
sample_count: number;
created_at: string;
updated_at: string;
}
@@ -28,14 +31,35 @@ export interface ProfileSampleResponse {
reference_text: string;
}
export interface EffectConfig {
type: string;
enabled: boolean;
params: Record<string, number>;
}
export interface GenerationRequest {
profile_id: string;
text: string;
language: LanguageCode;
seed?: number;
model_size?: '1.7B' | '0.6B';
engine?: 'qwen' | 'luxtts';
engine?: 'qwen' | 'luxtts' | 'chatterbox' | 'chatterbox_turbo';
instruct?: string;
max_chunk_chars?: number;
crossfade_ms?: number;
normalize?: boolean;
effects_chain?: EffectConfig[];
}
export interface GenerationVersionResponse {
id: string;
generation_id: string;
label: string;
audio_path: string;
effects_chain?: EffectConfig[];
source_version_id?: string;
is_default: boolean;
created_at: string;
}
export interface GenerationResponse {
@@ -43,10 +67,18 @@ export interface GenerationResponse {
profile_id: string;
text: string;
language: string;
audio_path: string;
duration: number;
audio_path?: string;
duration?: number;
seed?: number;
instruct?: string;
engine?: string;
model_size?: string;
status: 'generating' | 'completed' | 'failed';
error?: string;
is_favorited?: boolean;
created_at: string;
versions?: GenerationVersionResponse[];
active_version_id?: string;
}
export interface HistoryQuery {
@@ -58,6 +90,8 @@ export interface HistoryQuery {
export interface HistoryResponse extends GenerationResponse {
profile_name: string;
versions?: GenerationVersionResponse[];
active_version_id?: string;
}
export interface HistoryListResponse {
@@ -119,12 +153,29 @@ export interface ModelProgress {
export interface ModelStatus {
model_name: string;
display_name: string;
hf_repo_id?: string; // HuggingFace repository ID
downloaded: boolean;
downloading: boolean; // True if download is in progress
size_mb?: number;
loaded: boolean;
}
export interface HuggingFaceModelInfo {
id: string;
author: string;
lastModified: string;
pipeline_tag?: string;
library_name?: string;
downloads: number;
likes: number;
tags: string[];
cardData?: {
license?: string;
language?: string[];
pipeline_tag?: string;
};
}
export interface ModelStatusListResponse {
models: ModelStatus[];
}
@@ -138,6 +189,10 @@ export interface ActiveDownloadTask {
status: string;
started_at: string;
error?: string;
progress?: number; // 0-100 percentage
current?: number; // bytes downloaded
total?: number; // total bytes
filename?: string; // current file being downloaded
}
export interface ActiveGenerationTask {
@@ -170,6 +225,7 @@ export interface StoryItemDetail {
id: string;
story_id: string;
generation_id: string;
version_id?: string;
start_time_ms: number;
track: number;
trim_start_ms: number;
@@ -184,6 +240,12 @@ export interface StoryItemDetail {
seed?: number;
instruct?: string;
generation_created_at: string;
versions?: GenerationVersionResponse[];
active_version_id?: string;
}
export interface StoryItemVersionUpdate {
version_id: string | null;
}
export interface StoryDetailResponse {
@@ -227,3 +289,52 @@ export interface StoryItemTrim {
export interface StoryItemSplit {
split_time_ms: number;
}
// Effects
export interface EffectPresetResponse {
id: string;
name: string;
description?: string;
effects_chain: EffectConfig[];
is_builtin: boolean;
created_at: string;
}
export interface EffectPresetCreate {
name: string;
description?: string;
effects_chain: EffectConfig[];
}
export interface EffectPresetUpdate {
name?: string;
description?: string;
effects_chain?: EffectConfig[];
}
export interface AvailableEffectParam {
default: number;
min: number;
max: number;
step: number;
description: string;
}
export interface AvailableEffect {
type: string;
label: string;
description: string;
params: Record<string, AvailableEffectParam>;
}
export interface AvailableEffectsResponse {
effects: AvailableEffect[];
}
export interface ApplyEffectsRequest {
effects_chain: EffectConfig[];
source_version_id?: string;
label?: string;
set_as_default?: boolean;
}
+72 -12
View File
@@ -1,26 +1,86 @@
/**
* Supported languages for Qwen3-TTS
* Based on: https://github.com/QwenLM/Qwen3-TTS
* Supported languages for voice generation, per engine.
*
* Qwen3-TTS supports 10 languages.
* LuxTTS is English-only.
* Chatterbox Multilingual supports 23 languages.
* Chatterbox Turbo is English-only.
*/
export const SUPPORTED_LANGUAGES = {
zh: 'Chinese',
/** All languages that any engine supports. */
export const ALL_LANGUAGES = {
ar: 'Arabic',
da: 'Danish',
de: 'German',
el: 'Greek',
en: 'English',
es: 'Spanish',
fi: 'Finnish',
fr: 'French',
he: 'Hebrew',
hi: 'Hindi',
it: 'Italian',
ja: 'Japanese',
ko: 'Korean',
de: 'German',
fr: 'French',
ru: 'Russian',
ms: 'Malay',
nl: 'Dutch',
no: 'Norwegian',
pl: 'Polish',
pt: 'Portuguese',
es: 'Spanish',
it: 'Italian',
ru: 'Russian',
sv: 'Swedish',
sw: 'Swahili',
tr: 'Turkish',
zh: 'Chinese',
} as const;
export type LanguageCode = keyof typeof SUPPORTED_LANGUAGES;
export type LanguageCode = keyof typeof ALL_LANGUAGES;
export const LANGUAGE_CODES = Object.keys(SUPPORTED_LANGUAGES) as LanguageCode[];
/** Per-engine supported language codes. */
export const ENGINE_LANGUAGES: Record<string, readonly LanguageCode[]> = {
qwen: ['zh', 'en', 'ja', 'ko', 'de', 'fr', 'ru', 'pt', 'es', 'it'],
luxtts: ['en'],
chatterbox: [
'ar',
'da',
'de',
'el',
'en',
'es',
'fi',
'fr',
'he',
'hi',
'it',
'ja',
'ko',
'ms',
'nl',
'no',
'pl',
'pt',
'ru',
'sv',
'sw',
'tr',
'zh',
],
chatterbox_turbo: ['en'],
} as const;
/** Helper: get language options for a given engine. */
export function getLanguageOptionsForEngine(engine: string) {
const codes = ENGINE_LANGUAGES[engine] ?? ENGINE_LANGUAGES.qwen;
return codes.map((code) => ({
value: code,
label: ALL_LANGUAGES[code],
}));
}
// ── Backwards-compatible exports used elsewhere ──────────────────────
export const SUPPORTED_LANGUAGES = ALL_LANGUAGES;
export const LANGUAGE_CODES = Object.keys(ALL_LANGUAGES) as LanguageCode[];
export const LANGUAGE_OPTIONS = LANGUAGE_CODES.map((code) => ({
value: code,
label: SUPPORTED_LANGUAGES[code],
label: ALL_LANGUAGES[code],
}));
+5 -2
View File
@@ -2,11 +2,14 @@
* UI layout constants for safe area padding
*/
const isWindows = typeof navigator !== 'undefined' && navigator.userAgent.includes('Windows');
/**
* Top safe area padding - height of the drag region bar
* Corresponds to Tailwind's pt-12 (3rem / 48px)
* On macOS this accounts for the overlay titlebar (48px).
* On Windows the native title bar is outside the webview, so no padding is needed.
*/
export const TOP_SAFE_AREA_PADDING = 'pt-12';
export const TOP_SAFE_AREA_PADDING = isWindows ? 'pt-8' : 'pt-12';
/**
* Bottom safe area padding - height of the audio player
+37 -21
View File
@@ -4,19 +4,20 @@ import { useForm } from 'react-hook-form';
import * as z from 'zod';
import { useToast } from '@/components/ui/use-toast';
import { apiClient } from '@/lib/api/client';
import type { EffectConfig } from '@/lib/api/types';
import { LANGUAGE_CODES, type LanguageCode } from '@/lib/constants/languages';
import { useGeneration } from '@/lib/hooks/useGeneration';
import { useModelDownloadToast } from '@/lib/hooks/useModelDownloadToast';
import { useGenerationStore } from '@/stores/generationStore';
import { usePlayerStore } from '@/stores/playerStore';
import { useServerStore } from '@/stores/serverStore';
const generationSchema = z.object({
text: z.string().min(1, 'Text is required').max(5000),
text: z.string().min(1, '').max(50000),
language: z.enum(LANGUAGE_CODES as [LanguageCode, ...LanguageCode[]]),
seed: z.number().int().optional(),
modelSize: z.enum(['1.7B', '0.6B']).optional(),
instruct: z.string().max(500).optional(),
engine: z.enum(['qwen', 'luxtts']).optional(),
engine: z.enum(['qwen', 'luxtts', 'chatterbox', 'chatterbox_turbo']).optional(),
});
export type GenerationFormValues = z.infer<typeof generationSchema>;
@@ -24,13 +25,16 @@ export type GenerationFormValues = z.infer<typeof generationSchema>;
interface UseGenerationFormOptions {
onSuccess?: (generationId: string) => void;
defaultValues?: Partial<GenerationFormValues>;
getEffectsChain?: () => EffectConfig[] | undefined;
}
export function useGenerationForm(options: UseGenerationFormOptions = {}) {
const { toast } = useToast();
const generation = useGeneration();
const setAudioWithAutoPlay = usePlayerStore((state) => state.setAudioWithAutoPlay);
const setIsGenerating = useGenerationStore((state) => state.setIsGenerating);
const addPendingGeneration = useGenerationStore((state) => state.addPendingGeneration);
const maxChunkChars = useServerStore((state) => state.maxChunkChars);
const crossfadeMs = useServerStore((state) => state.crossfadeMs);
const normalizeAudio = useServerStore((state) => state.normalizeAudio);
const [downloadingModelName, setDownloadingModelName] = useState<string | null>(null);
const [downloadingDisplayName, setDownloadingDisplayName] = useState<string | null>(null);
@@ -67,17 +71,27 @@ export function useGenerationForm(options: UseGenerationFormOptions = {}) {
}
try {
setIsGenerating(true);
const engine = data.engine || 'qwen';
const modelName = engine === 'luxtts' ? 'luxtts' : `qwen-tts-${data.modelSize}`;
const modelName =
engine === 'luxtts'
? 'luxtts'
: engine === 'chatterbox'
? 'chatterbox-tts'
: engine === 'chatterbox_turbo'
? 'chatterbox-turbo'
: `qwen-tts-${data.modelSize}`;
const displayName =
engine === 'luxtts'
? 'LuxTTS'
: data.modelSize === '1.7B'
? 'Qwen TTS 1.7B'
: 'Qwen TTS 0.6B';
: engine === 'chatterbox'
? 'Chatterbox TTS'
: engine === 'chatterbox_turbo'
? 'Chatterbox Turbo'
: data.modelSize === '1.7B'
? 'Qwen TTS 1.7B'
: 'Qwen TTS 0.6B';
// Check if model needs downloading
try {
const modelStatus = await apiClient.getModelStatus();
const model = modelStatus.models.find((m) => m.model_name === modelName);
@@ -90,24 +104,27 @@ export function useGenerationForm(options: UseGenerationFormOptions = {}) {
console.error('Failed to check model status:', error);
}
const isQwen = engine === 'qwen';
const effectsChain = options.getEffectsChain?.();
// This now returns immediately with status="generating"
const result = await generation.mutateAsync({
profile_id: selectedProfileId,
text: data.text,
language: data.language,
seed: data.seed,
model_size: engine === 'luxtts' ? undefined : data.modelSize,
model_size: isQwen ? data.modelSize : undefined,
engine,
instruct: engine === 'luxtts' ? undefined : data.instruct || undefined,
instruct: isQwen ? data.instruct || undefined : undefined,
max_chunk_chars: maxChunkChars,
crossfade_ms: crossfadeMs,
normalize: normalizeAudio,
effects_chain: effectsChain?.length ? effectsChain : undefined,
});
toast({
title: 'Generation complete!',
description: `Audio generated (${result.duration.toFixed(2)}s)`,
});
const audioUrl = apiClient.getAudioUrl(result.id);
setAudioWithAutoPlay(audioUrl, result.id, selectedProfileId, data.text.substring(0, 50));
// Track this generation for SSE status updates
addPendingGeneration(result.id);
// Reset form immediately — user can start typing again
form.reset({
text: '',
language: data.language,
@@ -124,7 +141,6 @@ export function useGenerationForm(options: UseGenerationFormOptions = {}) {
variant: 'destructive',
});
} finally {
setIsGenerating(false);
setDownloadingModelName(null);
setDownloadingDisplayName(null);
}
+154
View File
@@ -0,0 +1,154 @@
import { useQueryClient } from '@tanstack/react-query';
import { useEffect, useRef } from 'react';
import { useToast } from '@/components/ui/use-toast';
import { apiClient } from '@/lib/api/client';
import { useGenerationStore } from '@/stores/generationStore';
import { usePlayerStore } from '@/stores/playerStore';
import { useServerStore } from '@/stores/serverStore';
interface GenerationStatusEvent {
id: string;
status: 'generating' | 'completed' | 'failed' | 'not_found';
duration?: number;
error?: string;
}
/**
* Subscribes to SSE for all pending generations. When a generation completes,
* invalidates the history query, removes it from pending, and auto-plays
* if the player is idle.
*/
export function useGenerationProgress() {
const queryClient = useQueryClient();
const { toast } = useToast();
const pendingIds = useGenerationStore((s) => s.pendingGenerationIds);
const removePendingGeneration = useGenerationStore((s) => s.removePendingGeneration);
const removePendingStoryAdd = useGenerationStore((s) => s.removePendingStoryAdd);
const isPlaying = usePlayerStore((s) => s.isPlaying);
const setAudioWithAutoPlay = usePlayerStore((s) => s.setAudioWithAutoPlay);
const autoplayOnGenerate = useServerStore((s) => s.autoplayOnGenerate);
// Keep refs to avoid stale closures in EventSource handlers
const isPlayingRef = useRef(isPlaying);
const autoplayRef = useRef(autoplayOnGenerate);
isPlayingRef.current = isPlaying;
autoplayRef.current = autoplayOnGenerate;
// Track active EventSource instances
const eventSourcesRef = useRef<Map<string, EventSource>>(new Map());
// Unmount-only cleanup — close all SSE connections when the hook is torn down
useEffect(() => {
const sources = eventSourcesRef.current;
return () => {
for (const source of sources.values()) {
source.close();
}
sources.clear();
};
}, []);
useEffect(() => {
const currentSources = eventSourcesRef.current;
// Close SSE connections for IDs no longer pending
for (const [id, source] of currentSources.entries()) {
if (!pendingIds.has(id)) {
source.close();
currentSources.delete(id);
}
}
// Open SSE connections for new pending IDs
for (const id of pendingIds) {
if (currentSources.has(id)) continue;
const url = apiClient.getGenerationStatusUrl(id);
const source = new EventSource(url);
source.onmessage = (event) => {
try {
const data: GenerationStatusEvent = JSON.parse(event.data);
if (data.status === 'completed') {
source.close();
currentSources.delete(id);
removePendingGeneration(id);
// Refresh history to pick up the completed generation
queryClient.invalidateQueries({ queryKey: ['history'] });
// If this generation was queued for a story, add it now
const storyId = removePendingStoryAdd(id);
if (storyId) {
apiClient
.addStoryItem(storyId, { generation_id: id })
.then(() => {
queryClient.invalidateQueries({ queryKey: ['stories'] });
queryClient.invalidateQueries({ queryKey: ['stories', storyId] });
toast({
title: 'Added to story',
description: data.duration
? `Audio generated (${data.duration.toFixed(2)}s) and added to story`
: 'Audio generated and added to story',
});
})
.catch(() => {
toast({
title: 'Generation complete',
description: 'Audio generated but failed to add to story',
variant: 'destructive',
});
});
} else {
// toast({
// title: 'Generation complete!',
// description: data.duration
// ? `Audio generated (${data.duration.toFixed(2)}s)`
// : 'Audio generated',
// });
}
// Auto-play if enabled and nothing is currently playing
if (autoplayRef.current && !isPlayingRef.current) {
const genAudioUrl = apiClient.getAudioUrl(id);
setAudioWithAutoPlay(genAudioUrl, id, '', '');
}
} else if (data.status === 'failed' || data.status === 'not_found') {
source.close();
currentSources.delete(id);
removePendingGeneration(id);
removePendingStoryAdd(id);
queryClient.invalidateQueries({ queryKey: ['history'] });
toast({
title: data.status === 'not_found' ? 'Generation not found' : 'Generation failed',
description: data.error || 'An error occurred during generation',
variant: 'destructive',
});
}
} catch {
// Ignore parse errors from heartbeats etc
}
};
source.onerror = () => {
// EventSource auto-reconnects, but if we get repeated errors
// just clean up
source.close();
currentSources.delete(id);
removePendingGeneration(id);
};
currentSources.set(id, source);
}
}, [
pendingIds,
removePendingGeneration,
removePendingStoryAdd,
queryClient,
toast,
setAudioWithAutoPlay,
]);
}
+12 -12
View File
@@ -1,23 +1,23 @@
import { useCallback, useEffect, useRef, useState } from 'react';
import { apiClient } from '@/lib/api/client';
import { useGenerationStore } from '@/stores/generationStore';
import type { ActiveDownloadTask } from '@/lib/api/types';
import { useGenerationStore } from '@/stores/generationStore';
// Polling interval in milliseconds
const POLL_INTERVAL = 2000;
const POLL_INTERVAL = 30000;
/**
* Hook to monitor active tasks (downloads and generations).
* Polls the server periodically to catch downloads triggered from anywhere
* (transcription, generation, explicit download, etc.).
*
*
* Returns the active downloads so components can render download toasts.
*/
export function useRestoreActiveTasks() {
const [activeDownloads, setActiveDownloads] = useState<ActiveDownloadTask[]>([]);
const setIsGenerating = useGenerationStore((state) => state.setIsGenerating);
const setActiveGenerationId = useGenerationStore((state) => state.setActiveGenerationId);
const addPendingGeneration = useGenerationStore((state) => state.addPendingGeneration);
// Track which downloads we've seen to detect new ones
const seenDownloadsRef = useRef<Set<string>>(new Set());
@@ -25,15 +25,15 @@ export function useRestoreActiveTasks() {
try {
const tasks = await apiClient.getActiveTasks();
// Update generation state
// Restore pending generations (e.g., after page refresh)
if (tasks.generations.length > 0) {
setIsGenerating(true);
setActiveGenerationId(tasks.generations[0].task_id);
for (const gen of tasks.generations) {
addPendingGeneration(gen.task_id);
}
} else {
// Only clear if we were tracking a generation
const currentId = useGenerationStore.getState().activeGenerationId;
if (currentId) {
setIsGenerating(false);
setActiveGenerationId(null);
}
}
@@ -41,14 +41,14 @@ export function useRestoreActiveTasks() {
// Update active downloads
// Keep track of all active downloads (including new ones)
const currentDownloadNames = new Set(tasks.downloads.map((d) => d.model_name));
// Remove completed downloads from our seen set
for (const name of seenDownloadsRef.current) {
if (!currentDownloadNames.has(name)) {
seenDownloadsRef.current.delete(name);
}
}
// Add new downloads to seen set
for (const download of tasks.downloads) {
seenDownloadsRef.current.add(download.model_name);
@@ -59,7 +59,7 @@ export function useRestoreActiveTasks() {
// Silently fail - server might be temporarily unavailable
console.debug('Failed to fetch active tasks:', error);
}
}, [setIsGenerating, setActiveGenerationId]);
}, [setActiveGenerationId, addPendingGeneration]);
useEffect(() => {
// Fetch immediately on mount
+61 -8
View File
@@ -1,6 +1,15 @@
import { useMutation, useQuery, useQueryClient } from '@tanstack/react-query';
import { apiClient } from '@/lib/api/client';
import type { StoryCreate, StoryItemCreate, StoryItemBatchUpdate, StoryItemReorder, StoryItemMove, StoryItemTrim, StoryItemSplit } from '@/lib/api/types';
import type {
StoryCreate,
StoryItemBatchUpdate,
StoryItemCreate,
StoryItemMove,
StoryItemReorder,
StoryItemSplit,
StoryItemTrim,
StoryItemVersionUpdate,
} from '@/lib/api/types';
import { usePlatform } from '@/platform/PlatformContext';
export function useStories() {
@@ -109,8 +118,15 @@ export function useMoveStoryItem() {
const queryClient = useQueryClient();
return useMutation({
mutationFn: ({ storyId, itemId, data }: { storyId: string; itemId: string; data: StoryItemMove }) =>
apiClient.moveStoryItem(storyId, itemId, data),
mutationFn: ({
storyId,
itemId,
data,
}: {
storyId: string;
itemId: string;
data: StoryItemMove;
}) => apiClient.moveStoryItem(storyId, itemId, data),
onSuccess: (_, variables) => {
queryClient.invalidateQueries({ queryKey: ['stories'] });
queryClient.invalidateQueries({ queryKey: ['stories', variables.storyId] });
@@ -122,8 +138,15 @@ export function useTrimStoryItem() {
const queryClient = useQueryClient();
return useMutation({
mutationFn: ({ storyId, itemId, data }: { storyId: string; itemId: string; data: StoryItemTrim }) =>
apiClient.trimStoryItem(storyId, itemId, data),
mutationFn: ({
storyId,
itemId,
data,
}: {
storyId: string;
itemId: string;
data: StoryItemTrim;
}) => apiClient.trimStoryItem(storyId, itemId, data),
onSuccess: (_, variables) => {
queryClient.invalidateQueries({ queryKey: ['stories'] });
queryClient.invalidateQueries({ queryKey: ['stories', variables.storyId] });
@@ -135,8 +158,15 @@ export function useSplitStoryItem() {
const queryClient = useQueryClient();
return useMutation({
mutationFn: ({ storyId, itemId, data }: { storyId: string; itemId: string; data: StoryItemSplit }) =>
apiClient.splitStoryItem(storyId, itemId, data),
mutationFn: ({
storyId,
itemId,
data,
}: {
storyId: string;
itemId: string;
data: StoryItemSplit;
}) => apiClient.splitStoryItem(storyId, itemId, data),
onSuccess: (_, variables) => {
queryClient.invalidateQueries({ queryKey: ['stories'] });
queryClient.invalidateQueries({ queryKey: ['stories', variables.storyId] });
@@ -157,6 +187,26 @@ export function useDuplicateStoryItem() {
});
}
export function useSetStoryItemVersion() {
const queryClient = useQueryClient();
return useMutation({
mutationFn: ({
storyId,
itemId,
data,
}: {
storyId: string;
itemId: string;
data: StoryItemVersionUpdate;
}) => apiClient.setStoryItemVersion(storyId, itemId, data),
onSuccess: (_, variables) => {
queryClient.invalidateQueries({ queryKey: ['stories'] });
queryClient.invalidateQueries({ queryKey: ['stories', variables.storyId] });
},
});
}
export function useExportStoryAudio() {
const platform = usePlatform();
@@ -165,7 +215,10 @@ export function useExportStoryAudio() {
const blob = await apiClient.exportStoryAudio(storyId);
// Create safe filename
const safeName = storyName.substring(0, 50).replace(/[^a-z0-9]/gi, '-').toLowerCase();
const safeName = storyName
.substring(0, 50)
.replace(/[^a-z0-9]/gi, '-')
.toLowerCase();
const filename = `${safeName || 'story'}.wav`;
await platform.filesystem.saveFile(filename, blob, [
+23 -11
View File
@@ -70,6 +70,16 @@ export function useStoryPlayback(items: StoryItemDetail[] | undefined) {
}
}, []);
// Resolve the audio buffer key and URL for an item.
// When a version_id is pinned, use that version's audio; otherwise use the generation default.
const getAudioKey = (item: StoryItemDetail) =>
item.version_id ? `v:${item.version_id}` : item.generation_id;
const getAudioUrlForItem = (item: StoryItemDetail) =>
item.version_id
? apiClient.getVersionAudioUrl(item.version_id)
: apiClient.getAudioUrl(item.generation_id);
// Preload audio files as AudioBuffers
useEffect(() => {
if (!items || items.length === 0) {
@@ -78,12 +88,12 @@ export function useStoryPlayback(items: StoryItemDetail[] | undefined) {
return;
}
const currentIds = new Set(items.map((item) => item.generation_id));
const currentKeys = new Set(items.map(getAudioKey));
const audioContext = getAudioContext();
// Remove buffers for items that no longer exist
for (const [id] of audioBuffersRef.current) {
if (!currentIds.has(id)) {
if (!currentKeys.has(id)) {
audioBuffersRef.current.delete(id);
}
}
@@ -91,24 +101,25 @@ export function useStoryPlayback(items: StoryItemDetail[] | undefined) {
// Preload audio for new items
const preloadPromises: Promise<void>[] = [];
for (const item of items) {
if (!audioBuffersRef.current.has(item.generation_id)) {
const audioUrl = apiClient.getAudioUrl(item.generation_id);
console.log('[StoryPlayback] Preloading audio buffer:', item.generation_id);
const key = getAudioKey(item);
if (!audioBuffersRef.current.has(key)) {
const audioUrl = getAudioUrlForItem(item);
console.log('[StoryPlayback] Preloading audio buffer:', key);
const preloadPromise = fetch(audioUrl)
.then((response) => response.arrayBuffer())
.then((arrayBuffer) => audioContext.decodeAudioData(arrayBuffer))
.then((audioBuffer) => {
audioBuffersRef.current.set(item.generation_id, audioBuffer);
audioBuffersRef.current.set(key, audioBuffer);
console.log(
'[StoryPlayback] Preloaded buffer:',
item.generation_id,
key,
'duration:',
audioBuffer.duration,
);
})
.catch((err) => {
console.error('[StoryPlayback] Failed to preload audio:', item.generation_id, err);
console.error('[StoryPlayback] Failed to preload audio:', key, err);
});
preloadPromises.push(preloadPromise);
@@ -216,15 +227,16 @@ export function useStoryPlayback(items: StoryItemDetail[] | undefined) {
// Schedule new sources for items that should be playing
for (const item of shouldBePlaying) {
if (!activeSourcesRef.current.has(item.id)) {
const buffer = audioBuffersRef.current.get(item.generation_id);
const bufferKey = getAudioKey(item);
const buffer = audioBuffersRef.current.get(bufferKey);
if (!buffer) {
console.warn('[StoryPlayback] Buffer not loaded for:', item.generation_id);
console.warn('[StoryPlayback] Buffer not loaded for:', bufferKey);
continue;
}
// Calculate when this item should start in AudioContext time
const itemStartContextTime = storyTimeToContextTime(item.start_time_ms);
// Calculate effective duration and trim offsets
const trimStartSec = (item.trim_start_ms || 0) / 1000;
const trimEndSec = (item.trim_end_ms || 0) / 1000;
+16 -1
View File
@@ -21,10 +21,25 @@ export function formatDate(date: string | Date): string {
} else {
dateObj = date;
}
return formatDistance(dateObj, new Date(), { addSuffix: true }).replace(/^about /i, '');
}
const ENGINE_DISPLAY_NAMES: Record<string, string> = {
qwen: 'Qwen',
luxtts: 'LuxTTS',
chatterbox: 'Chatterbox',
chatterbox_turbo: 'Chatterbox Turbo',
};
export function formatEngineName(engine?: string, modelSize?: string): string {
const name = ENGINE_DISPLAY_NAMES[engine ?? 'qwen'] ?? engine ?? 'Qwen';
if (engine === 'qwen' && modelSize) {
return `${name} ${modelSize}`;
}
return name;
}
export function formatFileSize(bytes: number): string {
if (bytes === 0) return '0 Bytes';
const k = 1024;
+4 -2
View File
@@ -10,6 +10,8 @@ export interface FileFilter {
export interface PlatformFilesystem {
saveFile(filename: string, blob: Blob, filters?: FileFilter[]): Promise<void>;
openPath(path: string): Promise<void>;
pickDirectory(title: string): Promise<string | null>;
}
export interface UpdateStatus {
@@ -49,9 +51,9 @@ export interface PlatformAudio {
}
export interface PlatformLifecycle {
startServer(remote?: boolean): Promise<string>;
startServer(remote?: boolean, modelsDir?: string | null): Promise<string>;
stopServer(): Promise<void>;
restartServer(): Promise<string>;
restartServer(modelsDir?: string | null): Promise<string>;
setKeepServerRunning(keep: boolean): Promise<void>;
setupWindowCloseHandler(): Promise<void>;
onServerReady?: () => void;
+14
View File
@@ -1,6 +1,7 @@
import { createRootRoute, createRoute, createRouter, Outlet } from '@tanstack/react-router';
import { AppFrame } from '@/components/AppFrame/AppFrame';
import { AudioTab } from '@/components/AudioTab/AudioTab';
import { EffectsTab } from '@/components/EffectsTab/EffectsTab';
import { MainEditor } from '@/components/MainEditor/MainEditor';
import { ModelsTab } from '@/components/ModelsTab/ModelsTab';
import { ServerTab } from '@/components/ServerTab/ServerTab';
@@ -8,8 +9,10 @@ import { Sidebar } from '@/components/Sidebar';
import { StoriesTab } from '@/components/StoriesTab/StoriesTab';
import { Toaster } from '@/components/ui/toaster';
import { VoicesTab } from '@/components/VoicesTab/VoicesTab';
import { useGenerationProgress } from '@/lib/hooks/useGenerationProgress';
import { useModelDownloadToast } from '@/lib/hooks/useModelDownloadToast';
import { MODEL_DISPLAY_NAMES, useRestoreActiveTasks } from '@/lib/hooks/useRestoreActiveTasks';
// Simple platform check that works in both web and Tauri
const isMacOS = () => navigator.platform.toLowerCase().includes('mac');
@@ -18,6 +21,9 @@ function RootLayout() {
// Monitor active downloads/generations and show toasts for them
const activeDownloads = useRestoreActiveTasks();
// Subscribe to SSE for pending generations — handles completion, auto-play, and history refresh
useGenerationProgress();
return (
<AppFrame>
<div className="flex flex-1 min-h-0 overflow-hidden">
@@ -100,6 +106,13 @@ const audioRoute = createRoute({
component: AudioTab,
});
// Effects route
const effectsRoute = createRoute({
getParentRoute: () => rootRoute,
path: '/effects',
component: EffectsTab,
});
// Models route
const modelsRoute = createRoute({
getParentRoute: () => rootRoute,
@@ -120,6 +133,7 @@ const routeTree = rootRoute.addChildren([
storiesRoute,
voicesRoute,
audioRoute,
effectsRoute,
modelsRoute,
serverRoute,
]);
+26
View File
@@ -0,0 +1,26 @@
import { create } from 'zustand';
import type { EffectConfig } from '@/lib/api/types';
interface EffectsStore {
selectedPresetId: string | null;
setSelectedPresetId: (id: string | null) => void;
// Working chain for the detail panel (editing a preset or building a new one)
workingChain: EffectConfig[];
setWorkingChain: (chain: EffectConfig[]) => void;
// Track if editing an existing preset vs creating new
isCreatingNew: boolean;
setIsCreatingNew: (v: boolean) => void;
}
export const useEffectsStore = create<EffectsStore>((set) => ({
selectedPresetId: null,
setSelectedPresetId: (id) => set({ selectedPresetId: id, isCreatingNew: false }),
workingChain: [],
setWorkingChain: (chain) => set({ workingChain: chain }),
isCreatingNew: false,
setIsCreatingNew: (v) => set({ isCreatingNew: v, ...(v && { selectedPresetId: null }) }),
}));
+47 -4
View File
@@ -1,15 +1,58 @@
import { create } from 'zustand';
interface GenerationState {
/** IDs of generations currently in progress */
pendingGenerationIds: Set<string>;
/** Whether any generation is in progress (derived from pendingGenerationIds) */
isGenerating: boolean;
activeGenerationId: string | null;
setIsGenerating: (generating: boolean) => void;
/** Map of generationId → storyId for deferred story additions */
pendingStoryAdds: Map<string, string>;
addPendingGeneration: (id: string) => void;
removePendingGeneration: (id: string) => void;
addPendingStoryAdd: (generationId: string, storyId: string) => void;
removePendingStoryAdd: (generationId: string) => string | undefined;
setActiveGenerationId: (id: string | null) => void;
activeGenerationId: string | null;
}
export const useGenerationStore = create<GenerationState>((set) => ({
export const useGenerationStore = create<GenerationState>((set, get) => ({
pendingGenerationIds: new Set(),
isGenerating: false,
activeGenerationId: null,
setIsGenerating: (generating) => set({ isGenerating: generating }),
pendingStoryAdds: new Map(),
addPendingGeneration: (id) =>
set((state) => {
const next = new Set(state.pendingGenerationIds);
next.add(id);
return { pendingGenerationIds: next, isGenerating: true };
}),
removePendingGeneration: (id) =>
set((state) => {
const next = new Set(state.pendingGenerationIds);
next.delete(id);
return { pendingGenerationIds: next, isGenerating: next.size > 0 };
}),
addPendingStoryAdd: (generationId, storyId) =>
set((state) => {
const next = new Map(state.pendingStoryAdds);
next.set(generationId, storyId);
return { pendingStoryAdds: next };
}),
removePendingStoryAdd: (generationId) => {
const storyId = get().pendingStoryAdds.get(generationId);
if (storyId) {
set((state) => {
const next = new Map(state.pendingStoryAdds);
next.delete(generationId);
return { pendingStoryAdds: next };
});
}
return storyId;
},
setActiveGenerationId: (id) => set({ activeGenerationId: id }),
}));
+30
View File
@@ -13,6 +13,21 @@ interface ServerStore {
keepServerRunningOnClose: boolean;
setKeepServerRunningOnClose: (keepRunning: boolean) => void;
maxChunkChars: number;
setMaxChunkChars: (value: number) => void;
crossfadeMs: number;
setCrossfadeMs: (value: number) => void;
normalizeAudio: boolean;
setNormalizeAudio: (value: boolean) => void;
autoplayOnGenerate: boolean;
setAutoplayOnGenerate: (value: boolean) => void;
customModelsDir: string | null;
setCustomModelsDir: (dir: string | null) => void;
}
export const useServerStore = create<ServerStore>()(
@@ -29,6 +44,21 @@ export const useServerStore = create<ServerStore>()(
keepServerRunningOnClose: false,
setKeepServerRunningOnClose: (keepRunning) => set({ keepServerRunningOnClose: keepRunning }),
maxChunkChars: 800,
setMaxChunkChars: (value) => set({ maxChunkChars: value }),
crossfadeMs: 50,
setCrossfadeMs: (value) => set({ crossfadeMs: value }),
normalizeAudio: true,
setNormalizeAudio: (value) => set({ normalizeAudio: value }),
autoplayOnGenerate: true,
setAutoplayOnGenerate: (value) => set({ autoplayOnGenerate: value }),
customModelsDir: null,
setCustomModelsDir: (dir) => set({ customModelsDir: dir }),
}),
{
name: 'voicebox-server',
+7
View File
@@ -31,6 +31,10 @@ interface UIStore {
selectedProfileId: string | null;
setSelectedProfileId: (id: string | null) => void;
// Selected voice in Voices tab inspector
selectedVoiceId: string | null;
setSelectedVoiceId: (id: string | null) => void;
// Profile form draft (for persisting create voice modal state)
profileFormDraft: ProfileFormDraft | null;
setProfileFormDraft: (draft: ProfileFormDraft | null) => void;
@@ -55,6 +59,9 @@ export const useUIStore = create<UIStore>((set) => ({
selectedProfileId: null,
setSelectedProfileId: (id) => set({ selectedProfileId: id }),
selectedVoiceId: null,
setSelectedVoiceId: (id) => set({ selectedVoiceId: id }),
profileFormDraft: null,
setProfileFormDraft: (draft) => set({ profileFormDraft: draft }),
+12 -9
View File
@@ -334,18 +334,21 @@ python -m backend.main --host 0.0.0.0 --port 8000
## Usage Examples
The desktop app, web client, and current development workflow use `http://localhost:17493` by default.
If you launch the backend manually with a different host or port, substitute that address in the examples below.
### Creating a Voice Profile
```bash
# 1. Create profile
curl -X POST http://localhost:8000/profiles \
curl -X POST http://localhost:17493/profiles \
-H "Content-Type: application/json" \
-d '{"name": "My Voice", "language": "en"}'
# Response: {"id": "abc-123", ...}
# 2. Add sample
curl -X POST http://localhost:8000/profiles/abc-123/samples \
curl -X POST http://localhost:17493/profiles/abc-123/samples \
-F "[email protected]" \
-F "reference_text=This is my voice sample"
```
@@ -353,7 +356,7 @@ curl -X POST http://localhost:8000/profiles/abc-123/samples \
### Generating Speech
```bash
curl -X POST http://localhost:8000/generate \
curl -X POST http://localhost:17493/generate \
-H "Content-Type: application/json" \
-d '{
"profile_id": "abc-123",
@@ -365,13 +368,13 @@ curl -X POST http://localhost:8000/generate \
# Response: {"id": "gen-456", "audio_path": "/path/to/audio.wav", ...}
# Download audio
curl http://localhost:8000/audio/gen-456 -o output.wav
curl http://localhost:17493/audio/gen-456 -o output.wav
```
### Transcribing Audio
```bash
curl -X POST http://localhost:8000/transcribe \
curl -X POST http://localhost:17493/transcribe \
-F "[email protected]" \
-F "language=en"
@@ -386,12 +389,12 @@ Add multiple samples to a profile for better quality:
```bash
# Add first sample
curl -X POST http://localhost:8000/profiles/abc-123/samples \
curl -X POST http://localhost:17493/profiles/abc-123/samples \
-F "[email protected]" \
-F "reference_text=First sample"
# Add second sample
curl -X POST http://localhost:8000/profiles/abc-123/samples \
curl -X POST http://localhost:17493/profiles/abc-123/samples \
-F "[email protected]" \
-F "reference_text=Second sample"
@@ -412,10 +415,10 @@ Models are lazy-loaded and can be manually unloaded:
```bash
# Unload TTS model
curl -X POST http://localhost:8000/models/unload
curl -X POST http://localhost:17493/models/unload
# Load specific model size
curl -X POST "http://localhost:8000/models/load?model_size=0.6B"
curl -X POST "http://localhost:17493/models/load?model_size=0.6B"
```
## Error Handling
+1 -1
View File
@@ -1,3 +1,3 @@
# Backend package
__version__ = "0.1.13"
__version__ = "0.2.0"
+8
View File
@@ -121,6 +121,8 @@ _stt_backend: Optional[STTBackend] = None
TTS_ENGINES = {
"qwen": "Qwen TTS",
"luxtts": "LuxTTS",
"chatterbox": "Chatterbox TTS",
"chatterbox_turbo": "Chatterbox Turbo",
}
@@ -167,6 +169,12 @@ def get_tts_backend_for_engine(engine: str) -> TTSBackend:
elif engine == "luxtts":
from .luxtts_backend import LuxTTSBackend
backend = LuxTTSBackend()
elif engine == "chatterbox":
from .chatterbox_backend import ChatterboxTTSBackend
backend = ChatterboxTTSBackend()
elif engine == "chatterbox_turbo":
from .chatterbox_turbo_backend import ChatterboxTurboTTSBackend
backend = ChatterboxTurboTTSBackend()
else:
raise ValueError(f"Unknown TTS engine: {engine}. Supported: {list(TTS_ENGINES.keys())}")
+360
View File
@@ -0,0 +1,360 @@
"""
Chatterbox TTS backend implementation.
Wraps ChatterboxMultilingualTTS from chatterbox-tts for zero-shot
voice cloning. Supports 23 languages including Hebrew. Forces CPU
on macOS due to known MPS tensor issues.
"""
import asyncio
import logging
import platform
import threading
from pathlib import Path
from typing import ClassVar, List, Optional, Tuple
import numpy as np
from . import TTSBackend
from ..utils.audio import normalize_audio, load_audio
from ..utils.progress import get_progress_manager
from ..utils.tasks import get_task_manager
logger = logging.getLogger(__name__)
CHATTERBOX_HF_REPO = "ResembleAI/chatterbox"
# Files that must be present for the multilingual model
_MTL_WEIGHT_FILES = [
"t3_mtl23ls_v2.safetensors",
"s3gen.pt",
"ve.pt",
]
class ChatterboxTTSBackend:
"""Chatterbox Multilingual TTS backend for voice cloning."""
# Class-level lock for torch.load monkey-patching
_load_lock: ClassVar[threading.Lock] = threading.Lock()
def __init__(self):
self.model = None
self.model_size = "default"
self._device = None
self._model_load_lock = asyncio.Lock()
def _get_device(self) -> str:
"""Get the best available device. Forces CPU on macOS (MPS issue)."""
if platform.system() == "Darwin":
return "cpu"
try:
import torch
if torch.cuda.is_available():
return "cuda"
except ImportError:
pass
return "cpu"
def is_loaded(self) -> bool:
return self.model is not None
def _get_model_path(self, model_size: str = "default") -> str:
return CHATTERBOX_HF_REPO
def _is_model_cached(self, model_size: str = "default") -> bool:
"""Check if the Chatterbox multilingual model is cached locally."""
try:
from huggingface_hub import constants as hf_constants
repo_cache = Path(hf_constants.HF_HUB_CACHE) / (
"models--" + CHATTERBOX_HF_REPO.replace("/", "--")
)
if not repo_cache.exists():
return False
blobs_dir = repo_cache / "blobs"
if blobs_dir.exists() and any(blobs_dir.glob("*.incomplete")):
return False
# Check for multilingual weight files
snapshots_dir = repo_cache / "snapshots"
if snapshots_dir.exists():
for fname in _MTL_WEIGHT_FILES:
if not any(snapshots_dir.rglob(fname)):
return False
return True
return False
except Exception as e:
logger.warning(f"Error checking Chatterbox cache: {e}")
return False
async def load_model(self, model_size: str = "default") -> None:
"""Load the Chatterbox multilingual model."""
if self.model is not None:
return
async with self._model_load_lock:
if self.model is not None:
return
await asyncio.to_thread(self._load_model_sync)
def _load_model_sync(self):
"""Synchronous model loading."""
from ..utils.hf_progress import HFProgressTracker, create_hf_progress_callback
progress_manager = get_progress_manager()
task_manager = get_task_manager()
model_name = "chatterbox-tts"
is_cached = self._is_model_cached()
# Set up HF progress tracking (intercepts tqdm for file-level progress)
progress_callback = create_hf_progress_callback(model_name, progress_manager)
tracker = HFProgressTracker(progress_callback, filter_non_downloads=is_cached)
tracker_context = tracker.patch_download()
tracker_context.__enter__()
if not is_cached:
task_manager.start_download(model_name)
progress_manager.update_progress(
model_name=model_name,
current=0,
total=0,
filename="Connecting to HuggingFace...",
status="downloading",
)
try:
device = self._get_device()
self._device = device
logger.info(f"Loading Chatterbox Multilingual TTS on {device}...")
import torch
from chatterbox.mtl_tts import ChatterboxMultilingualTTS
# Load into a local variable first, apply all patches, then
# assign to self.model. This avoids leaving a half-initialised
# model on self.model if any patch step raises an exception.
#
# Monkey-patch torch.load for CPU loading. The model's .pt files
# were saved on CUDA; from_pretrained() doesn't pass map_location
# so loading on CPU fails without this.
try:
if device == "cpu":
_orig_torch_load = torch.load
def _patched_load(*args, **kwargs):
kwargs.setdefault("map_location", "cpu")
return _orig_torch_load(*args, **kwargs)
with ChatterboxTTSBackend._load_lock:
torch.load = _patched_load
try:
model = ChatterboxMultilingualTTS.from_pretrained(
device=device,
)
finally:
torch.load = _orig_torch_load
else:
model = ChatterboxMultilingualTTS.from_pretrained(
device=device,
)
finally:
tracker_context.__exit__(None, None, None)
# Fix: transformers >= 4.36 defaults LlamaModel to sdpa attention
# which doesn't support output_attentions=True (needed by
# Chatterbox's AlignmentStreamAnalyzer). Force eager attention.
t3_tfmr = model.t3.tfmr
if hasattr(t3_tfmr, "config") and hasattr(
t3_tfmr.config, "_attn_implementation"
):
t3_tfmr.config._attn_implementation = "eager"
for layer in getattr(t3_tfmr, "layers", []):
if hasattr(layer, "self_attn"):
layer.self_attn._attn_implementation = "eager"
if not is_cached:
progress_manager.mark_complete(model_name)
task_manager.complete_download(model_name)
# Patch float64 → float32 dtype mismatches in upstream chatterbox.
# librosa.load returns float64 numpy; multiple upstream code paths
# convert it to a torch tensor via torch.from_numpy() without
# casting, then matmul it against float32 model weights.
import types
# Patch S3Tokenizer (used by s3gen.tokenizer)
_tokzr = model.s3gen.tokenizer
_orig_log_mel = _tokzr.log_mel_spectrogram.__func__
def _f32_log_mel(self_tokzr, audio, padding=0):
import torch as _torch
if _torch.is_tensor(audio):
audio = audio.float()
return _orig_log_mel(self_tokzr, audio, padding)
_tokzr.log_mel_spectrogram = types.MethodType(_f32_log_mel, _tokzr)
# Patch VoiceEncoder
_ve = model.ve
_orig_ve_forward = _ve.forward.__func__
def _f32_ve_forward(self_ve, mels):
return _orig_ve_forward(self_ve, mels.float())
_ve.forward = types.MethodType(_f32_ve_forward, _ve)
# All patches applied successfully — publish the model
self.model = model
logger.info("Chatterbox Multilingual TTS loaded successfully")
except ImportError as e:
logger.error(
"chatterbox-tts package not found. "
"Install with: pip install chatterbox-tts"
)
if not is_cached:
progress_manager.mark_error(model_name, str(e))
task_manager.error_download(model_name, str(e))
raise
except Exception as e:
logger.error(f"Failed to load Chatterbox: {e}")
if not is_cached:
progress_manager.mark_error(model_name, str(e))
task_manager.error_download(model_name, str(e))
raise
def unload_model(self) -> None:
"""Unload model to free memory."""
if self.model is not None:
device = self._device
del self.model
self.model = None
self._device = None
if device == "cuda":
import torch
torch.cuda.empty_cache()
logger.info("Chatterbox unloaded")
async def create_voice_prompt(
self,
audio_path: str,
reference_text: str,
use_cache: bool = True,
) -> Tuple[dict, bool]:
"""
Create voice prompt from reference audio.
Chatterbox processes reference audio at generation time, so the
prompt just stores the file path. The actual audio is loaded by
model.generate() via audio_prompt_path.
"""
voice_prompt = {
"ref_audio": str(audio_path),
"ref_text": reference_text,
}
return voice_prompt, False
async def combine_voice_prompts(
self,
audio_paths: List[str],
reference_texts: List[str],
) -> Tuple[np.ndarray, str]:
"""Combine multiple reference samples."""
combined_audio = []
for path in audio_paths:
audio, _sr = load_audio(path)
audio = normalize_audio(audio)
combined_audio.append(audio)
mixed = np.concatenate(combined_audio)
mixed = normalize_audio(mixed)
combined_text = " ".join(reference_texts)
return mixed, combined_text
# Per-language generation defaults. Lower temp + higher cfg = clearer speech.
_LANG_DEFAULTS: ClassVar[dict] = {
"he": {
"exaggeration": 0.4,
"cfg_weight": 0.7,
"temperature": 0.65,
"repetition_penalty": 2.5,
},
}
_GLOBAL_DEFAULTS: ClassVar[dict] = {
"exaggeration": 0.5,
"cfg_weight": 0.5,
"temperature": 0.8,
"repetition_penalty": 2.0,
}
async def generate(
self,
text: str,
voice_prompt: dict,
language: str = "en",
seed: Optional[int] = None,
instruct: Optional[str] = None,
) -> Tuple[np.ndarray, int]:
"""
Generate audio using Chatterbox Multilingual TTS.
Args:
text: Text to synthesize
voice_prompt: Dict with ref_audio path
language: BCP-47 language code
seed: Random seed for reproducibility
instruct: Unused (protocol compatibility)
Returns:
Tuple of (audio_array, sample_rate)
"""
await self.load_model()
ref_audio = voice_prompt.get("ref_audio")
if ref_audio and not Path(ref_audio).exists():
logger.warning(f"Reference audio not found: {ref_audio}")
ref_audio = None
# Merge language-specific defaults with global defaults
lang_defaults = self._LANG_DEFAULTS.get(language, self._GLOBAL_DEFAULTS)
def _generate_sync():
import torch
if seed is not None:
torch.manual_seed(seed)
logger.info(f"[Chatterbox] Generating: lang={language}")
wav = self.model.generate(
text,
language_id=language,
audio_prompt_path=ref_audio,
exaggeration=lang_defaults["exaggeration"],
cfg_weight=lang_defaults["cfg_weight"],
temperature=lang_defaults["temperature"],
repetition_penalty=lang_defaults["repetition_penalty"],
)
# Convert tensor -> numpy
if isinstance(wav, torch.Tensor):
audio = wav.squeeze().cpu().numpy().astype(np.float32)
else:
audio = np.asarray(wav, dtype=np.float32)
sample_rate = (
getattr(self.model, "sr", None)
or getattr(self.model, "sample_rate", 24000)
)
return audio, sample_rate
return await asyncio.to_thread(_generate_sync)
@@ -0,0 +1,345 @@
"""
Chatterbox Turbo TTS backend implementation.
Wraps ChatterboxTurboTTS from chatterbox-tts for fast, English-only
voice cloning with paralinguistic tag support ([laugh], [cough], etc.).
Forces CPU on macOS due to known MPS tensor issues.
"""
import asyncio
import logging
import platform
import threading
from pathlib import Path
from typing import ClassVar, List, Optional, Tuple
import numpy as np
from . import TTSBackend
from ..utils.audio import normalize_audio, load_audio
from ..utils.progress import get_progress_manager
from ..utils.tasks import get_task_manager
logger = logging.getLogger(__name__)
CHATTERBOX_TURBO_HF_REPO = "ResembleAI/chatterbox-turbo"
# Files that must be present for the turbo model
_TURBO_WEIGHT_FILES = [
"t3_turbo_v1.safetensors",
"s3gen_meanflow.safetensors",
"ve.safetensors",
]
class ChatterboxTurboTTSBackend:
"""Chatterbox Turbo TTS backend — fast, English-only, with paralinguistic tags."""
# Class-level lock for torch.load monkey-patching
_load_lock: ClassVar[threading.Lock] = threading.Lock()
def __init__(self):
self.model = None
self.model_size = "default"
self._device = None
self._model_load_lock = asyncio.Lock()
def _get_device(self) -> str:
"""Get the best available device. Forces CPU on macOS (MPS issue)."""
if platform.system() == "Darwin":
return "cpu"
try:
import torch
if torch.cuda.is_available():
return "cuda"
except ImportError:
pass
return "cpu"
def is_loaded(self) -> bool:
return self.model is not None
def _get_model_path(self, model_size: str = "default") -> str:
return CHATTERBOX_TURBO_HF_REPO
def _is_model_cached(self, model_size: str = "default") -> bool:
"""Check if the Chatterbox Turbo model is cached locally."""
try:
from huggingface_hub import constants as hf_constants
repo_cache = Path(hf_constants.HF_HUB_CACHE) / (
"models--" + CHATTERBOX_TURBO_HF_REPO.replace("/", "--")
)
if not repo_cache.exists():
return False
blobs_dir = repo_cache / "blobs"
if blobs_dir.exists() and any(blobs_dir.glob("*.incomplete")):
return False
# Check for turbo weight files
snapshots_dir = repo_cache / "snapshots"
if snapshots_dir.exists():
for fname in _TURBO_WEIGHT_FILES:
if not any(snapshots_dir.rglob(fname)):
return False
return True
return False
except Exception as e:
logger.warning(f"Error checking Chatterbox Turbo cache: {e}")
return False
async def load_model(self, model_size: str = "default") -> None:
"""Load the Chatterbox Turbo model."""
if self.model is not None:
return
async with self._model_load_lock:
if self.model is not None:
return
await asyncio.to_thread(self._load_model_sync)
def _load_model_sync(self):
"""Synchronous model loading."""
from ..utils.hf_progress import HFProgressTracker, create_hf_progress_callback
progress_manager = get_progress_manager()
task_manager = get_task_manager()
model_name = "chatterbox-turbo"
is_cached = self._is_model_cached()
# Set up HF progress tracking (intercepts tqdm for file-level progress)
progress_callback = create_hf_progress_callback(model_name, progress_manager)
tracker = HFProgressTracker(progress_callback, filter_non_downloads=is_cached)
tracker_context = tracker.patch_download()
tracker_context.__enter__()
if not is_cached:
task_manager.start_download(model_name)
progress_manager.update_progress(
model_name=model_name,
current=0,
total=0,
filename="Connecting to HuggingFace...",
status="downloading",
)
try:
device = self._get_device()
self._device = device
logger.info(f"Loading Chatterbox Turbo TTS on {device}...")
import torch
from huggingface_hub import snapshot_download
from chatterbox.tts_turbo import ChatterboxTurboTTS
# Download model files ourselves so we can pass token=None
# (upstream from_pretrained passes token=True which requires
# a stored HF token even though the repo is public).
try:
local_path = snapshot_download(
repo_id=CHATTERBOX_TURBO_HF_REPO,
token=None,
allow_patterns=[
"*.safetensors", "*.json", "*.txt", "*.pt", "*.model",
],
)
finally:
tracker_context.__exit__(None, None, None)
# Monkey-patch torch.load for CPU loading. The model's .pt files
# were saved on CUDA; from_local() doesn't pass map_location
# so loading on CPU fails without this.
# Load into a local var, apply patches, then publish to
# self.model so a failed patch doesn't leave us half-initialised.
if device == "cpu":
_orig_torch_load = torch.load
def _patched_load(*args, **kwargs):
kwargs.setdefault("map_location", "cpu")
return _orig_torch_load(*args, **kwargs)
with ChatterboxTurboTTSBackend._load_lock:
torch.load = _patched_load
try:
model = ChatterboxTurboTTS.from_local(
local_path, device,
)
finally:
torch.load = _orig_torch_load
else:
model = ChatterboxTurboTTS.from_local(
local_path, device,
)
if not is_cached:
progress_manager.mark_complete(model_name)
task_manager.complete_download(model_name)
# Patch float64 → float32 dtype mismatches in upstream chatterbox.
# librosa.load returns float64 numpy; multiple upstream code paths
# convert it to a torch tensor via torch.from_numpy() without
# casting, then matmul it against float32 model weights.
# We patch the two known entry points:
#
# 1. S3Tokenizer.log_mel_spectrogram — the audio tensor from
# librosa hits _mel_filters (float32) in a matmul.
# 2. VoiceEncoder.forward — float64 mel spectrograms hit the
# float32 LSTM weights.
import types
# Patch S3Tokenizer (used by s3gen.tokenizer)
_tokzr = model.s3gen.tokenizer
_orig_log_mel = _tokzr.log_mel_spectrogram.__func__
def _f32_log_mel(self_tokzr, audio, padding=0):
import torch as _torch
if _torch.is_tensor(audio):
audio = audio.float()
return _orig_log_mel(self_tokzr, audio, padding)
_tokzr.log_mel_spectrogram = types.MethodType(_f32_log_mel, _tokzr)
# Patch VoiceEncoder
_ve = model.ve
_orig_ve_forward = _ve.forward.__func__
def _f32_ve_forward(self_ve, mels):
return _orig_ve_forward(self_ve, mels.float())
_ve.forward = types.MethodType(_f32_ve_forward, _ve)
# Only publish after all patches succeed
self.model = model
logger.info("Chatterbox Turbo TTS loaded successfully")
except ImportError as e:
logger.error(
"chatterbox-tts package not found. "
"Install with: pip install chatterbox-tts"
)
if not is_cached:
progress_manager.mark_error(model_name, str(e))
task_manager.error_download(model_name, str(e))
raise
except Exception as e:
logger.error(f"Failed to load Chatterbox Turbo: {e}")
if not is_cached:
progress_manager.mark_error(model_name, str(e))
task_manager.error_download(model_name, str(e))
raise
def unload_model(self) -> None:
"""Unload model to free memory."""
if self.model is not None:
device = self._device
del self.model
self.model = None
self._device = None
if device == "cuda":
import torch
torch.cuda.empty_cache()
logger.info("Chatterbox Turbo unloaded")
async def create_voice_prompt(
self,
audio_path: str,
reference_text: str,
use_cache: bool = True,
) -> Tuple[dict, bool]:
"""
Create voice prompt from reference audio.
Chatterbox Turbo processes reference audio at generation time, so the
prompt just stores the file path.
"""
voice_prompt = {
"ref_audio": str(audio_path),
"ref_text": reference_text,
}
return voice_prompt, False
async def combine_voice_prompts(
self,
audio_paths: List[str],
reference_texts: List[str],
) -> Tuple[np.ndarray, str]:
"""Combine multiple reference samples."""
combined_audio = []
for path in audio_paths:
audio, _sr = load_audio(path)
audio = normalize_audio(audio)
combined_audio.append(audio)
mixed = np.concatenate(combined_audio)
mixed = normalize_audio(mixed)
combined_text = " ".join(reference_texts)
return mixed, combined_text
async def generate(
self,
text: str,
voice_prompt: dict,
language: str = "en",
seed: Optional[int] = None,
instruct: Optional[str] = None,
) -> Tuple[np.ndarray, int]:
"""
Generate audio using Chatterbox Turbo TTS.
Supports paralinguistic tags in text: [laugh], [cough], [chuckle], etc.
Args:
text: Text to synthesize (may include paralinguistic tags)
voice_prompt: Dict with ref_audio path
language: Ignored (Turbo is English-only)
seed: Random seed for reproducibility
instruct: Unused (protocol compatibility)
Returns:
Tuple of (audio_array, sample_rate)
"""
await self.load_model()
ref_audio = voice_prompt.get("ref_audio")
if ref_audio and not Path(ref_audio).exists():
logger.warning(f"Reference audio not found: {ref_audio}")
ref_audio = None
def _generate_sync():
import torch
if seed is not None:
torch.manual_seed(seed)
logger.info("[Chatterbox Turbo] Generating (English)")
wav = self.model.generate(
text,
audio_prompt_path=ref_audio,
temperature=0.8,
top_k=1000,
top_p=0.95,
repetition_penalty=1.2,
)
# Convert tensor -> numpy
if isinstance(wav, torch.Tensor):
audio = wav.squeeze().cpu().numpy().astype(np.float32)
else:
audio = np.asarray(wav, dtype=np.float32)
sample_rate = (
getattr(self.model, "sr", None)
or getattr(self.model, "sample_rate", 24000)
)
return audio, sample_rate
return await asyncio.to_thread(_generate_sync)
+25 -14
View File
@@ -94,19 +94,27 @@ class LuxTTSBackend:
def _load_model_sync(self):
"""Synchronous model loading."""
from ..utils.hf_progress import HFProgressTracker, create_hf_progress_callback
progress_manager = get_progress_manager()
task_manager = get_task_manager()
model_name = "luxtts"
is_cached = self._is_model_cached()
# Set up HF progress tracking (intercepts tqdm for file-level progress)
progress_callback = create_hf_progress_callback(model_name, progress_manager)
tracker = HFProgressTracker(progress_callback, filter_non_downloads=is_cached)
tracker_context = tracker.patch_download()
tracker_context.__enter__()
if not is_cached:
task_manager.start_download(model_name)
progress_manager.update_progress(
model_name=model_name,
current=0,
total=0,
filename="Downloading LuxTTS model...",
filename="Connecting to HuggingFace...",
status="downloading",
)
@@ -117,19 +125,22 @@ class LuxTTSBackend:
logger.info(f"Loading LuxTTS on {device}...")
# LuxTTS constructor downloads model and loads everything
if device == "cpu":
import os
threads = os.cpu_count() or 4
self.model = LuxTTS(
model_path=LUXTTS_HF_REPO,
device="cpu",
threads=min(threads, 8),
)
else:
self.model = LuxTTS(
model_path=LUXTTS_HF_REPO,
device=device,
)
try:
if device == "cpu":
import os
threads = os.cpu_count() or 4
self.model = LuxTTS(
model_path=LUXTTS_HF_REPO,
device="cpu",
threads=min(threads, 8),
)
else:
self.model = LuxTTS(
model_path=LUXTTS_HF_REPO,
device=device,
)
finally:
tracker_context.__exit__(None, None, None)
if not is_cached:
progress_manager.mark_complete(model_name)
+39 -5
View File
@@ -5,8 +5,15 @@ MLX backend implementation for TTS and STT using mlx-audio.
from typing import Optional, List, Tuple
import asyncio
import numpy as np
import os
from pathlib import Path
# PATCH: Import and apply offline patch BEFORE any huggingface_hub usage
# This prevents mlx_audio from making network requests when models are cached
from ..utils.hf_offline_patch import patch_huggingface_hub_offline, ensure_original_qwen_config_cached
patch_huggingface_hub_offline()
ensure_original_qwen_config_cached()
from . import TTSBackend, STTBackend
from ..utils.cache import get_cache_key, get_cached_voice_prompt, cache_voice_prompt
from ..utils.audio import normalize_audio, load_audio
@@ -14,6 +21,12 @@ from ..utils.progress import get_progress_manager
from ..utils.hf_progress import HFProgressTracker, create_hf_progress_callback
from ..utils.tasks import get_task_manager
LANGUAGE_CODE_TO_NAME = {
"zh": "chinese", "en": "english", "ja": "japanese", "ko": "korean",
"de": "german", "fr": "french", "ru": "russian", "pt": "portuguese",
"es": "spanish", "it": "italian",
}
class MLXTTSBackend:
"""MLX-based TTS backend using mlx-audio."""
@@ -159,15 +172,35 @@ class MLXTTSBackend:
tracker_context = tracker.patch_download()
tracker_context.__enter__()
# PATCH: Force offline mode when model is already cached
# This prevents crashes when HuggingFace is unreachable
original_hf_hub_offline = os.environ.get("HF_HUB_OFFLINE")
if is_cached:
os.environ["HF_HUB_OFFLINE"] = "1"
print(f"[PATCH] Model {model_size} is cached, forcing HF_HUB_OFFLINE=1 to avoid network requests")
# Import mlx_audio AFTER patching tqdm
from mlx_audio.tts import load
# Load MLX model (downloads automatically)
try:
self.model = load(model_path)
except Exception as load_error:
# If offline mode failed, try with network enabled as fallback
if is_cached and "offline" in str(load_error).lower():
print(f"[PATCH] Offline load failed, trying with network: {load_error}")
os.environ.pop("HF_HUB_OFFLINE", None)
self.model = load(model_path)
else:
raise
finally:
# Exit the patch context
tracker_context.__exit__(None, None, None)
# Restore original HF_HUB_OFFLINE setting
if original_hf_hub_offline is not None:
os.environ["HF_HUB_OFFLINE"] = original_hf_hub_offline
else:
os.environ.pop("HF_HUB_OFFLINE", None)
# Only mark download as complete if we were tracking it
if not is_cached:
@@ -316,7 +349,8 @@ class MLXTTSBackend:
# MLX generate() returns a generator yielding GenerationResult objects
audio_chunks = []
sample_rate = 24000
lang = LANGUAGE_CODE_TO_NAME.get(language, "auto")
# Set seed if provided (MLX uses numpy random)
if seed is not None:
import mlx.core as mx
@@ -344,23 +378,23 @@ class MLXTTSBackend:
sig = inspect.signature(self.model.generate)
if "ref_audio" in sig.parameters:
# Generate with voice cloning
for result in self.model.generate(text, ref_audio=ref_audio, ref_text=ref_text):
for result in self.model.generate(text, ref_audio=ref_audio, ref_text=ref_text, lang_code=lang):
audio_chunks.append(np.array(result.audio))
sample_rate = result.sample_rate
else:
# Fallback: generate without voice cloning
for result in self.model.generate(text):
for result in self.model.generate(text, lang_code=lang):
audio_chunks.append(np.array(result.audio))
sample_rate = result.sample_rate
else:
# No voice prompt, generate normally
for result in self.model.generate(text):
for result in self.model.generate(text, lang_code=lang):
audio_chunks.append(np.array(result.audio))
sample_rate = result.sample_rate
except Exception as e:
# If voice cloning fails, try without it
print(f"Warning: Voice cloning failed, generating without voice prompt: {e}")
for result in self.model.generate(text):
for result in self.model.generate(text, lang_code=lang):
audio_chunks.append(np.array(result.audio))
sample_rate = result.sample_rate
+13 -6
View File
@@ -15,6 +15,12 @@ from ..utils.progress import get_progress_manager
from ..utils.hf_progress import HFProgressTracker, create_hf_progress_callback
from ..utils.tasks import get_task_manager
LANGUAGE_CODE_TO_NAME = {
"zh": "chinese", "en": "english", "ja": "japanese", "ko": "korean",
"de": "german", "fr": "french", "ru": "russian", "pt": "portuguese",
"es": "spanish", "it": "italian",
}
class PyTorchTTSBackend:
"""PyTorch-based TTS backend using Qwen3-TTS."""
@@ -359,6 +365,7 @@ class PyTorchTTSBackend:
wavs, sample_rate = self.model.generate_voice_clone(
text=text,
voice_clone_prompt=voice_prompt,
language=LANGUAGE_CODE_TO_NAME.get(language, "auto"),
instruct=instruct,
)
return wavs[0], sample_rate
@@ -374,6 +381,7 @@ WHISPER_HF_REPOS = {
"small": "openai/whisper-small",
"medium": "openai/whisper-medium",
"large": "openai/whisper-large-v3",
"turbo": "openai/whisper-large-v3-turbo",
}
@@ -591,21 +599,20 @@ class PyTorchSTTBackend:
)
inputs = inputs.to(self.device)
# Set language if provided
forced_decoder_ids = None
# Generate transcription
# If language is provided, force it; otherwise let Whisper auto-detect
generate_kwargs = {}
if language:
# Support all languages from frontend: en, zh, ja, ko, de, fr, ru, pt, es, it
# Whisper supports these and many more
forced_decoder_ids = self.processor.get_decoder_prompt_ids(
language=language,
task="transcribe",
)
generate_kwargs["forced_decoder_ids"] = forced_decoder_ids
# Generate transcription
with torch.no_grad():
predicted_ids = self.model.generate(
inputs["input_features"],
forced_decoder_ids=forced_decoder_ids,
**generate_kwargs,
)
# Decode
+12
View File
@@ -90,6 +90,18 @@ def build_server(cuda=False):
'--hidden-import', 'torch.cuda',
'--hidden-import', 'torch.backends.cudnn',
])
else:
# Exclude NVIDIA CUDA packages from CPU-only builds to keep binary under 4GB.
# On Linux, pip may pull CUDA-enabled PyTorch by default which includes ~3GB
# of NVIDIA shared libraries that PyInstaller would bundle.
nvidia_packages = [
'nvidia', 'nvidia.cublas', 'nvidia.cuda_cupti', 'nvidia.cuda_nvrtc',
'nvidia.cuda_runtime', 'nvidia.cudnn', 'nvidia.cufft', 'nvidia.curand',
'nvidia.cusolver', 'nvidia.cusparse', 'nvidia.nccl', 'nvidia.nvjitlink',
'nvidia.nvtx',
]
for pkg in nvidia_packages:
args.extend(['--exclude-module', pkg])
# Add MLX-specific imports if building on Apple Silicon (never for CUDA builds)
if is_apple_silicon() and not cuda:
+191 -2
View File
@@ -23,6 +23,7 @@ class VoiceProfile(Base):
description = Column(Text)
language = Column(String, default="en")
avatar_path = Column(String, nullable=True)
effects_chain = Column(Text, nullable=True) # JSON-serialized default effects chain
created_at = Column(DateTime, default=datetime.utcnow)
updated_at = Column(DateTime, default=datetime.utcnow, onupdate=datetime.utcnow)
@@ -45,10 +46,15 @@ class Generation(Base):
profile_id = Column(String, ForeignKey("profiles.id"), nullable=False)
text = Column(Text, nullable=False)
language = Column(String, default="en")
audio_path = Column(String, nullable=False)
duration = Column(Float, nullable=False)
audio_path = Column(String, nullable=True)
duration = Column(Float, nullable=True)
seed = Column(Integer)
instruct = Column(Text)
engine = Column(String, default="qwen")
model_size = Column(String, nullable=True)
status = Column(String, default="completed") # generating, completed, failed
error = Column(Text, nullable=True)
is_favorited = Column(Boolean, default=False)
created_at = Column(DateTime, default=datetime.utcnow)
@@ -70,6 +76,7 @@ class StoryItem(Base):
id = Column(String, primary_key=True, default=lambda: str(uuid.uuid4()))
story_id = Column(String, ForeignKey("stories.id"), nullable=False)
generation_id = Column(String, ForeignKey("generations.id"), nullable=False)
version_id = Column(String, ForeignKey("generation_versions.id"), nullable=True) # Pin to specific version, null = use generation default
start_time_ms = Column(Integer, nullable=False, default=0) # Milliseconds from story start
track = Column(Integer, nullable=False, default=0) # Track number (0 = main track)
trim_start_ms = Column(Integer, nullable=False, default=0) # Milliseconds trimmed from start
@@ -88,6 +95,33 @@ class Project(Base):
updated_at = Column(DateTime, default=datetime.utcnow, onupdate=datetime.utcnow)
class GenerationVersion(Base):
"""A version of a generation's audio (clean, processed, alternate takes)."""
__tablename__ = "generation_versions"
id = Column(String, primary_key=True, default=lambda: str(uuid.uuid4()))
generation_id = Column(String, ForeignKey("generations.id"), nullable=False)
label = Column(String, nullable=False) # "clean", "processed", or user-defined
audio_path = Column(String, nullable=False)
effects_chain = Column(Text, nullable=True) # JSON-serialized effects config, null for clean
source_version_id = Column(String, ForeignKey("generation_versions.id"), nullable=True) # Which version was used as input
is_default = Column(Boolean, default=False)
created_at = Column(DateTime, default=datetime.utcnow)
class EffectPreset(Base):
"""Saved effect chain preset."""
__tablename__ = "effect_presets"
id = Column(String, primary_key=True, default=lambda: str(uuid.uuid4()))
name = Column(String, unique=True, nullable=False)
description = Column(Text, nullable=True)
effects_chain = Column(Text, nullable=False) # JSON-serialized effects config
is_builtin = Column(Boolean, default=False)
sort_order = Column(Integer, default=100)
created_at = Column(DateTime, default=datetime.utcnow)
class AudioChannel(Base):
"""Audio channel (bus) database model."""
__tablename__ = "audio_channels"
@@ -165,6 +199,12 @@ def init_db():
finally:
db.close()
# Backfill: create "clean" GenerationVersion entries for existing generations
_backfill_generation_versions()
# Seed built-in effect presets
_seed_builtin_presets()
def _run_migrations(engine):
"""Run database migrations."""
@@ -288,6 +328,155 @@ def _run_migrations(engine):
conn.commit()
print("Added avatar_path column to profiles")
# Migration: Add status and error columns to generations table
if 'generations' in inspector.get_table_names():
columns = {col['name'] for col in inspector.get_columns('generations')}
if 'status' not in columns:
print("Migrating generations: adding status column")
with engine.connect() as conn:
conn.execute(text("ALTER TABLE generations ADD COLUMN status VARCHAR DEFAULT 'completed'"))
conn.commit()
print("Added status column to generations")
if 'error' not in columns:
print("Migrating generations: adding error column")
with engine.connect() as conn:
conn.execute(text("ALTER TABLE generations ADD COLUMN error TEXT"))
conn.commit()
print("Added error column to generations")
if 'engine' not in columns:
print("Migrating generations: adding engine column")
with engine.connect() as conn:
conn.execute(text("ALTER TABLE generations ADD COLUMN engine VARCHAR DEFAULT 'qwen'"))
conn.commit()
print("Added engine column to generations")
# Re-read columns after engine migration (variable name shadows outer `engine`)
columns = {col['name'] for col in inspector.get_columns('generations')}
if 'model_size' not in columns:
print("Migrating generations: adding model_size column")
with engine.connect() as conn:
conn.execute(text("ALTER TABLE generations ADD COLUMN model_size VARCHAR"))
conn.commit()
print("Added model_size column to generations")
# Migration: Add effects_chain to profiles table
if 'profiles' in inspector.get_table_names():
columns = {col['name'] for col in inspector.get_columns('profiles')}
if 'effects_chain' not in columns:
print("Migrating profiles: adding effects_chain column")
with engine.connect() as conn:
conn.execute(text("ALTER TABLE profiles ADD COLUMN effects_chain TEXT"))
conn.commit()
print("Added effects_chain column to profiles")
# Migration: Add sort_order to effect_presets table
if 'effect_presets' in inspector.get_table_names():
columns = {col['name'] for col in inspector.get_columns('effect_presets')}
if 'sort_order' not in columns:
print("Migrating effect_presets: adding sort_order column")
with engine.connect() as conn:
conn.execute(text("ALTER TABLE effect_presets ADD COLUMN sort_order INTEGER DEFAULT 100"))
conn.commit()
print("Added sort_order column to effect_presets")
# Migration: Add version_id column to story_items table
if 'story_items' in inspector.get_table_names():
columns = {col['name'] for col in inspector.get_columns('story_items')}
if 'version_id' not in columns:
print("Migrating story_items: adding version_id column")
with engine.connect() as conn:
conn.execute(text("ALTER TABLE story_items ADD COLUMN version_id VARCHAR"))
conn.commit()
print("Added version_id column to story_items")
# Migration: Add source_version_id to generation_versions table
if 'generation_versions' in inspector.get_table_names():
columns = {col['name'] for col in inspector.get_columns('generation_versions')}
if 'source_version_id' not in columns:
print("Migrating generation_versions: adding source_version_id column")
with engine.connect() as conn:
conn.execute(text("ALTER TABLE generation_versions ADD COLUMN source_version_id VARCHAR"))
conn.commit()
print("Added source_version_id column to generation_versions")
if 'generations' in inspector.get_table_names():
columns = {col['name'] for col in inspector.get_columns('generations')}
if 'is_favorited' not in columns:
print("Migrating generations: adding is_favorited column")
with engine.connect() as conn:
conn.execute(text("ALTER TABLE generations ADD COLUMN is_favorited BOOLEAN DEFAULT 0"))
conn.commit()
print("Added is_favorited column to generations")
# Migration: Create generation_versions for existing generations
# (populate after tables are created, handled in init_db)
def _backfill_generation_versions():
"""Create 'clean' version entries for existing generations that don't have any."""
db = SessionLocal()
try:
from pathlib import Path as _Path
# Find generations that have no version entries
existing_version_gen_ids = {
row[0] for row in db.query(GenerationVersion.generation_id).all()
}
generations = db.query(Generation).filter(
Generation.status == "completed",
Generation.audio_path.isnot(None),
Generation.audio_path != "",
).all()
count = 0
for gen in generations:
if gen.id in existing_version_gen_ids:
continue
if not _Path(gen.audio_path).exists():
continue
version = GenerationVersion(
id=str(uuid.uuid4()),
generation_id=gen.id,
label="clean",
audio_path=gen.audio_path,
effects_chain=None,
is_default=True,
)
db.add(version)
count += 1
if count > 0:
db.commit()
print(f"Backfilled {count} generation version entries")
finally:
db.close()
def _seed_builtin_presets():
"""Ensure built-in effect presets exist in the database."""
import json
from .utils.effects import BUILTIN_PRESETS
db = SessionLocal()
try:
for idx, (key, preset_data) in enumerate(BUILTIN_PRESETS.items()):
sort_order = preset_data.get("sort_order", idx)
existing = db.query(EffectPreset).filter_by(name=preset_data["name"]).first()
if not existing:
preset = EffectPreset(
id=str(uuid.uuid4()),
name=preset_data["name"],
description=preset_data.get("description"),
effects_chain=json.dumps(preset_data["effects_chain"]),
is_builtin=True,
sort_order=sort_order,
)
db.add(preset)
elif existing.sort_order != sort_order:
existing.sort_order = sort_order
db.commit()
finally:
db.close()
def get_db():
"""Get database session (generator for dependency injection)."""
+120
View File
@@ -0,0 +1,120 @@
"""
Effect presets CRUD operations.
"""
from __future__ import annotations
import json
import uuid
from typing import List, Optional
from sqlalchemy.orm import Session
from sqlalchemy.exc import IntegrityError
from .database import EffectPreset as DBEffectPreset
from .models import EffectPresetResponse, EffectPresetCreate, EffectPresetUpdate, EffectConfig
def _preset_response(p: DBEffectPreset) -> EffectPresetResponse:
"""Convert a DB preset row to a Pydantic response."""
effects_chain = [EffectConfig(**e) for e in json.loads(p.effects_chain)]
return EffectPresetResponse(
id=p.id,
name=p.name,
description=p.description,
effects_chain=effects_chain,
is_builtin=p.is_builtin or False,
created_at=p.created_at,
)
def list_presets(db: Session) -> List[EffectPresetResponse]:
"""List all effect presets (built-in + user-created)."""
presets = db.query(DBEffectPreset).order_by(DBEffectPreset.sort_order, DBEffectPreset.name).all()
return [_preset_response(p) for p in presets]
def get_preset(preset_id: str, db: Session) -> Optional[EffectPresetResponse]:
"""Get a preset by ID."""
p = db.query(DBEffectPreset).filter_by(id=preset_id).first()
if not p:
return None
return _preset_response(p)
def get_preset_by_name(name: str, db: Session) -> Optional[EffectPresetResponse]:
"""Get a preset by name."""
p = db.query(DBEffectPreset).filter_by(name=name).first()
if not p:
return None
return _preset_response(p)
def create_preset(data: EffectPresetCreate, db: Session) -> EffectPresetResponse:
"""Create a new user effect preset."""
from .utils.effects import validate_effects_chain
chain_dicts = [e.model_dump() for e in data.effects_chain]
error = validate_effects_chain(chain_dicts)
if error:
raise ValueError(error)
# Check for duplicate name before insert
existing = db.query(DBEffectPreset).filter_by(name=data.name).first()
if existing:
raise ValueError(f"A preset named '{data.name}' already exists")
preset = DBEffectPreset(
id=str(uuid.uuid4()),
name=data.name,
description=data.description,
effects_chain=json.dumps(chain_dicts),
is_builtin=False,
)
db.add(preset)
try:
db.commit()
except IntegrityError:
db.rollback()
raise ValueError(f"A preset named '{data.name}' already exists")
db.refresh(preset)
return _preset_response(preset)
def update_preset(preset_id: str, data: EffectPresetUpdate, db: Session) -> Optional[EffectPresetResponse]:
"""Update a user effect preset. Cannot modify built-in presets."""
preset = db.query(DBEffectPreset).filter_by(id=preset_id).first()
if not preset:
return None
if preset.is_builtin:
raise ValueError("Cannot modify built-in presets")
if data.name is not None:
preset.name = data.name
if data.description is not None:
preset.description = data.description
if data.effects_chain is not None:
from .utils.effects import validate_effects_chain
chain_dicts = [e.model_dump() for e in data.effects_chain]
error = validate_effects_chain(chain_dicts)
if error:
raise ValueError(error)
preset.effects_chain = json.dumps(chain_dicts)
db.commit()
db.refresh(preset)
return _preset_response(preset)
def delete_preset(preset_id: str, db: Session) -> bool:
"""Delete a user effect preset. Cannot delete built-in presets."""
preset = db.query(DBEffectPreset).filter_by(id=preset_id).first()
if not preset:
return False
if preset.is_builtin:
raise ValueError("Cannot delete built-in presets")
db.delete(preset)
db.commit()
return True
+37 -11
View File
@@ -13,7 +13,7 @@ from typing import Optional
from sqlalchemy.orm import Session
from .models import VoiceProfileResponse
from .database import VoiceProfile as DBVoiceProfile, ProfileSample as DBProfileSample, Generation as DBGeneration
from .database import VoiceProfile as DBVoiceProfile, ProfileSample as DBProfileSample, Generation as DBGeneration, GenerationVersion as DBGenerationVersion
from .profiles import create_profile, add_profile_sample
from .models import VoiceProfileCreate
from . import config
@@ -269,16 +269,33 @@ def export_generation_to_zip(generation_id: str, db: Session) -> bytes:
if not profile:
raise ValueError(f"Profile {generation.profile_id} not found")
# Get audio file
audio_path = Path(generation.audio_path)
if not audio_path.exists():
raise ValueError(f"Audio file not found: {audio_path}")
# Get all versions for this generation
versions = (
db.query(DBGenerationVersion)
.filter_by(generation_id=generation_id)
.order_by(DBGenerationVersion.created_at)
.all()
)
# Create ZIP in memory
zip_buffer = io.BytesIO()
with zipfile.ZipFile(zip_buffer, 'w', zipfile.ZIP_DEFLATED) as zip_file:
# Create manifest.json
# Build version manifest entries
version_entries = []
for v in versions:
v_path = Path(v.audio_path)
effects_chain = None
if v.effects_chain:
effects_chain = json.loads(v.effects_chain)
version_entries.append({
"id": v.id,
"label": v.label,
"is_default": v.is_default,
"effects_chain": effects_chain,
"filename": v_path.name,
})
manifest = {
"version": "1.0",
"generation": {
@@ -295,13 +312,22 @@ def export_generation_to_zip(generation_id: str, db: Session) -> bytes:
"name": profile.name,
"description": profile.description,
"language": profile.language,
}
},
"versions": version_entries,
}
zip_file.writestr("manifest.json", json.dumps(manifest, indent=2))
# Add audio file
filename = audio_path.name
zip_file.write(audio_path, f"audio/{filename}")
# Add all version audio files
for v in versions:
v_path = Path(v.audio_path)
if v_path.exists():
zip_file.write(v_path, f"audio/{v_path.name}")
# Fallback: if no versions exist, include the generation's main audio
if not versions:
audio_path = Path(generation.audio_path)
if audio_path.exists():
zip_file.write(audio_path, f"audio/{audio_path.name}")
zip_buffer.seek(0)
return zip_buffer.read()
+96 -9
View File
@@ -10,8 +10,8 @@ from pathlib import Path
from sqlalchemy.orm import Session
from sqlalchemy import or_
from .models import GenerationRequest, GenerationResponse, HistoryQuery, HistoryResponse, HistoryListResponse
from .database import Generation as DBGeneration, VoiceProfile as DBVoiceProfile
from .models import GenerationRequest, GenerationResponse, HistoryQuery, HistoryResponse, HistoryListResponse, GenerationVersionResponse, EffectConfig
from .database import Generation as DBGeneration, GenerationVersion as DBGenerationVersion, VoiceProfile as DBVoiceProfile
from . import config
@@ -20,6 +20,43 @@ def _get_generations_dir() -> Path:
return config.get_generations_dir()
def _get_versions_for_generation(generation_id: str, db: Session) -> tuple:
"""Get versions list and active version ID for a generation."""
import json
versions_rows = (
db.query(DBGenerationVersion)
.filter_by(generation_id=generation_id)
.order_by(DBGenerationVersion.created_at)
.all()
)
if not versions_rows:
return None, None
versions = []
active_version_id = None
for v in versions_rows:
effects_chain = None
if v.effects_chain:
try:
raw = json.loads(v.effects_chain)
effects_chain = [EffectConfig(**e) for e in raw]
except Exception:
pass
versions.append(GenerationVersionResponse(
id=v.id,
generation_id=v.generation_id,
label=v.label,
audio_path=v.audio_path,
effects_chain=effects_chain,
is_default=v.is_default,
created_at=v.created_at,
))
if v.is_default:
active_version_id = v.id
return versions, active_version_id
async def create_generation(
profile_id: str,
text: str,
@@ -29,6 +66,10 @@ async def create_generation(
seed: Optional[int],
db: Session,
instruct: Optional[str] = None,
generation_id: Optional[str] = None,
status: str = "completed",
engine: Optional[str] = "qwen",
model_size: Optional[str] = None,
) -> GenerationResponse:
"""
Create a new generation history entry.
@@ -42,12 +83,16 @@ async def create_generation(
seed: Random seed used (if any)
db: Database session
instruct: Natural language instruction used (if any)
generation_id: Pre-assigned ID (for async generation flow)
status: Generation status (generating, completed, failed)
engine: TTS engine used (qwen, luxtts, chatterbox, chatterbox_turbo)
model_size: Model size variant (1.7B, 0.6B) — only relevant for qwen
Returns:
Created generation entry
"""
db_generation = DBGeneration(
id=str(uuid.uuid4()),
id=generation_id or str(uuid.uuid4()),
profile_id=profile_id,
text=text,
language=language,
@@ -55,6 +100,9 @@ async def create_generation(
duration=duration,
seed=seed,
instruct=instruct,
engine=engine,
model_size=model_size,
status=status,
created_at=datetime.utcnow(),
)
@@ -65,6 +113,32 @@ async def create_generation(
return GenerationResponse.model_validate(db_generation)
async def update_generation_status(
generation_id: str,
status: str,
db: Session,
audio_path: Optional[str] = None,
duration: Optional[float] = None,
error: Optional[str] = None,
) -> Optional[GenerationResponse]:
"""Update the status of a generation (used by async generation flow)."""
generation = db.query(DBGeneration).filter_by(id=generation_id).first()
if not generation:
return None
generation.status = status
if audio_path is not None:
generation.audio_path = audio_path
if duration is not None:
generation.duration = duration
if error is not None:
generation.error = error
db.commit()
db.refresh(generation)
return GenerationResponse.model_validate(generation)
async def get_generation(
generation_id: str,
db: Session,
@@ -133,6 +207,7 @@ async def list_generations(
# Convert to HistoryResponse with profile_name
items = []
for generation, profile_name in results:
versions, active_version_id = _get_versions_for_generation(generation.id, db)
items.append(HistoryResponse(
id=generation.id,
profile_id=generation.profile_id,
@@ -143,7 +218,14 @@ async def list_generations(
duration=generation.duration,
seed=generation.seed,
instruct=generation.instruct,
engine=generation.engine or "qwen",
model_size=generation.model_size,
status=generation.status or "completed",
error=generation.error,
is_favorited=bool(generation.is_favorited),
created_at=generation.created_at,
versions=versions,
active_version_id=active_version_id,
))
return HistoryListResponse(
@@ -169,12 +251,17 @@ async def delete_generation(
generation = db.query(DBGeneration).filter_by(id=generation_id).first()
if not generation:
return False
# Delete audio file
audio_path = Path(generation.audio_path)
if audio_path.exists():
audio_path.unlink()
# Delete all version files and records
from . import versions as versions_mod
versions_mod.delete_versions_for_generation(generation_id, db)
# Delete main audio file (if not already removed by version cleanup)
if generation.audio_path:
audio_path = Path(generation.audio_path)
if audio_path.exists():
audio_path.unlink()
# Delete from database
db.delete(generation)
db.commit()
+1261 -135
View File
File diff suppressed because it is too large Load Diff
+161 -12
View File
@@ -11,7 +11,7 @@ class VoiceProfileCreate(BaseModel):
"""Request model for creating a voice profile."""
name: str = Field(..., min_length=1, max_length=100)
description: Optional[str] = Field(None, max_length=500)
language: str = Field(default="en", pattern="^(zh|en|ja|ko|de|fr|ru|pt|es|it)$")
language: str = Field(default="en", pattern="^(zh|en|ja|ko|de|fr|ru|pt|es|it|he|ar|da|el|fi|hi|ms|nl|no|pl|sv|sw|tr)$")
class VoiceProfileResponse(BaseModel):
@@ -21,6 +21,9 @@ class VoiceProfileResponse(BaseModel):
description: Optional[str]
language: str
avatar_path: Optional[str] = None
effects_chain: Optional[List["EffectConfig"]] = None
generation_count: int = 0
sample_count: int = 0
created_at: datetime
updated_at: datetime
@@ -52,12 +55,16 @@ class ProfileSampleResponse(BaseModel):
class GenerationRequest(BaseModel):
"""Request model for voice generation."""
profile_id: str
text: str = Field(..., min_length=1, max_length=5000)
language: str = Field(default="en", pattern="^(zh|en|ja|ko|de|fr|ru|pt|es|it)$")
text: str = Field(..., min_length=1, max_length=50000)
language: str = Field(default="en", pattern="^(zh|en|ja|ko|de|fr|ru|pt|es|it|he)$")
seed: Optional[int] = Field(None, ge=0)
model_size: Optional[str] = Field(default="1.7B", pattern="^(1\\.7B|0\\.6B)$")
instruct: Optional[str] = Field(None, max_length=500)
engine: Optional[str] = Field(default="qwen", pattern="^(qwen|luxtts)$")
engine: Optional[str] = Field(default="qwen", pattern="^(qwen|luxtts|chatterbox|chatterbox_turbo)$")
max_chunk_chars: int = Field(default=800, ge=100, le=5000, description="Max characters per chunk for long text splitting")
crossfade_ms: int = Field(default=50, ge=0, le=500, description="Crossfade duration in ms between chunks (0 for hard cut)")
normalize: bool = Field(default=True, description="Normalize output audio volume")
effects_chain: Optional[List["EffectConfig"]] = Field(None, description="Effects chain to apply after generation (overrides profile default)")
class GenerationResponse(BaseModel):
@@ -66,11 +73,18 @@ class GenerationResponse(BaseModel):
profile_id: str
text: str
language: str
audio_path: str
duration: float
seed: Optional[int]
instruct: Optional[str]
audio_path: Optional[str] = None
duration: Optional[float] = None
seed: Optional[int] = None
instruct: Optional[str] = None
engine: Optional[str] = "qwen"
model_size: Optional[str] = None
status: str = "completed"
error: Optional[str] = None
is_favorited: bool = False
created_at: datetime
versions: Optional[List["GenerationVersionResponse"]] = None
active_version_id: Optional[str] = None
class Config:
from_attributes = True
@@ -91,11 +105,18 @@ class HistoryResponse(BaseModel):
profile_name: str
text: str
language: str
audio_path: str
duration: float
seed: Optional[int]
instruct: Optional[str]
audio_path: Optional[str] = None
duration: Optional[float] = None
seed: Optional[int] = None
instruct: Optional[str] = None
engine: Optional[str] = "qwen"
model_size: Optional[str] = None
status: str = "completed"
error: Optional[str] = None
is_favorited: bool = False
created_at: datetime
versions: Optional[List["GenerationVersionResponse"]] = None
active_version_id: Optional[str] = None
class Config:
from_attributes = True
@@ -131,10 +152,27 @@ class HealthResponse(BaseModel):
backend_variant: Optional[str] = None # Binary variant (cpu or cuda)
class DirectoryCheck(BaseModel):
"""Health status for a single directory."""
path: str
exists: bool
writable: bool
error: Optional[str] = None
class FilesystemHealthResponse(BaseModel):
"""Response model for filesystem health check."""
healthy: bool
disk_free_mb: Optional[float] = None
disk_total_mb: Optional[float] = None
directories: List[DirectoryCheck]
class ModelStatus(BaseModel):
"""Response model for model status."""
model_name: str
display_name: str
hf_repo_id: Optional[str] = None # HuggingFace repository ID
downloaded: bool
downloading: bool = False # True if download is in progress
size_mb: Optional[float] = None
@@ -151,12 +189,21 @@ class ModelDownloadRequest(BaseModel):
model_name: str
class ModelMigrateRequest(BaseModel):
"""Request model for migrating models to a new directory."""
destination: str
class ActiveDownloadTask(BaseModel):
"""Response model for active download task."""
model_name: str
status: str
started_at: datetime
error: Optional[str] = None
progress: Optional[float] = None # 0-100 percentage
current: Optional[int] = None # bytes downloaded
total: Optional[int] = None # total bytes
filename: Optional[str] = None # current file being downloaded
class ActiveGenerationTask(BaseModel):
@@ -231,6 +278,7 @@ class StoryItemDetail(BaseModel):
id: str
story_id: str
generation_id: str
version_id: Optional[str] = None
start_time_ms: int
track: int = 0
trim_start_ms: int = 0
@@ -246,6 +294,9 @@ class StoryItemDetail(BaseModel):
seed: Optional[int]
instruct: Optional[str]
generation_created_at: datetime
# Versions available for this generation
versions: Optional[List["GenerationVersionResponse"]] = None
active_version_id: Optional[str] = None
class Config:
from_attributes = True
@@ -302,3 +353,101 @@ class StoryItemTrim(BaseModel):
class StoryItemSplit(BaseModel):
"""Request model for splitting a story item."""
split_time_ms: int = Field(..., ge=0) # Time within the clip to split at (relative to clip start)
class StoryItemVersionUpdate(BaseModel):
"""Request model for setting a story item's pinned version."""
version_id: Optional[str] = None # null = use generation default
# ============================================
# Effects & Versions
# ============================================
class EffectConfig(BaseModel):
"""A single effect in an effects chain."""
type: str
enabled: bool = True
params: dict = Field(default_factory=dict)
class EffectsChain(BaseModel):
"""An ordered list of effects to apply."""
effects: List[EffectConfig] = Field(default_factory=list)
class EffectPresetCreate(BaseModel):
"""Request model for creating an effect preset."""
name: str = Field(..., min_length=1, max_length=100)
description: Optional[str] = Field(None, max_length=500)
effects_chain: List[EffectConfig]
class EffectPresetUpdate(BaseModel):
"""Request model for updating an effect preset."""
name: Optional[str] = Field(None, min_length=1, max_length=100)
description: Optional[str] = None
effects_chain: Optional[List[EffectConfig]] = None
class EffectPresetResponse(BaseModel):
"""Response model for effect preset."""
id: str
name: str
description: Optional[str] = None
effects_chain: List[EffectConfig]
is_builtin: bool = False
created_at: datetime
class Config:
from_attributes = True
class GenerationVersionResponse(BaseModel):
"""Response model for a generation version."""
id: str
generation_id: str
label: str
audio_path: str
effects_chain: Optional[List[EffectConfig]] = None
source_version_id: Optional[str] = None
is_default: bool
created_at: datetime
class Config:
from_attributes = True
class ApplyEffectsRequest(BaseModel):
"""Request to apply effects to an existing generation."""
effects_chain: List[EffectConfig]
source_version_id: Optional[str] = Field(None, description="Version to use as source audio (defaults to clean/original)")
label: Optional[str] = Field(None, max_length=100, description="Label for this version (auto-generated if omitted)")
set_as_default: bool = Field(default=True, description="Set this version as the default")
class ProfileEffectsUpdate(BaseModel):
"""Request to update the default effects chain on a profile."""
effects_chain: Optional[List[EffectConfig]] = Field(None, description="Effects chain (null to remove)")
class AvailableEffectParam(BaseModel):
"""Description of a single effect parameter."""
default: float
min: float
max: float
step: float
description: str
class AvailableEffect(BaseModel):
"""Description of an available effect type."""
type: str
label: str
description: str
params: dict # param_name -> AvailableEffectParam
class AvailableEffectsResponse(BaseModel):
"""Response listing all available effect types."""
effects: List[AvailableEffect]
+92 -18
View File
@@ -8,7 +8,7 @@ import uuid
import shutil
from pathlib import Path
from sqlalchemy.orm import Session
from sqlalchemy import select
from sqlalchemy import func, select
from .models import (
VoiceProfileCreate,
@@ -19,12 +19,43 @@ from .models import (
from .database import (
VoiceProfile as DBVoiceProfile,
ProfileSample as DBProfileSample,
Generation as DBGeneration,
)
from .models import EffectConfig
from .utils.audio import validate_reference_audio, load_audio, save_audio
from .utils.images import validate_image, process_avatar
from .utils.cache import _get_cache_dir, clear_profile_cache
from .tts import get_tts_model
from . import config
import json as _json
def _profile_to_response(
profile: DBVoiceProfile,
generation_count: int = 0,
sample_count: int = 0,
) -> VoiceProfileResponse:
"""Convert a DB profile to a VoiceProfileResponse, deserializing effects_chain."""
effects_chain = None
if profile.effects_chain:
try:
raw = _json.loads(profile.effects_chain)
effects_chain = [EffectConfig(**e) for e in raw]
except Exception as e:
import logging
logging.warning(f"Failed to parse effects_chain for profile {profile.id}: {e}")
return VoiceProfileResponse(
id=profile.id,
name=profile.name,
description=profile.description,
language=profile.language,
avatar_path=profile.avatar_path,
effects_chain=effects_chain,
generation_count=generation_count,
sample_count=sample_count,
created_at=profile.created_at,
updated_at=profile.updated_at,
)
def _get_profiles_dir() -> Path:
@@ -38,14 +69,22 @@ async def create_profile(
) -> VoiceProfileResponse:
"""
Create a new voice profile.
Args:
data: Profile creation data
db: Database session
Returns:
Created profile
Raises:
ValueError: If a profile with the same name already exists
"""
# Check if profile name already exists
existing_profile = db.query(DBVoiceProfile).filter_by(name=data.name).first()
if existing_profile:
raise ValueError(f"A profile with the name '{data.name}' already exists. Please choose a different name.")
# Create profile in database
db_profile = DBVoiceProfile(
id=str(uuid.uuid4()),
@@ -55,16 +94,16 @@ async def create_profile(
created_at=datetime.utcnow(),
updated_at=datetime.utcnow(),
)
db.add(db_profile)
db.commit()
db.refresh(db_profile)
# Create profile directory
profile_dir = _get_profiles_dir() / db_profile.id
profile_dir.mkdir(parents=True, exist_ok=True)
return VoiceProfileResponse.model_validate(db_profile)
return _profile_to_response(db_profile)
async def add_profile_sample(
@@ -146,7 +185,7 @@ async def get_profile(
if not profile:
return None
return VoiceProfileResponse.model_validate(profile)
return _profile_to_response(profile)
async def get_profile_samples(
@@ -169,7 +208,7 @@ async def get_profile_samples(
async def list_profiles(db: Session) -> List[VoiceProfileResponse]:
"""
List all voice profiles.
List all voice profiles with generation and sample counts.
Args:
db: Database session
@@ -180,8 +219,34 @@ async def list_profiles(db: Session) -> List[VoiceProfileResponse]:
profiles = db.query(DBVoiceProfile).order_by(
DBVoiceProfile.created_at.desc()
).all()
return [VoiceProfileResponse.model_validate(p) for p in profiles]
if not profiles:
return []
# Batch-fetch generation counts
gen_counts_rows = (
db.query(DBGeneration.profile_id, func.count(DBGeneration.id))
.group_by(DBGeneration.profile_id)
.all()
)
gen_counts = {row[0]: row[1] for row in gen_counts_rows}
# Batch-fetch sample counts
sample_counts_rows = (
db.query(DBProfileSample.profile_id, func.count(DBProfileSample.id))
.group_by(DBProfileSample.profile_id)
.all()
)
sample_counts = {row[0]: row[1] for row in sample_counts_rows}
return [
_profile_to_response(
p,
generation_count=gen_counts.get(p.id, 0),
sample_count=sample_counts.get(p.id, 0),
)
for p in profiles
]
async def update_profile(
@@ -191,29 +256,38 @@ async def update_profile(
) -> Optional[VoiceProfileResponse]:
"""
Update a voice profile.
Args:
profile_id: Profile ID
data: Updated profile data
db: Database session
Returns:
Updated profile or None if not found
Raises:
ValueError: If a profile with the same name already exists (different profile)
"""
profile = db.query(DBVoiceProfile).filter_by(id=profile_id).first()
if not profile:
return None
# Check if the new name conflicts with another profile
if profile.name != data.name:
existing_profile = db.query(DBVoiceProfile).filter_by(name=data.name).first()
if existing_profile:
raise ValueError(f"A profile with the name '{data.name}' already exists. Please choose a different name.")
# Update fields
profile.name = data.name
profile.description = data.description
profile.language = data.language
profile.updated_at = datetime.utcnow()
db.commit()
db.refresh(profile)
return VoiceProfileResponse.model_validate(profile)
return _profile_to_response(profile)
async def delete_profile(
@@ -455,7 +529,7 @@ async def upload_avatar(
db.commit()
db.refresh(profile)
return VoiceProfileResponse.model_validate(profile)
return _profile_to_response(profile)
async def delete_avatar(
+13
View File
@@ -21,11 +21,24 @@ qwen-tts>=0.0.5
linacodec @ git+https://github.com/ysharma3501/LinaCodec.git
Zipvoice @ git+https://github.com/ysharma3501/LuxTTS.git
# Chatterbox TTS sub-dependencies (chatterbox-tts itself is installed
# --no-deps in the setup script because it pins numpy<1.26 / torch==2.6
# which are incompatible with Python 3.12+)
conformer>=0.3.2
diffusers>=0.29.0
omegaconf
pykakasi
resemble-perth>=1.0.1
s3tokenizer
spacy-pkuseg
pyloudnorm
# Audio processing
librosa>=0.10.0
soundfile>=0.12.0
numpy>=1.24.0
numba>=0.60.0,<0.61.0
pedalboard>=0.9.0
# HTTP client (for CUDA backend download)
httpx>=0.27.0
+125 -183
View File
@@ -20,12 +20,55 @@ from .models import (
StoryItemMove,
StoryItemTrim,
StoryItemSplit,
StoryItemVersionUpdate,
)
from .database import Story as DBStory, StoryItem as DBStoryItem, Generation as DBGeneration, VoiceProfile as DBVoiceProfile
from .history import _get_versions_for_generation
from .utils.audio import load_audio, save_audio
import numpy as np
def _build_item_detail(
item: DBStoryItem,
generation: DBGeneration,
profile_name: str,
db: Session,
) -> StoryItemDetail:
"""Build a StoryItemDetail with version info from a story item and its generation."""
versions, active_version_id = _get_versions_for_generation(generation.id, db)
# Resolve the audio path: if version_id is set, use that version's audio
audio_path = generation.audio_path
if item.version_id and versions:
for v in versions:
if v.id == item.version_id:
audio_path = v.audio_path
break
return StoryItemDetail(
id=item.id,
story_id=item.story_id,
generation_id=item.generation_id,
version_id=getattr(item, 'version_id', None),
start_time_ms=item.start_time_ms,
track=item.track,
trim_start_ms=getattr(item, 'trim_start_ms', 0),
trim_end_ms=getattr(item, 'trim_end_ms', 0),
created_at=item.created_at,
profile_id=generation.profile_id,
profile_name=profile_name,
text=generation.text,
language=generation.language,
audio_path=audio_path,
duration=generation.duration,
seed=generation.seed,
instruct=generation.instruct,
generation_created_at=generation.created_at,
versions=versions,
active_version_id=active_version_id,
)
async def create_story(
data: StoryCreate,
db: Session,
@@ -125,26 +168,7 @@ async def get_story(
# Build item details
item_details = []
for item, generation, profile_name in items:
item_detail = StoryItemDetail(
id=item.id,
story_id=item.story_id,
generation_id=item.generation_id,
start_time_ms=item.start_time_ms,
track=item.track,
trim_start_ms=getattr(item, 'trim_start_ms', 0),
trim_end_ms=getattr(item, 'trim_end_ms', 0),
created_at=item.created_at,
profile_id=generation.profile_id,
profile_name=profile_name,
text=generation.text,
language=generation.language,
audio_path=generation.audio_path,
duration=generation.duration,
seed=generation.seed,
instruct=generation.instruct,
generation_created_at=generation.created_at,
)
item_details.append(item_detail)
item_details.append(_build_item_detail(item, generation, profile_name, db))
response = StoryDetailResponse.model_validate(story)
response.items = item_details
@@ -250,31 +274,16 @@ async def add_item_to_story(
if existing:
# Return existing item
profile = db.query(DBVoiceProfile).filter_by(id=generation.profile_id).first()
return StoryItemDetail(
id=existing.id,
story_id=existing.story_id,
generation_id=existing.generation_id,
start_time_ms=existing.start_time_ms,
track=existing.track,
trim_start_ms=getattr(existing, 'trim_start_ms', 0),
trim_end_ms=getattr(existing, 'trim_end_ms', 0),
created_at=existing.created_at,
profile_id=generation.profile_id,
profile_name=profile.name if profile else "Unknown",
text=generation.text,
language=generation.language,
audio_path=generation.audio_path,
duration=generation.duration,
seed=generation.seed,
instruct=generation.instruct,
generation_created_at=generation.created_at,
)
return _build_item_detail(existing, generation, profile.name if profile else "Unknown", db)
# Get track from data or default to 0
track = data.track if data.track is not None else 0
# Calculate start_time_ms if not provided
if data.start_time_ms is not None:
start_time_ms = data.start_time_ms
else:
# Find the maximum end time (start_time_ms + duration_ms) of existing items
# Find the maximum end time on the target track only
existing_items = db.query(
DBStoryItem,
DBGeneration
@@ -282,11 +291,11 @@ async def add_item_to_story(
DBGeneration,
DBStoryItem.generation_id == DBGeneration.id
).filter(
DBStoryItem.story_id == story_id
DBStoryItem.story_id == story_id,
DBStoryItem.track == track,
).all()
if not existing_items:
# First item starts at 0
start_time_ms = 0
else:
max_end_time_ms = 0
@@ -297,9 +306,6 @@ async def add_item_to_story(
# Add 200ms gap after the last item
start_time_ms = max_end_time_ms + 200
# Get track from data or default to 0
track = data.track if data.track is not None else 0
# Create item
item = DBStoryItem(
id=str(uuid.uuid4()),
@@ -321,25 +327,7 @@ async def add_item_to_story(
# Get profile name
profile = db.query(DBVoiceProfile).filter_by(id=generation.profile_id).first()
return StoryItemDetail(
id=item.id,
story_id=item.story_id,
generation_id=item.generation_id,
start_time_ms=item.start_time_ms,
track=item.track,
trim_start_ms=getattr(item, 'trim_start_ms', 0),
trim_end_ms=getattr(item, 'trim_end_ms', 0),
created_at=item.created_at,
profile_id=generation.profile_id,
profile_name=profile.name if profile else "Unknown",
text=generation.text,
language=generation.language,
audio_path=generation.audio_path,
duration=generation.duration,
seed=generation.seed,
instruct=generation.instruct,
generation_created_at=generation.created_at,
)
return _build_item_detail(item, generation, profile.name if profile else "Unknown", db)
async def move_story_item(
@@ -388,25 +376,7 @@ async def move_story_item(
# Get profile name
profile = db.query(DBVoiceProfile).filter_by(id=generation.profile_id).first()
return StoryItemDetail(
id=item.id,
story_id=item.story_id,
generation_id=item.generation_id,
start_time_ms=item.start_time_ms,
track=item.track,
trim_start_ms=getattr(item, 'trim_start_ms', 0),
trim_end_ms=getattr(item, 'trim_end_ms', 0),
created_at=item.created_at,
profile_id=generation.profile_id,
profile_name=profile.name if profile else "Unknown",
text=generation.text,
language=generation.language,
audio_path=generation.audio_path,
duration=generation.duration,
seed=generation.seed,
instruct=generation.instruct,
generation_created_at=generation.created_at,
)
return _build_item_detail(item, generation, profile.name if profile else "Unknown", db)
async def remove_item_from_story(
@@ -495,25 +465,7 @@ async def trim_story_item(
# Get profile name
profile = db.query(DBVoiceProfile).filter_by(id=generation.profile_id).first()
return StoryItemDetail(
id=item.id,
story_id=item.story_id,
generation_id=item.generation_id,
start_time_ms=item.start_time_ms,
track=item.track,
trim_start_ms=item.trim_start_ms,
trim_end_ms=item.trim_end_ms,
created_at=item.created_at,
profile_id=generation.profile_id,
profile_name=profile.name if profile else "Unknown",
text=generation.text,
language=generation.language,
audio_path=generation.audio_path,
duration=generation.duration,
seed=generation.seed,
instruct=generation.instruct,
generation_created_at=generation.created_at,
)
return _build_item_detail(item, generation, profile.name if profile else "Unknown", db)
async def split_story_item(
@@ -568,6 +520,7 @@ async def split_story_item(
id=str(uuid.uuid4()),
story_id=story_id,
generation_id=item.generation_id, # Same generation, different trim
version_id=getattr(item, 'version_id', None), # Preserve pinned version
start_time_ms=item.start_time_ms + data.split_time_ms,
track=item.track,
trim_start_ms=absolute_split_ms,
@@ -590,48 +543,10 @@ async def split_story_item(
profile = db.query(DBVoiceProfile).filter_by(id=generation.profile_id).first()
profile_name = profile.name if profile else "Unknown"
# Build response items
original_item_detail = StoryItemDetail(
id=item.id,
story_id=item.story_id,
generation_id=item.generation_id,
start_time_ms=item.start_time_ms,
track=item.track,
trim_start_ms=item.trim_start_ms,
trim_end_ms=item.trim_end_ms,
created_at=item.created_at,
profile_id=generation.profile_id,
profile_name=profile_name,
text=generation.text,
language=generation.language,
audio_path=generation.audio_path,
duration=generation.duration,
seed=generation.seed,
instruct=generation.instruct,
generation_created_at=generation.created_at,
)
new_item_detail = StoryItemDetail(
id=new_item.id,
story_id=new_item.story_id,
generation_id=new_item.generation_id,
start_time_ms=new_item.start_time_ms,
track=new_item.track,
trim_start_ms=new_item.trim_start_ms,
trim_end_ms=new_item.trim_end_ms,
created_at=new_item.created_at,
profile_id=generation.profile_id,
profile_name=profile_name,
text=generation.text,
language=generation.language,
audio_path=generation.audio_path,
duration=generation.duration,
seed=generation.seed,
instruct=generation.instruct,
generation_created_at=generation.created_at,
)
return [original_item_detail, new_item_detail]
return [
_build_item_detail(item, generation, profile_name, db),
_build_item_detail(new_item, generation, profile_name, db),
]
async def duplicate_story_item(
@@ -674,6 +589,7 @@ async def duplicate_story_item(
id=str(uuid.uuid4()),
story_id=story_id,
generation_id=original_item.generation_id, # Same generation as original
version_id=getattr(original_item, 'version_id', None), # Preserve pinned version
start_time_ms=original_item.start_time_ms + effective_duration_ms + 200, # 200ms gap
track=original_item.track,
trim_start_ms=current_trim_start,
@@ -694,25 +610,7 @@ async def duplicate_story_item(
# Get profile name
profile = db.query(DBVoiceProfile).filter_by(id=generation.profile_id).first()
return StoryItemDetail(
id=new_item.id,
story_id=new_item.story_id,
generation_id=new_item.generation_id,
start_time_ms=new_item.start_time_ms,
track=new_item.track,
trim_start_ms=new_item.trim_start_ms,
trim_end_ms=new_item.trim_end_ms,
created_at=new_item.created_at,
profile_id=generation.profile_id,
profile_name=profile.name if profile else "Unknown",
text=generation.text,
language=generation.language,
audio_path=generation.audio_path,
duration=generation.duration,
seed=generation.seed,
instruct=generation.instruct,
generation_created_at=generation.created_at,
)
return _build_item_detail(new_item, generation, profile.name if profile else "Unknown", db)
async def update_story_item_times(
@@ -813,25 +711,7 @@ async def reorder_story_items(
current_time_ms += duration_ms + gap_ms
# Build the response item
updated_items.append(StoryItemDetail(
id=item.id,
story_id=item.story_id,
generation_id=item.generation_id,
start_time_ms=item.start_time_ms,
track=item.track,
trim_start_ms=getattr(item, 'trim_start_ms', 0),
trim_end_ms=getattr(item, 'trim_end_ms', 0),
created_at=item.created_at,
profile_id=generation.profile_id,
profile_name=profile_name,
text=generation.text,
language=generation.language,
audio_path=generation.audio_path,
duration=generation.duration,
seed=generation.seed,
instruct=generation.instruct,
generation_created_at=generation.created_at,
))
updated_items.append(_build_item_detail(item, generation, profile_name, db))
# Update story updated_at
story.updated_at = datetime.utcnow()
@@ -840,6 +720,60 @@ async def reorder_story_items(
return updated_items
async def set_story_item_version(
story_id: str,
item_id: str,
data: StoryItemVersionUpdate,
db: Session,
) -> Optional[StoryItemDetail]:
"""
Pin a story item to a specific generation version.
Args:
story_id: Story ID
item_id: Story item ID
data: Version update data (version_id or null for default)
db: Database session
Returns:
Updated item detail or None if not found
"""
item = db.query(DBStoryItem).filter_by(
id=item_id,
story_id=story_id,
).first()
if not item:
return None
generation = db.query(DBGeneration).filter_by(id=item.generation_id).first()
if not generation:
return None
# Validate version_id belongs to this generation if provided
if data.version_id:
from .database import GenerationVersion as DBGenerationVersion
version = db.query(DBGenerationVersion).filter_by(
id=data.version_id,
generation_id=item.generation_id,
).first()
if not version:
return None
item.version_id = data.version_id
# Update story updated_at
story = db.query(DBStory).filter_by(id=story_id).first()
if story:
story.updated_at = datetime.utcnow()
db.commit()
db.refresh(item)
profile = db.query(DBVoiceProfile).filter_by(id=generation.profile_id).first()
return _build_item_detail(item, generation, profile.name if profile else "Unknown", db)
async def export_story_audio(
story_id: str,
db: Session,
@@ -877,7 +811,15 @@ async def export_story_audio(
sample_rate = 24000 # Default sample rate
for item, generation in items:
audio_path = Path(generation.audio_path)
# Resolve audio path: use pinned version if set, otherwise generation default
resolved_audio_path = generation.audio_path
if getattr(item, 'version_id', None):
from .database import GenerationVersion as DBGenerationVersion
version = db.query(DBGenerationVersion).filter_by(id=item.version_id).first()
if version:
resolved_audio_path = version.audio_path
audio_path = Path(resolved_audio_path)
if not audio_path.exists():
continue
+162
View File
@@ -0,0 +1,162 @@
"""
Tests for CORS origin restrictions.
Validates that the CORS middleware only allows known local origins
and respects the VOICEBOX_CORS_ORIGINS environment variable.
Uses a minimal FastAPI app that mirrors the exact CORS configuration
from backend/main.py, so tests run without heavy ML dependencies.
Usage:
pip install httpx pytest fastapi starlette
python -m pytest backend/tests/test_cors.py -v
"""
import os
import pytest
from unittest.mock import patch
from fastapi import FastAPI
from fastapi.middleware.cors import CORSMiddleware
from starlette.testclient import TestClient
def _build_app(env_origins: str = "") -> FastAPI:
"""
Build a minimal FastAPI app with the same CORS logic as backend/main.py.
This mirrors the exact code in main.py so the test validates the real
configuration without needing torch/numpy/transformers installed.
"""
app = FastAPI()
_default_origins = [
"http://localhost:5173",
"http://127.0.0.1:5173",
"http://localhost:17493",
"http://127.0.0.1:17493",
"tauri://localhost",
"https://tauri.localhost",
]
_cors_origins = _default_origins + [o.strip() for o in env_origins.split(",") if o.strip()]
app.add_middleware(
CORSMiddleware,
allow_origins=_cors_origins,
allow_credentials=True,
allow_methods=["*"],
allow_headers=["*"],
)
@app.get("/health")
async def health():
return {"status": "ok"}
return app
@pytest.fixture()
def client():
return TestClient(_build_app())
@pytest.fixture()
def client_with_custom_origins():
return TestClient(_build_app("https://custom.example.com,https://other.example.com"))
def _get_with_origin(client: TestClient, origin: str) -> dict:
"""Send a GET with Origin header, return response headers."""
response = client.get("/health", headers={"Origin": origin})
return dict(response.headers)
def _preflight(client: TestClient, origin: str) -> dict:
"""Send CORS preflight OPTIONS request, return response headers."""
response = client.options(
"/health",
headers={
"Origin": origin,
"Access-Control-Request-Method": "GET",
},
)
return dict(response.headers)
class TestCORSDefaultOrigins:
"""CORS should allow known local origins and block everything else."""
@pytest.mark.parametrize("origin", [
"http://localhost:5173",
"http://127.0.0.1:5173",
"http://localhost:17493",
"http://127.0.0.1:17493",
"tauri://localhost",
"https://tauri.localhost",
])
def test_allowed_origins(self, client, origin):
headers = _get_with_origin(client, origin)
assert headers.get("access-control-allow-origin") == origin
@pytest.mark.parametrize("origin", [
"http://evil.com",
"http://localhost:9999",
"https://attacker.example.com",
"null",
])
def test_blocked_origins(self, client, origin):
headers = _get_with_origin(client, origin)
assert "access-control-allow-origin" not in headers
def test_preflight_allowed(self, client):
headers = _preflight(client, "http://localhost:5173")
assert headers.get("access-control-allow-origin") == "http://localhost:5173"
def test_preflight_blocked(self, client):
headers = _preflight(client, "http://evil.com")
assert "access-control-allow-origin" not in headers
def test_credentials_header_present(self, client):
headers = _get_with_origin(client, "http://localhost:5173")
assert headers.get("access-control-allow-credentials") == "true"
class TestCORSCustomOrigins:
"""VOICEBOX_CORS_ORIGINS env var should extend the allowlist."""
def test_custom_origin_allowed(self, client_with_custom_origins):
headers = _get_with_origin(client_with_custom_origins, "https://custom.example.com")
assert headers.get("access-control-allow-origin") == "https://custom.example.com"
def test_other_custom_origin_allowed(self, client_with_custom_origins):
headers = _get_with_origin(client_with_custom_origins, "https://other.example.com")
assert headers.get("access-control-allow-origin") == "https://other.example.com"
def test_default_origins_still_work(self, client_with_custom_origins):
headers = _get_with_origin(client_with_custom_origins, "http://localhost:5173")
assert headers.get("access-control-allow-origin") == "http://localhost:5173"
def test_unlisted_origin_still_blocked(self, client_with_custom_origins):
headers = _get_with_origin(client_with_custom_origins, "http://evil.com")
assert "access-control-allow-origin" not in headers
class TestCORSEnvVarParsing:
"""Edge cases for VOICEBOX_CORS_ORIGINS parsing."""
def test_empty_env_var(self):
app = _build_app("")
client = TestClient(app)
headers = _get_with_origin(client, "http://evil.com")
assert "access-control-allow-origin" not in headers
def test_whitespace_trimmed(self):
app = _build_app(" https://spaced.example.com ")
client = TestClient(app)
headers = _get_with_origin(client, "https://spaced.example.com")
assert headers.get("access-control-allow-origin") == "https://spaced.example.com"
def test_trailing_comma_ignored(self):
app = _build_app("https://one.example.com,")
client = TestClient(app)
headers = _get_with_origin(client, "https://one.example.com")
assert headers.get("access-control-allow-origin") == "https://one.example.com"
@@ -0,0 +1,217 @@
"""
Tests for profile duplicate name validation.
This test suite verifies that the application correctly handles
duplicate profile names and provides user-friendly error messages.
"""
import pytest
import tempfile
import shutil
from pathlib import Path
from sqlalchemy import create_engine
from sqlalchemy.orm import sessionmaker
# Add parent directory to path to import backend modules
import sys
sys.path.insert(0, str(Path(__file__).parent.parent))
from database import Base, VoiceProfile as DBVoiceProfile
from models import VoiceProfileCreate
from profiles import create_profile, update_profile
@pytest.fixture
def test_db():
"""Create a temporary test database."""
# Create temporary directory for test database
temp_dir = tempfile.mkdtemp()
db_path = Path(temp_dir) / "test.db"
# Create engine and session
engine = create_engine(f"sqlite:///{db_path}")
Base.metadata.create_all(bind=engine)
SessionLocal = sessionmaker(autocommit=False, autoflush=False, bind=engine)
db = SessionLocal()
yield db
# Cleanup
db.close()
shutil.rmtree(temp_dir)
@pytest.fixture
def mock_profiles_dir(monkeypatch, tmp_path):
"""Mock the profiles directory to use a temporary path."""
import profiles
monkeypatch.setattr(profiles, '_get_profiles_dir', lambda: tmp_path)
return tmp_path
@pytest.mark.asyncio
async def test_create_profile_duplicate_name_raises_error(test_db, mock_profiles_dir):
"""Test that creating a profile with a duplicate name raises a ValueError."""
# Create first profile
profile_data_1 = VoiceProfileCreate(
name="Test Profile",
description="First profile",
language="en"
)
profile_1 = await create_profile(profile_data_1, test_db)
assert profile_1.name == "Test Profile"
# Try to create second profile with same name
profile_data_2 = VoiceProfileCreate(
name="Test Profile",
description="Second profile",
language="en"
)
with pytest.raises(ValueError) as exc_info:
await create_profile(profile_data_2, test_db)
# Verify error message is user-friendly
assert "already exists" in str(exc_info.value)
assert "Test Profile" in str(exc_info.value)
assert "choose a different name" in str(exc_info.value).lower()
@pytest.mark.asyncio
async def test_create_profile_different_names_succeeds(test_db, mock_profiles_dir):
"""Test that creating profiles with different names succeeds."""
# Create first profile
profile_data_1 = VoiceProfileCreate(
name="Profile One",
description="First profile",
language="en"
)
profile_1 = await create_profile(profile_data_1, test_db)
assert profile_1.name == "Profile One"
# Create second profile with different name
profile_data_2 = VoiceProfileCreate(
name="Profile Two",
description="Second profile",
language="en"
)
profile_2 = await create_profile(profile_data_2, test_db)
assert profile_2.name == "Profile Two"
# Verify both profiles exist
assert profile_1.id != profile_2.id
@pytest.mark.asyncio
async def test_update_profile_to_duplicate_name_raises_error(test_db, mock_profiles_dir):
"""Test that updating a profile to a duplicate name raises a ValueError."""
# Create two profiles with different names
profile_data_1 = VoiceProfileCreate(
name="Profile A",
description="First profile",
language="en"
)
profile_1 = await create_profile(profile_data_1, test_db)
profile_data_2 = VoiceProfileCreate(
name="Profile B",
description="Second profile",
language="en"
)
profile_2 = await create_profile(profile_data_2, test_db)
# Try to update profile_2 to use profile_1's name
update_data = VoiceProfileCreate(
name="Profile A", # Duplicate name
description="Updated description",
language="en"
)
with pytest.raises(ValueError) as exc_info:
await update_profile(profile_2.id, update_data, test_db)
# Verify error message is user-friendly
assert "already exists" in str(exc_info.value)
assert "Profile A" in str(exc_info.value)
@pytest.mark.asyncio
async def test_update_profile_keep_same_name_succeeds(test_db, mock_profiles_dir):
"""Test that updating a profile while keeping the same name succeeds."""
# Create profile
profile_data = VoiceProfileCreate(
name="My Profile",
description="Original description",
language="en"
)
profile = await create_profile(profile_data, test_db)
# Update profile with same name but different description
update_data = VoiceProfileCreate(
name="My Profile", # Same name
description="Updated description",
language="en"
)
updated_profile = await update_profile(profile.id, update_data, test_db)
# Verify update succeeded
assert updated_profile is not None
assert updated_profile.id == profile.id
assert updated_profile.name == "My Profile"
assert updated_profile.description == "Updated description"
@pytest.mark.asyncio
async def test_update_profile_to_new_unique_name_succeeds(test_db, mock_profiles_dir):
"""Test that updating a profile to a new unique name succeeds."""
# Create profile
profile_data = VoiceProfileCreate(
name="Original Name",
description="Profile description",
language="en"
)
profile = await create_profile(profile_data, test_db)
# Update profile with new unique name
update_data = VoiceProfileCreate(
name="New Unique Name",
description="Updated description",
language="en"
)
updated_profile = await update_profile(profile.id, update_data, test_db)
# Verify update succeeded
assert updated_profile is not None
assert updated_profile.id == profile.id
assert updated_profile.name == "New Unique Name"
@pytest.mark.asyncio
async def test_case_sensitive_names_allowed(test_db, mock_profiles_dir):
"""Test that profile names are case-sensitive (e.g., 'Test' and 'test' are different)."""
# Create profile with lowercase name
profile_data_1 = VoiceProfileCreate(
name="test profile",
description="Lowercase",
language="en"
)
profile_1 = await create_profile(profile_data_1, test_db)
# Create profile with different case
profile_data_2 = VoiceProfileCreate(
name="Test Profile",
description="Title case",
language="en"
)
profile_2 = await create_profile(profile_data_2, test_db)
# Both should succeed since SQLite unique constraint is case-sensitive by default
assert profile_1.name == "test profile"
assert profile_2.name == "Test Profile"
assert profile_1.id != profile_2.id
+122 -3
View File
@@ -70,14 +70,133 @@ def save_audio(
sample_rate: int = 24000,
) -> None:
"""
Save audio file.
Save audio file with atomic write and error handling.
Writes to a temporary file first, then atomically renames to the
target path. This prevents corrupted/partial WAV files if the
process is interrupted mid-write.
Args:
audio: Audio array
path: Output path
sample_rate: Sample rate
Raises:
OSError: If file cannot be written
"""
sf.write(path, audio, sample_rate)
from pathlib import Path
import os
temp_path = f"{path}.tmp"
try:
# Ensure parent directory exists
Path(path).parent.mkdir(parents=True, exist_ok=True)
# Write to temporary file first (explicit format since .tmp
# extension is not recognised by soundfile)
sf.write(temp_path, audio, sample_rate, format='WAV')
# Atomic rename to final path
os.replace(temp_path, path)
except Exception as e:
# Clean up temp file on failure
try:
if Path(temp_path).exists():
Path(temp_path).unlink()
except Exception:
pass # Best effort cleanup
raise OSError(f"Failed to save audio to {path}: {e}") from e
def trim_tts_output(
audio: np.ndarray,
sample_rate: int = 24000,
frame_ms: int = 20,
silence_threshold_db: float = -40.0,
min_silence_ms: int = 200,
max_internal_silence_ms: int = 1000,
fade_ms: int = 30,
) -> np.ndarray:
"""
Trim trailing silence and post-silence hallucination from TTS output.
Chatterbox sometimes produces ``[speech][silence][hallucinated noise]``.
This detects internal silence gaps longer than *max_internal_silence_ms*
and cuts the audio at that boundary, then trims trailing silence and
applies a short cosine fade-out.
Args:
audio: Input audio array (mono float32)
sample_rate: Sample rate in Hz
frame_ms: Frame size for RMS energy calculation
silence_threshold_db: dB threshold below which a frame is silence
min_silence_ms: Minimum trailing silence to keep
max_internal_silence_ms: Cut after any silence gap longer than this
fade_ms: Cosine fade-out duration in ms
Returns:
Trimmed audio array
"""
frame_len = int(sample_rate * frame_ms / 1000)
if frame_len == 0 or len(audio) < frame_len:
return audio
n_frames = len(audio) // frame_len
threshold_linear = 10 ** (silence_threshold_db / 20)
# Compute per-frame RMS
rms = np.array(
[
np.sqrt(np.mean(audio[i * frame_len : (i + 1) * frame_len] ** 2))
for i in range(n_frames)
]
)
is_speech = rms >= threshold_linear
# Find first speech frame
first_speech = 0
for i, s in enumerate(is_speech):
if s:
first_speech = max(0, i - 1) # keep 1 frame padding
break
# Walk forward from first speech; cut at long internal silence gaps
max_silence_frames = int(max_internal_silence_ms / frame_ms)
consecutive_silence = 0
cut_frame = n_frames
for i in range(first_speech, n_frames):
if is_speech[i]:
consecutive_silence = 0
else:
consecutive_silence += 1
if consecutive_silence >= max_silence_frames:
cut_frame = i - consecutive_silence + 1
break
# Trim trailing silence from the cut point
min_silence_frames = int(min_silence_ms / frame_ms)
end_frame = cut_frame
while end_frame > first_speech and not is_speech[end_frame - 1]:
end_frame -= 1
# Keep a short tail
end_frame = min(end_frame + min_silence_frames, cut_frame)
# Convert frames back to samples
start_sample = first_speech * frame_len
end_sample = min(end_frame * frame_len, len(audio))
trimmed = audio[start_sample:end_sample].copy()
# Cosine fade-out
fade_samples = int(sample_rate * fade_ms / 1000)
if fade_samples > 0 and len(trimmed) > fade_samples:
fade = np.cos(np.linspace(0, np.pi / 2, fade_samples)) ** 2
trimmed[-fade_samples:] *= fade
return trimmed
def validate_reference_audio(
+302
View File
@@ -0,0 +1,302 @@
"""
Chunked TTS generation utilities.
Splits long text into sentence-boundary chunks, generates audio per-chunk
via any TTSBackend, and concatenates with crossfade. All logic is
engine-agnostic — it wraps the standard ``TTSBackend.generate()`` interface.
Short text (≤ max_chunk_chars) uses the single-shot fast path with zero
overhead.
"""
import logging
import re
from typing import List, Tuple
import numpy as np
logger = logging.getLogger("voicebox.chunked-tts")
# Default chunk size in characters. Can be overridden per-request via
# the ``max_chunk_chars`` field on GenerationRequest.
DEFAULT_MAX_CHUNK_CHARS = 800
# Common abbreviations that should NOT be treated as sentence endings.
# Lowercase for case-insensitive matching.
_ABBREVIATIONS = frozenset(
{
"mr",
"mrs",
"ms",
"dr",
"prof",
"sr",
"jr",
"st",
"ave",
"blvd",
"inc",
"ltd",
"corp",
"dept",
"est",
"approx",
"vs",
"etc",
"e.g",
"i.e",
"a.m",
"p.m",
"u.s",
"u.s.a",
"u.k",
}
)
# Paralinguistic tags used by Chatterbox Turbo. The splitter must never
# cut inside one of these.
_PARA_TAG_RE = re.compile(r"\[[^\]]*\]")
# ---------------------------------------------------------------------------
# Text splitting
# ---------------------------------------------------------------------------
def split_text_into_chunks(text: str, max_chars: int = DEFAULT_MAX_CHUNK_CHARS) -> List[str]:
"""Split *text* at natural boundaries into chunks of at most *max_chars*.
Priority: sentence-end (``.!?`` not preceded by an abbreviation and not
inside brackets) → clause boundary (``;:,—``) → whitespace → hard cut.
Paralinguistic tags like ``[laugh]`` are treated as atomic and will not
be split across chunks.
"""
text = text.strip()
if not text:
return []
if len(text) <= max_chars:
return [text]
chunks: List[str] = []
remaining = text
while remaining:
remaining = remaining.lstrip()
if not remaining:
break
if len(remaining) <= max_chars:
chunks.append(remaining)
break
segment = remaining[:max_chars]
# Try to split at the last real sentence ending
split_pos = _find_last_sentence_end(segment)
if split_pos == -1:
split_pos = _find_last_clause_boundary(segment)
if split_pos == -1:
split_pos = segment.rfind(" ")
if split_pos == -1:
# Absolute fallback: hard cut but avoid splitting inside a tag
split_pos = _safe_hard_cut(segment, max_chars)
chunk = remaining[: split_pos + 1].strip()
if chunk:
chunks.append(chunk)
remaining = remaining[split_pos + 1 :]
return chunks
def _find_last_sentence_end(text: str) -> int:
"""Return the index of the last sentence-ending punctuation in *text*.
Skips periods that follow common abbreviations (``Dr.``, ``Mr.``, etc.)
and periods inside bracket tags (``[laugh]``). Also handles CJK
sentence-ending punctuation (``。!?``).
"""
best = -1
# ASCII sentence ends
for m in re.finditer(r"[.!?](?:\s|$)", text):
pos = m.start()
char = text[pos]
# Skip periods after abbreviations
if char == ".":
# Walk backwards to find the preceding word
word_start = pos - 1
while word_start >= 0 and text[word_start].isalpha():
word_start -= 1
word = text[word_start + 1 : pos].lower()
if word in _ABBREVIATIONS:
continue
# Skip decimal numbers (digit immediately before the period)
if word_start >= 0 and text[word_start].isdigit():
continue
# Skip if we're inside a bracket tag
if _inside_bracket_tag(text, pos):
continue
best = pos
# CJK sentence-ending punctuation
for m in re.finditer(r"[\u3002\uff01\uff1f]", text):
if m.start() > best:
best = m.start()
return best
def _find_last_clause_boundary(text: str) -> int:
"""Return the index of the last clause-boundary punctuation."""
best = -1
for m in re.finditer(r"[;:,\u2014](?:\s|$)", text):
pos = m.start()
# Skip if inside a bracket tag
if _inside_bracket_tag(text, pos):
continue
best = pos
return best
def _inside_bracket_tag(text: str, pos: int) -> bool:
"""Return True if *pos* falls inside a ``[...]`` tag."""
for m in _PARA_TAG_RE.finditer(text):
if m.start() < pos < m.end():
return True
return False
def _safe_hard_cut(segment: str, max_chars: int) -> int:
"""Find a hard-cut position that doesn't split a ``[tag]``."""
cut = max_chars - 1
# Check if the cut falls inside a bracket tag; if so, move before it
for m in _PARA_TAG_RE.finditer(segment):
if m.start() < cut < m.end():
return m.start() - 1 if m.start() > 0 else cut
return cut
# ---------------------------------------------------------------------------
# Audio concatenation
# ---------------------------------------------------------------------------
def concatenate_audio_chunks(
chunks: List[np.ndarray],
sample_rate: int,
crossfade_ms: int = 50,
) -> np.ndarray:
"""Concatenate audio arrays with a short crossfade to eliminate clicks.
Each chunk is expected to be a 1-D float32 ndarray at *sample_rate* Hz.
"""
if not chunks:
return np.array([], dtype=np.float32)
if len(chunks) == 1:
return chunks[0]
crossfade_samples = int(sample_rate * crossfade_ms / 1000)
result = np.array(chunks[0], dtype=np.float32, copy=True)
for chunk in chunks[1:]:
if len(chunk) == 0:
continue
overlap = min(crossfade_samples, len(result), len(chunk))
if overlap > 0:
fade_out = np.linspace(1.0, 0.0, overlap, dtype=np.float32)
fade_in = np.linspace(0.0, 1.0, overlap, dtype=np.float32)
result[-overlap:] = result[-overlap:] * fade_out + chunk[:overlap] * fade_in
result = np.concatenate([result, chunk[overlap:]])
else:
result = np.concatenate([result, chunk])
return result
# ---------------------------------------------------------------------------
# Engine-agnostic chunked generation
# ---------------------------------------------------------------------------
async def generate_chunked(
backend,
text: str,
voice_prompt: dict,
language: str = "en",
seed: int | None = None,
instruct: str | None = None,
max_chunk_chars: int = DEFAULT_MAX_CHUNK_CHARS,
crossfade_ms: int = 50,
trim_fn=None,
) -> Tuple[np.ndarray, int]:
"""Generate audio with automatic chunking for long text.
For text shorter than *max_chunk_chars* this is a thin wrapper around
``backend.generate()`` with zero overhead.
For longer text the input is split at natural sentence boundaries,
each chunk is generated independently, optionally trimmed (useful for
Chatterbox engines that hallucinate trailing noise), and the results
are concatenated with a crossfade (or hard cut if *crossfade_ms* is 0).
Parameters
----------
backend : TTSBackend
Any backend implementing the ``generate()`` protocol.
text : str
Input text (may be arbitrarily long).
voice_prompt, language, seed, instruct
Forwarded to ``backend.generate()`` verbatim.
max_chunk_chars : int
Maximum characters per chunk (default 800).
crossfade_ms : int
Crossfade duration in milliseconds between chunks. 0 for a hard
cut with no overlap (default 50).
trim_fn : callable | None
Optional ``(audio, sample_rate) -> audio`` post-processing
function applied to each chunk before concatenation (e.g.
``trim_tts_output`` for Chatterbox engines).
Returns
-------
(audio, sample_rate) : Tuple[np.ndarray, int]
"""
chunks = split_text_into_chunks(text, max_chunk_chars)
if len(chunks) <= 1:
# Short text — single-shot fast path
audio, sample_rate = await backend.generate(
text, voice_prompt, language, seed, instruct,
)
if trim_fn is not None:
audio = trim_fn(audio, sample_rate)
return audio, sample_rate
# Long text — chunked generation
logger.info(
"Splitting %d chars into %d chunks (max %d chars each)",
len(text), len(chunks), max_chunk_chars,
)
audio_chunks: List[np.ndarray] = []
sample_rate: int | None = None
for i, chunk_text in enumerate(chunks):
logger.info(
"Generating chunk %d/%d (%d chars)",
i + 1, len(chunks), len(chunk_text),
)
# Vary the seed per chunk to avoid correlated RNG artefacts,
# but keep it deterministic so the same (text, seed) pair
# always produces the same output.
chunk_seed = (seed + i) if seed is not None else None
chunk_audio, chunk_sr = await backend.generate(
chunk_text, voice_prompt, language, chunk_seed, instruct,
)
if trim_fn is not None:
chunk_audio = trim_fn(chunk_audio, chunk_sr)
audio_chunks.append(np.asarray(chunk_audio, dtype=np.float32))
if sample_rate is None:
sample_rate = chunk_sr
audio = concatenate_audio_chunks(audio_chunks, sample_rate, crossfade_ms=crossfade_ms)
return audio, sample_rate
+356
View File
@@ -0,0 +1,356 @@
"""
Audio post-processing effects engine.
Uses Spotify's pedalboard library to apply professional-grade DSP effects
to generated audio. Effects are described as a JSON-serializable chain
(list of effect dicts) so they can be stored in the database and sent
over the API.
Supported effect types:
- chorus (flanger-style with short delays)
- reverb (room reverb)
- delay (echo / delay line)
- compressor (dynamic range compression)
- gain (volume adjustment in dB)
- highpass (high-pass filter)
- lowpass (low-pass filter)
- pitch_shift (semitone pitch shifting)
"""
from __future__ import annotations
import numpy as np
from typing import Any, Dict, List, Optional
from pedalboard import (
Pedalboard,
Chorus,
Reverb,
Compressor,
Gain,
HighpassFilter,
LowpassFilter,
Delay,
PitchShift,
)
# ---------------------------------------------------------------------------
# Effect registry: maps type names -> (pedalboard class, param definitions)
# ---------------------------------------------------------------------------
# Each param definition: (default, min, max, description)
EFFECT_REGISTRY: Dict[str, Dict[str, Any]] = {
"chorus": {
"cls": Chorus,
"label": "Chorus / Flanger",
"description": "Modulated delay for flanging or chorus effects. Short centre_delay_ms (<10) gives flanger; longer gives chorus.",
"params": {
"rate_hz": {"default": 1.0, "min": 0.01, "max": 20.0, "step": 0.01, "description": "LFO speed (Hz)"},
"depth": {"default": 0.5, "min": 0.0, "max": 1.0, "step": 0.01, "description": "Modulation depth"},
"feedback": {"default": 0.0, "min": 0.0, "max": 0.95, "step": 0.01, "description": "Feedback amount"},
"centre_delay_ms": {"default": 7.0, "min": 0.5, "max": 50.0, "step": 0.1, "description": "Centre delay (ms)"},
"mix": {"default": 0.5, "min": 0.0, "max": 1.0, "step": 0.01, "description": "Wet/dry mix"},
},
},
"reverb": {
"cls": Reverb,
"label": "Reverb",
"description": "Room reverb effect.",
"params": {
"room_size": {"default": 0.5, "min": 0.0, "max": 1.0, "step": 0.01, "description": "Room size"},
"damping": {"default": 0.5, "min": 0.0, "max": 1.0, "step": 0.01, "description": "High frequency damping"},
"wet_level": {"default": 0.33, "min": 0.0, "max": 1.0, "step": 0.01, "description": "Wet level"},
"dry_level": {"default": 0.4, "min": 0.0, "max": 1.0, "step": 0.01, "description": "Dry level"},
"width": {"default": 1.0, "min": 0.0, "max": 1.0, "step": 0.01, "description": "Stereo width"},
},
},
"delay": {
"cls": Delay,
"label": "Delay",
"description": "Echo / delay line.",
"params": {
"delay_seconds": {"default": 0.3, "min": 0.01, "max": 2.0, "step": 0.01, "description": "Delay time (seconds)"},
"feedback": {"default": 0.3, "min": 0.0, "max": 0.95, "step": 0.01, "description": "Feedback amount"},
"mix": {"default": 0.3, "min": 0.0, "max": 1.0, "step": 0.01, "description": "Wet/dry mix"},
},
},
"compressor": {
"cls": Compressor,
"label": "Compressor",
"description": "Dynamic range compression for consistent loudness.",
"params": {
"threshold_db": {"default": -20.0, "min": -60.0, "max": 0.0, "step": 0.5, "description": "Threshold (dB)"},
"ratio": {"default": 4.0, "min": 1.0, "max": 20.0, "step": 0.1, "description": "Compression ratio"},
"attack_ms": {"default": 10.0, "min": 0.1, "max": 100.0, "step": 0.1, "description": "Attack time (ms)"},
"release_ms": {"default": 100.0, "min": 10.0, "max": 1000.0,"step": 1.0, "description": "Release time (ms)"},
},
},
"gain": {
"cls": Gain,
"label": "Gain",
"description": "Volume adjustment in decibels.",
"params": {
"gain_db": {"default": 0.0, "min": -40.0, "max": 40.0, "step": 0.5, "description": "Gain (dB)"},
},
},
"highpass": {
"cls": HighpassFilter,
"label": "High-Pass Filter",
"description": "Removes frequencies below the cutoff.",
"params": {
"cutoff_frequency_hz": {"default": 80.0, "min": 20.0, "max": 8000.0, "step": 1.0, "description": "Cutoff frequency (Hz)"},
},
},
"lowpass": {
"cls": LowpassFilter,
"label": "Low-Pass Filter",
"description": "Removes frequencies above the cutoff.",
"params": {
"cutoff_frequency_hz": {"default": 8000.0, "min": 200.0, "max": 20000.0, "step": 1.0, "description": "Cutoff frequency (Hz)"},
},
},
"pitch_shift": {
"cls": PitchShift,
"label": "Pitch Shift",
"description": "Shift pitch up or down by semitones.",
"params": {
"semitones": {"default": 0.0, "min": -12.0, "max": 12.0, "step": 0.5, "description": "Semitones to shift"},
},
},
}
# ---------------------------------------------------------------------------
# Built-in presets
# ---------------------------------------------------------------------------
BUILTIN_PRESETS: Dict[str, Dict[str, Any]] = {
"robotic": {
"name": "Robotic",
"sort_order": 0,
"description": "Metallic robotic voice (flanger with slow LFO and high feedback)",
"effects_chain": [
{
"type": "chorus",
"enabled": True,
"params": {
"rate_hz": 0.2,
"depth": 1.0,
"feedback": 0.35,
"centre_delay_ms": 7.0,
"mix": 0.5,
},
},
],
},
"radio": {
"name": "Radio",
"sort_order": 1,
"description": "Thin AM-radio voice with band-pass filtering and light compression",
"effects_chain": [
{
"type": "highpass",
"enabled": True,
"params": {"cutoff_frequency_hz": 300.0},
},
{
"type": "lowpass",
"enabled": True,
"params": {"cutoff_frequency_hz": 3500.0},
},
{
"type": "compressor",
"enabled": True,
"params": {
"threshold_db": -15.0,
"ratio": 6.0,
"attack_ms": 5.0,
"release_ms": 50.0,
},
},
{
"type": "gain",
"enabled": True,
"params": {"gain_db": 6.0},
},
],
},
"echo_chamber": {
"name": "Echo Chamber",
"sort_order": 2,
"description": "Spacious reverb with trailing echo",
"effects_chain": [
{
"type": "reverb",
"enabled": True,
"params": {
"room_size": 0.85,
"damping": 0.3,
"wet_level": 0.45,
"dry_level": 0.55,
"width": 1.0,
},
},
{
"type": "delay",
"enabled": True,
"params": {
"delay_seconds": 0.25,
"feedback": 0.3,
"mix": 0.2,
},
},
],
},
"deep_voice": {
"name": "Deep Voice",
"sort_order": 99,
"description": "Lower pitch with added warmth",
"effects_chain": [
{
"type": "pitch_shift",
"enabled": True,
"params": {"semitones": -3.0},
},
{
"type": "lowpass",
"enabled": True,
"params": {"cutoff_frequency_hz": 6000.0},
},
{
"type": "compressor",
"enabled": True,
"params": {
"threshold_db": -18.0,
"ratio": 3.0,
"attack_ms": 10.0,
"release_ms": 150.0,
},
},
],
},
}
# ---------------------------------------------------------------------------
# Public API
# ---------------------------------------------------------------------------
def get_available_effects() -> List[Dict[str, Any]]:
"""Return the list of available effect types with their parameter definitions.
Used by the frontend to build the effects chain editor UI.
"""
result = []
for effect_type, info in EFFECT_REGISTRY.items():
result.append({
"type": effect_type,
"label": info["label"],
"description": info["description"],
"params": {
name: {k: v for k, v in pdef.items()}
for name, pdef in info["params"].items()
},
})
return result
def get_builtin_presets() -> Dict[str, Dict[str, Any]]:
"""Return all built-in effect presets."""
return BUILTIN_PRESETS
def validate_effects_chain(effects_chain: List[Dict[str, Any]]) -> Optional[str]:
"""Validate an effects chain configuration.
Returns None if valid, or an error message string.
"""
if not isinstance(effects_chain, list):
return "effects_chain must be a list"
for i, effect in enumerate(effects_chain):
if not isinstance(effect, dict):
return f"Effect at index {i} must be a dict"
effect_type = effect.get("type")
if effect_type not in EFFECT_REGISTRY:
return f"Unknown effect type '{effect_type}' at index {i}. Available: {list(EFFECT_REGISTRY.keys())}"
params = effect.get("params", {})
if not isinstance(params, dict):
return f"Effect '{effect_type}' at index {i}: params must be a dict"
registry = EFFECT_REGISTRY[effect_type]
for param_name, value in params.items():
if param_name not in registry["params"]:
return f"Effect '{effect_type}' at index {i}: unknown param '{param_name}'"
pdef = registry["params"][param_name]
if not isinstance(value, (int, float)):
return f"Effect '{effect_type}' at index {i}: param '{param_name}' must be a number"
if value < pdef["min"] or value > pdef["max"]:
return (
f"Effect '{effect_type}' at index {i}: param '{param_name}' "
f"must be between {pdef['min']} and {pdef['max']} (got {value})"
)
return None
def build_pedalboard(effects_chain: List[Dict[str, Any]]) -> Pedalboard:
"""Build a Pedalboard instance from an effects chain config.
Skips effects where ``enabled`` is ``False``.
"""
plugins = []
for effect in effects_chain:
if not effect.get("enabled", True):
continue
effect_type = effect["type"]
registry = EFFECT_REGISTRY[effect_type]
cls = registry["cls"]
# Merge defaults with provided params
params = {}
for pname, pdef in registry["params"].items():
params[pname] = effect.get("params", {}).get(pname, pdef["default"])
plugins.append(cls(**params))
return Pedalboard(plugins)
def apply_effects(
audio: np.ndarray,
sample_rate: int,
effects_chain: List[Dict[str, Any]],
) -> np.ndarray:
"""Apply an effects chain to audio data.
Args:
audio: Input audio array (1-D mono float32).
sample_rate: Sample rate in Hz.
effects_chain: List of effect configuration dicts.
Returns:
Processed audio array.
"""
if not effects_chain:
return audio
board = build_pedalboard(effects_chain)
# pedalboard expects shape (channels, samples)
if audio.ndim == 1:
audio_2d = audio[np.newaxis, :]
else:
audio_2d = audio
processed = board(audio_2d.astype(np.float32), sample_rate)
# Return same dimensionality as input
if audio.ndim == 1:
return processed[0]
return processed
+100
View File
@@ -0,0 +1,100 @@
"""
Monkey patch for huggingface_hub to force offline mode with cached models.
This prevents mlx_audio from making network requests when models are already downloaded.
"""
import os
from pathlib import Path
from typing import Optional, Union
def patch_huggingface_hub_offline():
"""
Monkey-patch huggingface_hub to force offline mode.
This must be called BEFORE importing mlx_audio.
"""
try:
import huggingface_hub
from huggingface_hub import constants as hf_constants
from huggingface_hub.file_download import _try_to_load_from_cache
# Store original function
original_try_load = _try_to_load_from_cache
def _patched_try_to_load_from_cache(
repo_id: str,
filename: str,
cache_dir: Union[str, Path, None] = None,
revision: Optional[str] = None,
repo_type: Optional[str] = None,
):
"""
Patched version that forces offline mode.
Returns None if not cached (instead of making network request).
"""
# Always use the original function, but we're already in HF_HUB_OFFLINE mode
result = original_try_load(
repo_id=repo_id,
filename=filename,
cache_dir=cache_dir,
revision=revision,
repo_type=repo_type,
)
if result is None:
# File not in cache - log this for debugging
cache_path = Path(hf_constants.HF_HUB_CACHE) / f"models--{repo_id.replace('/', '--')}"
print(f"[HF_PATCH] File not cached: {repo_id}/{filename}")
print(f"[HF_PATCH] Expected at: {cache_path}")
else:
print(f"[HF_PATCH] Cache hit: {repo_id}/{filename}")
return result
# Replace the function
import huggingface_hub.file_download as fd
fd._try_to_load_from_cache = _patched_try_to_load_from_cache
print("[HF_PATCH] huggingface_hub patched for offline mode")
except ImportError:
print("[HF_PATCH] huggingface_hub not found, skipping patch")
except Exception as e:
print(f"[HF_PATCH] Error patching huggingface_hub: {e}")
def ensure_original_qwen_config_cached():
"""
The MLX community model is based on the original Qwen model.
mlx_audio may try to fetch config from the original repo.
We need to ensure that config is available in the cache.
"""
from huggingface_hub import constants as hf_constants
# Original Qwen model that mlx_audio might reference
original_repo = "Qwen/Qwen3-TTS-12Hz-1.7B-Base"
mlx_repo = "mlx-community/Qwen3-TTS-12Hz-1.7B-Base-bf16"
cache_dir = Path(hf_constants.HF_HUB_CACHE)
original_path = cache_dir / f"models--{original_repo.replace('/', '--')}"
mlx_path = cache_dir / f"models--{mlx_repo.replace('/', '--')}"
# If original repo cache doesn't exist but MLX does, create a symlink or copy config
if not original_path.exists() and mlx_path.exists():
print(f"[HF_PATCH] Original repo not cached, but MLX version is")
print(f"[HF_PATCH] Creating symlink from {original_repo} -> {mlx_repo}")
try:
# Create a symlink so the cache lookup succeeds
original_path.parent.mkdir(parents=True, exist_ok=True)
original_path.symlink_to(mlx_path, target_is_directory=True)
print(f"[HF_PATCH] Symlink created successfully")
except Exception as e:
print(f"[HF_PATCH] Could not create symlink: {e}")
# Auto-apply patch when module is imported
if os.environ.get("VOICEBOX_OFFLINE_PATCH", "1") != "0":
patch_huggingface_hub_offline()
ensure_original_qwen_config_cached()
+211
View File
@@ -0,0 +1,211 @@
"""
Generation versions management module.
Each generation can have multiple audio versions: a clean (unprocessed)
version and any number of processed versions with different effects chains.
"""
from __future__ import annotations
import json
import uuid
from pathlib import Path
from typing import List, Optional
from sqlalchemy.orm import Session
from .database import (
GenerationVersion as DBGenerationVersion,
Generation as DBGeneration,
)
from .models import GenerationVersionResponse, EffectConfig
from . import config
def _version_response(v: DBGenerationVersion) -> GenerationVersionResponse:
"""Convert a DB version row to a Pydantic response."""
effects_chain = None
if v.effects_chain:
raw = json.loads(v.effects_chain)
effects_chain = [EffectConfig(**e) for e in raw]
return GenerationVersionResponse(
id=v.id,
generation_id=v.generation_id,
label=v.label,
audio_path=v.audio_path,
effects_chain=effects_chain,
source_version_id=v.source_version_id,
is_default=v.is_default,
created_at=v.created_at,
)
def list_versions(generation_id: str, db: Session) -> List[GenerationVersionResponse]:
"""List all versions for a generation."""
versions = (
db.query(DBGenerationVersion)
.filter_by(generation_id=generation_id)
.order_by(DBGenerationVersion.created_at)
.all()
)
return [_version_response(v) for v in versions]
def get_version(version_id: str, db: Session) -> Optional[GenerationVersionResponse]:
"""Get a specific version by ID."""
v = db.query(DBGenerationVersion).filter_by(id=version_id).first()
if not v:
return None
return _version_response(v)
def get_default_version(generation_id: str, db: Session) -> Optional[GenerationVersionResponse]:
"""Get the default version for a generation."""
v = (
db.query(DBGenerationVersion)
.filter_by(generation_id=generation_id, is_default=True)
.first()
)
if not v:
# Fallback: return the first version
v = (
db.query(DBGenerationVersion)
.filter_by(generation_id=generation_id)
.order_by(DBGenerationVersion.created_at)
.first()
)
if not v:
return None
return _version_response(v)
def create_version(
generation_id: str,
label: str,
audio_path: str,
db: Session,
effects_chain: Optional[List[dict]] = None,
is_default: bool = False,
source_version_id: Optional[str] = None,
) -> GenerationVersionResponse:
"""Create a new version for a generation.
If ``is_default`` is True, all other versions for this generation
are un-defaulted first.
"""
if is_default:
_clear_defaults(generation_id, db)
version = DBGenerationVersion(
id=str(uuid.uuid4()),
generation_id=generation_id,
label=label,
audio_path=audio_path,
effects_chain=json.dumps(effects_chain) if effects_chain else None,
source_version_id=source_version_id,
is_default=is_default,
)
db.add(version)
db.commit()
db.refresh(version)
# If this version is the default, update the generation's audio_path
if is_default:
gen = db.query(DBGeneration).filter_by(id=generation_id).first()
if gen:
gen.audio_path = audio_path
db.commit()
return _version_response(version)
def set_default_version(version_id: str, db: Session) -> Optional[GenerationVersionResponse]:
"""Set a version as the default for its generation."""
version = db.query(DBGenerationVersion).filter_by(id=version_id).first()
if not version:
return None
_clear_defaults(version.generation_id, db)
version.is_default = True
db.commit()
db.refresh(version)
# Update generation's audio_path to point to this version
gen = db.query(DBGeneration).filter_by(id=version.generation_id).first()
if gen:
gen.audio_path = version.audio_path
db.commit()
return _version_response(version)
def delete_version(version_id: str, db: Session) -> bool:
"""Delete a version. Cannot delete the last remaining version."""
version = db.query(DBGenerationVersion).filter_by(id=version_id).first()
if not version:
return False
# Don't allow deleting the last version
count = (
db.query(DBGenerationVersion)
.filter_by(generation_id=version.generation_id)
.count()
)
if count <= 1:
return False
was_default = version.is_default
gen_id = version.generation_id
# Delete audio file
audio_path = Path(version.audio_path)
if audio_path.exists():
audio_path.unlink()
db.delete(version)
db.commit()
# If this was the default, promote the first remaining version
if was_default:
first = (
db.query(DBGenerationVersion)
.filter_by(generation_id=gen_id)
.order_by(DBGenerationVersion.created_at)
.first()
)
if first:
first.is_default = True
db.commit()
gen = db.query(DBGeneration).filter_by(id=gen_id).first()
if gen:
gen.audio_path = first.audio_path
db.commit()
return True
def delete_versions_for_generation(generation_id: str, db: Session) -> int:
"""Delete all versions for a generation (used when deleting a generation)."""
versions = (
db.query(DBGenerationVersion)
.filter_by(generation_id=generation_id)
.all()
)
count = 0
for v in versions:
audio_path = Path(v.audio_path)
if audio_path.exists():
audio_path.unlink()
db.delete(v)
count += 1
if count > 0:
db.commit()
return count
def _clear_defaults(generation_id: str, db: Session) -> None:
"""Clear the is_default flag on all versions for a generation."""
db.query(DBGenerationVersion).filter_by(
generation_id=generation_id, is_default=True
).update({"is_default": False})
db.flush()
+19 -4
View File
@@ -4,6 +4,10 @@
"workspaces": {
"": {
"name": "voicebox",
"dependencies": {
"loaders.css": "^0.1.2",
"react-loaders": "^3.0.1",
},
"devDependencies": {
"@biomejs/biome": "2.3.12",
"@types/node": "^20.0.0",
@@ -13,7 +17,7 @@
},
"app": {
"name": "@voicebox/app",
"version": "0.1.11",
"version": "0.1.13",
"dependencies": {
"@dnd-kit/core": "^6.3.1",
"@dnd-kit/sortable": "^10.0.0",
@@ -68,7 +72,7 @@
},
"landing": {
"name": "@voicebox/landing",
"version": "0.1.11",
"version": "0.1.13",
"dependencies": {
"@radix-ui/react-separator": "^1.1.8",
"@radix-ui/react-slot": "^1.2.4",
@@ -93,7 +97,7 @@
},
"tauri": {
"name": "@voicebox/tauri",
"version": "0.1.11",
"version": "0.1.13",
"dependencies": {
"@tauri-apps/api": "^2.0.0",
"@tauri-apps/plugin-dialog": "^2.0.0",
@@ -116,7 +120,7 @@
},
"web": {
"name": "@voicebox/web",
"version": "0.1.11",
"version": "0.1.13",
"dependencies": {
"@tanstack/react-query": "^5.0.0",
"react": "^18.3.0",
@@ -125,6 +129,7 @@
"zustand": "^4.5.0",
},
"devDependencies": {
"@tailwindcss/vite": "^4.0.0",
"@types/react": "^18.3.0",
"@types/react-dom": "^18.3.0",
"@typescript-eslint/eslint-plugin": "^7.0.0",
@@ -677,6 +682,8 @@
"class-variance-authority": ["[email protected]", "", { "dependencies": { "clsx": "^2.1.1" } }, "sha512-Ka+9Trutv7G8M6WT6SeiRWz792K5qEqIGEGzXKhAE6xOWAY6pPH8U+9IY3oCMv6kqTmLsv7Xh/2w2RigkePMsg=="],
"classnames": ["[email protected]", "", {}, "sha512-saHYOzhIQs6wy2sVxTM6bUDsQO4F50V9RQ22qBpEdCW+I+/Wmke2HOl6lS6dTpdxVhb88/I6+Hs+438c3lfUow=="],
"client-only": ["[email protected]", "", {}, "sha512-IV3Ou0jSMzZrd3pZ48nLkT9DA7Ag1pnPzaiQhpW7c3RbcqqzvzzVu+L8gfqMp/8IM2MQtSiqaCxrrcfu8I8rMA=="],
"clsx": ["[email protected]", "", {}, "sha512-eYm0QWBtUrBWZWG0d386OGAw16Z995PiOVo2B7bjWSbHedGl5e0ZWaq65kOGgUSNesEIDkB9ISbTg/JK9dhCZA=="],
@@ -873,6 +880,8 @@
"lines-and-columns": ["[email protected]", "", {}, "sha512-7ylylesZQ/PV29jhEDl3Ufjo6ZX7gCqJr5F7PKrqc93v7fzSymt1BpwEU8nAUXs8qzzvqhbjhK5QZg6Mt/HkBg=="],
"loaders.css": ["[email protected]", "", {}, "sha512-Rhowlq24ey1VOeor+3wYOt9+MjaxBOJm1u4KlQgNC3+0xJ0LS4wq4iG57D/BPzvuD/7HHDGQOWJ+81oR2EI9bQ=="],
"locate-path": ["[email protected]", "", { "dependencies": { "p-locate": "^5.0.0" } }, "sha512-iPZK6eYjbxRu3uB4/WZ3EsEIMJFMqAoopl3R+zuq0UjcAm/MO6KCweDgPfP3elTztoKP3KtnVHxTn2NHBSDVUw=="],
"lodash.merge": ["[email protected]", "", {}, "sha512-0KpjqXRVvrYyCsX1swR/XTK0va6VQkQM6MNo7PqW77ByjAhoARA8EfrP1N4+KlKj8YS0ZUCtRT/YUuhyYDujIQ=="],
@@ -959,6 +968,8 @@
"prelude-ls": ["[email protected]", "", {}, "sha512-vkcDPrRZo1QZLbn5RLGPpg/WmIQ65qoWWhcGKf/b5eplkkarX0m9z8ppCat4mlOqUsWpyNuYgO3VRyrYHSzX5g=="],
"prop-types": ["[email protected]", "", { "dependencies": { "loose-envify": "^1.4.0", "object-assign": "^4.1.1", "react-is": "^16.13.1" } }, "sha512-oj87CgZICdulUohogVAR7AjlC0327U4el4L6eAvOqCeudMDVU0NThNaV+b9Df4dXgSP1gXMTnPdhfe/2qDH5cg=="],
"punycode": ["[email protected]", "", {}, "sha512-vYt7UD1U9Wg6138shLtLOvdAu+8DsC/ilFtEVHcH+wydcSpNE20AfSOduf6MkRFahL5FY7X1oU7nKVZFtfq8Fg=="],
"queue-microtask": ["[email protected]", "", {}, "sha512-NuaNSa6flKT5JaSYQzJok04JzTL1CA6aGhv5rfLW3PgqA+M2ChpZQnAC8h8i4ZFkBS8X5RqkDBHA7r4hej3K9A=="],
@@ -969,6 +980,10 @@
"react-hook-form": ["[email protected]", "", { "peerDependencies": { "react": "^16.8.0 || ^17 || ^18 || ^19" } }, "sha512-9SUJKCGKo8HUSsCO+y0CtqkqI5nNuaDqTxyqPsZPqIwudpj4rCrAz/jZV+jn57bx5gtZKOh3neQu94DXMc+w5w=="],
"react-is": ["[email protected]", "", {}, "sha512-24e6ynE2H+OKt4kqsOvNd8kBpV65zoxbA4BVsEOB3ARVWQki/DHzaUoC5KuON/BiccDaCCTZBuOcfZs70kR8bQ=="],
"react-loaders": ["[email protected]", "", { "dependencies": { "classnames": "^2.2.3" }, "peerDependencies": { "prop-types": ">=15.6.0", "react": ">=15" } }, "sha512-4igMNqs9Fb3d4Z+0UHIGQNJsw/37gX0nUO8QxupnEKRn1dtyYC1LGwk5GuaoDciMQCQc/MmPwb4Fn6ZfdoX1FQ=="],
"react-refresh": ["[email protected]", "", {}, "sha512-z6F7K9bV85EfseRCp2bzrpyQ0Gkw1uLoCel9XBVWPg/TjRj94SkJzUTGfOa4bs7iJvBWtQG0Wq7wnI0syw3EBQ=="],
"react-remove-scroll": ["[email protected]", "", { "dependencies": { "react-remove-scroll-bar": "^2.3.7", "react-style-singleton": "^2.2.3", "tslib": "^2.1.0", "use-callback-ref": "^1.3.3", "use-sidecar": "^1.1.3" }, "peerDependencies": { "@types/react": "*", "react": "^16.8.0 || ^17.0.0 || ^18.0.0 || ^19.0.0 || ^19.0.0-rc" }, "optionalPeers": ["@types/react"] }, "sha512-Iqb9NjCCTt6Hf+vOdNIZGdTiH1QSqr27H/Ek9sv/a97gfueI/5h1s3yRi1nngzMUaOOToin5dI1dXKdXiF+u0Q=="],
+41
View File
@@ -0,0 +1,41 @@
services:
voicebox:
build: .
container_name: voicebox
restart: unless-stopped
ports:
# Bind to localhost only for security
- "127.0.0.1:17493:17493"
volumes:
# Bind-mount for generated audio (customize the host path as needed)
# Host side: ./output/
# Container side: /app/data/generations/
- ./output:/app/data/generations
# Named volume for profiles, DB, cache (persists across container restarts)
- voicebox-data:/app/data
# HuggingFace model cache (so models aren't re-downloaded on rebuild)
- huggingface-cache:/home/voicebox/.cache/huggingface
environment:
- LOG_LEVEL=info
networks:
- voicebox-net
deploy:
resources:
limits:
cpus: '4'
memory: 8G
networks:
voicebox-net:
driver: bridge
volumes:
voicebox-data:
huggingface-cache:
+70
View File
@@ -0,0 +1,70 @@
# Accessibility: screen reader and keyboard improvements
## Summary
Improvements to support screen reader and keyboard users across the main app surfaces: audio player, generation UI, voice selection, history, voices tab, model management, server tab, and stories.
**Tested with NVDA and Narrator on Windows.**
---
## What changed
### Audio player (after generating audio)
- **Play/Pause, Loop, Mute, Close** – `aria-label` added so each control is announced (e.g. "Play", "Pause", "Loop", "Mute", "Close player").
- **Playback position slider** – `aria-label="Playback position"` and `aria-valuetext` with current/total time (e.g. "0:30 of 2:15").
- **Volume** – Wrapped in a labelled group; volume slider has an associated screen-reader-only label and `aria-valuetext` for the level (e.g. "Volume level, 75%").
### Generation UI (text box and voice choice)
- **Generate speech** (submit) and **Fine-tune instructions** (sliders) – Icon buttons now have `aria-label` (and state for fine-tune, e.g. "Fine-tune instructions, on").
### Voice selection (cards on Generate screen)
- Each **voice card** is focusable (`tabIndex={0}`), has `role="button"`, and an `aria-label` (e.g. "Prashant, en. Select as voice for generation.") with `aria-pressed` when selected.
- **Enter/Space** on the card selects that voice; tab order is card → Export/Edit/Delete.
### History list (generated samples)
- Each **sample row** is focusable with `role="button"` and an `aria-label` (e.g. "Sample from [profile], [duration], [date]. Press Enter to play."); **Enter/Space** plays or restarts.
- **Transcript textarea** has `aria-label` (e.g. "Transcript for sample from [profile], [duration]") so when you focus on the text area, the sample is announced in context.
### Voices tab (table)
- Each **voice row** is focusable with `role="button"` and an `aria-label` (e.g. "[Name], [language], [N] generations, [N] samples. Press Enter to edit."); **Enter/Space** opens edit (except when focus is in a control).
- **Actions** dropdown trigger has `aria-label="Actions for [profile name]"`.
### Model management
- Each **model row** is a focusable region (`tabIndex={0}`, `role="group"`) with an `aria-label` (e.g. "[Model name], [status], [size]. Use Tab to reach Download or Delete.").
- **Download** and **Delete** (and Downloading) buttons have `aria-label` (e.g. "Download [name]", "Delete [name]").
### Server tab (panels)
- **Server Connection**, **Server Status**, and **App Updates** cards are landmarks: `role="region"`, `aria-label`, and `tabIndex={0}` so each panel is focusable and announced (e.g. "Server Connection", "Server Status", "App Updates").
### Stories list
- Each **story row** is a focusable control (`role="button"`, `tabIndex={0}`) with `aria-label` (e.g. "Story [name], [N] items, [date]. Press Enter to select."); **Enter/Space** selects the story. Actions button has `aria-label="Actions for [story name]"`.
### Other controls
- **Story list** – Actions (⋮) button: `aria-label="Actions for [story name]"`.
- **Story track editor** – Play/Pause, Stop, Split, Duplicate, Delete, Zoom in/out: `aria-label` on all icon buttons.
- **Voice profile samples** (SampleList, AudioSampleUpload, AudioSampleRecording, AudioSampleSystem) – Play/Pause and Stop: `aria-label` (e.g. "Play sample", "Pause", "Stop playback").
- **SampleList** mini sample player – Seek slider has `aria-label="Sample playback position"` and `aria-valuetext` for time.
---
## Testing
- **Screen readers:** Tested with **NVDA** and **Narrator** on Windows.
- **Keyboard:** Tab order and Enter/Space activation verified for focusable rows and buttons.
---
## Tech note
- React + TypeScript; Radix UI primitives; labels added via `aria-label`, `aria-labelledby`, `aria-valuetext`, and `role`/`tabIndex` where needed.
- No new dependencies.
+3 -3
View File
@@ -162,7 +162,7 @@ chmod +x voicebox-*.AppImage
**Solutions:**
1. **Check server is running**
```bash
curl http://localhost:8000/health
curl http://localhost:17493/health
```
2. **Check remote mode**
@@ -170,7 +170,7 @@ chmod +x voicebox-*.AppImage
- Check firewall settings
3. **Check port availability**
- Default port is 8000
- The current local app and dev workflow uses port 17493 by default
- Ensure no other service is using it
### CORS errors in browser
@@ -276,7 +276,7 @@ chmod +x voicebox-*.AppImage
2. **Check OpenAPI endpoint**
```bash
curl http://localhost:8000/openapi.json
curl http://localhost:17493/openapi.json
```
3. **Regenerate client**
+67
View File
@@ -0,0 +1,67 @@
# Voicebox Issue Pain Points (Snapshot)
## Scope
- Dataset: **128 total issues** (**107 open**, **21 closed**)
- Source: GitHub issues in `jamiepine/voicebox`
- Classification: keyword/theme clustering
- Note: counts below are **non-exclusive** (one issue can belong to multiple pain points)
## Most Common Pain Points (Open Issues)
| Rank | Pain Point | Open Issues | What users are reporting |
|---|---|---:|---|
| 1 | Model download & offline reliability | **32** | Downloads failing/stalling, cache/offline behavior inconsistent, wrong model size selected, Errno issues |
| 2 | GPU/backend compatibility | **22** | GPU not detected, backend fallback surprises, platform-specific runtime failures (Windows/Mac) |
| 3 | Export/save/file persistence | **15** | Export fails, "failed to fetch/download audio", samples/profiles not saving |
| 4 | Language/accent quality & coverage | **14** | Missing language support, accent mismatch, robotic outputs |
| 5 | Update/restart safety + long-op controls | **4** | Auto-restart without warning, update confusion, lack of cancel/pause controls |
## Representative Issues by Pain Point
### 1) Model download & offline reliability (32)
- [#159](https://github.com/jamiepine/voicebox/issues/159) - Qwen download fails with Errno 22
- [#151](https://github.com/jamiepine/voicebox/issues/151) - Model loading hangs / server crashes
- [#150](https://github.com/jamiepine/voicebox/issues/150) - Internet required despite downloaded models
- [#149](https://github.com/jamiepine/voicebox/issues/149) - Cancel/pause controls for large downloads
- [#96](https://github.com/jamiepine/voicebox/issues/96) - 0.6B selection still uses/downloads 1.7B
### 2) GPU/backend compatibility (22)
- [#164](https://github.com/jamiepine/voicebox/issues/164) - Windows: no GPU usage + multiple breakages
- [#141](https://github.com/jamiepine/voicebox/issues/141) - Using CPU only, GPU not used
- [#131](https://github.com/jamiepine/voicebox/issues/131) - Numpy ABI mismatch in bundled app
- [#130](https://github.com/jamiepine/voicebox/issues/130) - Intel Mac tensor/padding generation error
- [#127](https://github.com/jamiepine/voicebox/issues/127) - GPU not found
### 3) Export/save/file persistence (15)
- [#148](https://github.com/jamiepine/voicebox/issues/148) - Japanese export fails on 0.1.12
- [#143](https://github.com/jamiepine/voicebox/issues/143) - Samples not saving
- [#134](https://github.com/jamiepine/voicebox/issues/134) - Can't save profile
- [#105](https://github.com/jamiepine/voicebox/issues/105) - Export audio fails (failed to fetch)
- [#49](https://github.com/jamiepine/voicebox/issues/49) - Export filename/location ignored on Windows
### 4) Language/accent quality & coverage (14)
- [#162](https://github.com/jamiepine/voicebox/issues/162) - Persian audio request/problem
- [#117](https://github.com/jamiepine/voicebox/issues/117) - Arabic language support
- [#113](https://github.com/jamiepine/voicebox/issues/113) - Polish language support
- [#109](https://github.com/jamiepine/voicebox/issues/109) - Ukrainian support
- [#100](https://github.com/jamiepine/voicebox/issues/100) - Non-US accent quality issues
### 5) Update/restart safety + controls (4)
- [#164](https://github.com/jamiepine/voicebox/issues/164) - Update behavior + usability failures
- [#136](https://github.com/jamiepine/voicebox/issues/136) - Auto-restart without warning
- [#86](https://github.com/jamiepine/voicebox/issues/86) - Unexpected restart with no confirmation
- [#149](https://github.com/jamiepine/voicebox/issues/149) - Need pause/cancel and pre-download confirmation
## Additional Signal
- There is also a large **feature-request/misc** bucket (**36 open**) that is competing with stability triage (audiobook, Linux build, additional ASR/TTS models, integrations).
## Takeaway
Most user pain is concentrated in four stability areas: **download/offline path**, **GPU/backend detection**, **save/export reliability**, and **language/accent correctness**. Addressing those first should reduce the majority of current support friction.
+222 -194
View File
@@ -1,6 +1,6 @@
# Voicebox Project Status & Roadmap
> Last updated: 2026-03-12 | Current version: **v0.1.13** | 13.1k stars | 176 open issues | 28 open PRs
> Last updated: 2026-03-13 | Current version: **v0.1.13** | 13.1k stars | ~176 open issues | 25 open PRs
---
@@ -30,14 +30,18 @@
│ │ HTTP :17493 │
│ ┌──────────────────────▼────────────────────────┐ │
│ │ FastAPI Backend (backend/) │ │
│ │ ┌─────────────┐ ┌───────────┐ ┌─────────┐ │ │
│ │ │ TTSBackend │ │ STTBackend│ │ Profiles│ │ │
│ │ │ (Protocol) │ │ (Whisper) │ │ History │ │ │
│ │ │ ┌────────┐ │ └───────────┘ │ Stories │ │ │
│ │ │ │PyTorch │ │ └─────────┘ │ │
│ │ │ │or MLX │ │ │ │
│ │ │ └────────┘ │ │ │
│ │ └─────────────┘ │ │
│ │ ┌─────────────────────────────────────────┐ │ │
│ │ │ TTSBackend Protocol │ │ │
│ │ │ ┌──────────┐ ┌───────┐ ┌───────────┐ │ │ │
│ │ │ │ Qwen3-TTS│ │LuxTTS │ │Chatterbox │ │ │ │
│ │ │ │(Py/MLX) │ │ │ │(MTL+Turbo)│ │ │ │
│ │ │ └──────────┘ └───────┘ └───────────┘ │ │ │
│ │ └─────────────────────────────────────────┘ │ │
│ │ ┌───────────┐ ┌─────────┐ │ │
│ │ │ STTBackend│ │ Profiles│ │ │
│ │ │ (Whisper) │ │ History │ │ │
│ │ └───────────┘ │ Stories │ │ │
│ │ └─────────┘ │ │
│ └───────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────┘
```
@@ -46,131 +50,180 @@
| Layer | File | Purpose |
|-------|------|---------|
| Backend entry | `backend/main.py` | FastAPI app, all API routes (~1700 lines) |
| Backend entry | `backend/main.py` | FastAPI app, all API routes (~2100 lines) |
| TTS protocol | `backend/backends/__init__.py:14-81` | `TTSBackend` Protocol definition |
| TTS factory | `backend/backends/__init__.py:118-137` | Singleton backend selection (MLX vs PyTorch) |
| TTS factory | `backend/backends/__init__.py:138-178` | Thread-safe engine registry (double-checked locking) |
| PyTorch TTS | `backend/backends/pytorch_backend.py` | Qwen3-TTS via `qwen_tts` package |
| MLX TTS | `backend/backends/mlx_backend.py` | Qwen3-TTS via `mlx_audio.tts` |
| LuxTTS | `backend/backends/luxtts_backend.py` | LuxTTS — fast, CPU-friendly |
| Chatterbox MTL | `backend/backends/chatterbox_backend.py` | Chatterbox Multilingual — 23 languages |
| Chatterbox Turbo | `backend/backends/chatterbox_turbo_backend.py` | Chatterbox Turbo — English, paralinguistic tags |
| Platform detect | `backend/platform_detect.py` | Apple Silicon → MLX, else → PyTorch |
| API types | `backend/models.py` | Pydantic request/response models |
| HF progress | `backend/utils/hf_progress.py` | HFProgressTracker (tqdm patching for download progress) |
| Audio utils | `backend/utils/audio.py` | `trim_tts_output()`, normalize, load/save audio |
| Frontend API | `app/src/lib/api/client.ts` | Hand-written fetch wrapper |
| Frontend types | `app/src/lib/api/types.ts` | TypeScript API types |
| Generation form | `app/src/components/Generation/GenerationForm.tsx` | TTS generation UI |
| Model manager | `app/src/components/ServerSettings/ModelManagement.tsx` | Model download/status UI |
| Floating gen box | `app/src/components/Generation/FloatingGenerateBox.tsx` | Compact generation UI |
| Model manager | `app/src/components/ServerSettings/ModelManagement.tsx` | Model download/status/progress UI |
| GPU acceleration | `app/src/components/ServerSettings/GpuAcceleration.tsx` | CUDA backend swap UI |
| Gen form hook | `app/src/lib/hooks/useGenerationForm.ts` | Form validation + submission |
| Language constants | `app/src/lib/constants/languages.ts` | Per-engine language maps |
### How TTS Generation Works (Current Flow)
```
POST /generate
1. Look up voice profile from DB
2. Check model cache → if missing, trigger background download, return HTTP 202
3. Load model (lazy): tts_backend.load_model(model_size)
4. Create voice prompt: profiles.create_voice_prompt_for_profile()
2. Resolve engine from request (qwen | luxtts | chatterbox | chatterbox_turbo)
3. Get backend: get_tts_backend_for_engine(engine) # thread-safe singleton per engine
4. Check model cache → if missing, trigger background download, return HTTP 202
5. Load model (lazy): tts_backend.load_model(model_size)
6. Create voice prompt: profiles.create_voice_prompt_for_profile(engine=engine)
→ tts_backend.create_voice_prompt(audio_path, reference_text)
5. Generate: tts_backend.generate(text, voice_prompt, language, seed, instruct)
6. Save WAV → data/generations/{id}.wav
7. Insert history record in SQLite
8. Return GenerationResponse
7. Generate: tts_backend.generate(text, voice_prompt, language, seed, instruct)
8. Post-process: trim_tts_output() for Chatterbox engines
9. Save WAV → data/generations/{id}.wav
10. Insert history record in SQLite
11. Return GenerationResponse
```
---
## Current State
### What's Shipped (v0.1.13)
### What's Shipped (v0.1.13 + recent merges)
**Core TTS:**
- Qwen3-TTS voice cloning (1.7B and 0.6B models)
- MLX backend for Apple Silicon, PyTorch for everything else
- Multi-engine TTS architecture with thread-safe backend registry (PR #254)
- LuxTTS integration — fast, CPU-friendly English TTS (PR #254)
- Chatterbox Multilingual TTS — 23 languages including Hebrew (PR #257)
- Delivery instructions (instruct parameter, Qwen only)
- Single flat model dropdown (Qwen 1.7B, Qwen 0.6B, LuxTTS, Chatterbox, Chatterbox Turbo)
**Infrastructure:**
- CUDA backend swap via binary download and restart (PR #252)
- GPU acceleration settings UI
- Voice profiles with multi-sample support
- Stories editor (multi-track DAW timeline)
- Whisper transcription (base, small, medium, large variants)
- Model management UI with download progress (SSE)
- Model management UI with inline download progress bars (HFProgressTracker)
- Download cancel/clear UI with error panel (PR #238)
- Generation history with caching
- Streaming generation endpoint (MLX only)
- Delivery instructions (instruct parameter)
- Duplicate profile name validation (PR #175)
- Linux NVIDIA GBM buffer + WebKitGTK microphone fix (PR #210)
### What's NOT Shipped But Has Code
### What's In-Flight
| Feature | Branch | Status |
|---------|--------|--------|
| External provider binaries (CUDA split) | `external-provider-binaries` | PR #33, significant work done, stale since Feb |
| Dual server binaries | `feat/dual-server-binaries` | Branch exists, no PR |
| Multi-sample fix | `fix-multi-sample` | Branch exists, no PR |
| Model download notification fix | `fix-dl-notification-...` | Branch exists, no PR |
| Feature | Branch/PR | Status |
|---------|-----------|--------|
| Chatterbox Turbo + per-engine language lists | `feat/chatterbox-turbo` / PR #258 | Open, ready for review |
### Hardcoded Qwen3-TTS Assumptions
### TTS Engine Comparison
These are the specific coupling points that block multi-model support:
| Engine | Model Name | Languages | Size | Key Features |
|--------|-----------|-----------|------|-------------|
| Qwen3-TTS 1.7B | `qwen-tts-1.7B` | 10 (zh, en, ja, ko, de, fr, ru, pt, es, it) | ~3.5 GB | Instruct mode, highest quality |
| Qwen3-TTS 0.6B | `qwen-tts-0.6B` | 10 | ~1.2 GB | Lighter, faster |
| LuxTTS | `luxtts` | English | ~300 MB | CPU-friendly, 48 kHz, fast |
| Chatterbox | `chatterbox-tts` | 23 (incl. Hebrew, Arabic, Hindi, etc.) | ~3.2 GB | Zero-shot cloning, multilingual |
| Chatterbox Turbo | `chatterbox-turbo` | English | ~1.5 GB | Paralinguistic tags ([laugh], [cough]), 350M params, low latency |
| Location | What's Hardcoded |
|----------|-----------------|
| `backend/models.py:58` | `model_size` regex: `^(1\.7B\|0\.6B)$` |
| `backend/main.py:611` | Default: `model_size or "1.7B"` |
| `backend/main.py:1322-1365` | Model status list (2 Qwen + 4 Whisper) |
| `backend/main.py:1523-1548` | Download trigger map |
| `backend/main.py:1597-1628` | Delete map |
| `backend/backends/pytorch_backend.py:65-68` | HF repo ID map |
| `backend/backends/mlx_backend.py:41-44` | MLX repo ID map |
| `backend/backends/__init__.py:118-137` | Single global TTS backend |
| `app/src/lib/hooks/useGenerationForm.ts:17` | `modelSize: z.enum(['1.7B', '0.6B'])` |
| `app/src/lib/hooks/useGenerationForm.ts:70-71` | `modelName = "qwen-tts-${data.modelSize}"` |
| `app/src/components/Generation/GenerationForm.tsx:140-141` | Hardcoded "Qwen TTS" labels |
| `app/src/components/ServerSettings/ModelManagement.tsx:166-213` | Filters by `qwen-tts` and `whisper` prefix |
| `backend/utils/cache.py` | Voice prompt cache uses `torch.save()` |
### Multi-Engine Architecture (Shipped)
The singleton TTS backend blocker described in the previous version of this doc has been **resolved**. The architecture now supports:
- **Thread-safe backend registry** (`_tts_backends` dict + `_tts_backends_lock`) with double-checked locking
- **Per-engine backend instances** — each engine gets its own singleton, loaded lazily
- **Engine field on GenerationRequest** — frontend sends `engine: 'qwen' | 'luxtts' | 'chatterbox' | 'chatterbox_turbo'`
- **Per-engine language filtering** — `ENGINE_LANGUAGES` map in frontend, backend regex accepts all languages
- **Per-engine voice prompts** — `create_voice_prompt_for_profile()` dispatches to the correct backend
- **Trim post-processing** — `trim_tts_output()` for Chatterbox engines (cuts trailing silence/hallucination)
### Known Limitations
- **HF XET progress**: Large files downloaded via `hf-xet` (HuggingFace's new transfer backend) report `n=0` in tqdm updates. Progress bars may appear stuck for large `.safetensors` files even though the download is proceeding. This is a known upstream limitation.
- **Chatterbox Turbo upstream token bug**: `from_pretrained()` passes `token=os.getenv("HF_TOKEN") or True` which fails without a stored HF token. Our backend works around this by calling `snapshot_download(token=None)` + `from_local()`.
- **chatterbox-tts must install with `--no-deps`**: It pins `numpy<1.26`, `torch==2.6.0`, `transformers==4.46.3` — all incompatible with our stack (Python 3.12, torch 2.10, transformers 4.57.3). Sub-deps listed explicitly in `requirements.txt`.
- **Streaming generation** only works for Qwen on MLX. Other engines use the non-streaming `/generate` endpoint.
- **dicta-onnx** (Hebrew diacritization) not included — upstream Chatterbox bug requires `model_path` arg but calls `Dicta()` with none. Hebrew works fine without it.
---
## Open PRs — Triage & Analysis
### Recently Merged (Since Last Update)
| PR | Title | Merged |
|----|-------|--------|
| **#257** | feat: Chatterbox TTS engine with multilingual voice cloning | 2026-03-13 |
| **#254** | feat: LuxTTS integration — multi-engine TTS support | 2026-03-13 |
| **#252** | feat: CUDA backend swap via binary download and restart | 2026-03-13 |
| **#238** | Download cancel/clear UI, fixed model downloading | 2026-03-13 |
| **#250** | docs: align local API port examples | 2026-03-13 |
| **#210** | fix: Linux NVIDIA GBM buffer crash | 2026-03-13 |
| **#175** | Fix #134: duplicate profile name validation | 2026-03-13 |
### In-Flight (Our Work)
| PR | Title | Status | Notes |
|----|-------|--------|-------|
| **#258** | feat: Chatterbox Turbo engine + per-engine language lists | Open | Ready for review. Adds Turbo engine + dynamic language dropdown. |
### Merge-Ready / Near-Ready (Bug Fixes & Small Features)
| PR | Title | Risk | Notes |
|----|-------|------|-------|
| **#250** | docs: align local API port examples | None | Docs-only |
| **#230** | docs: fix README grammar | None | Docs-only |
| **#243** | a11y: screen reader and keyboard improvements | Low | Accessibility, no backend changes |
| **#175** | Fix #134: duplicate profile name validation | Low | Simple validation |
| **#178** | Fix #168 #140: generation error handling | Low | Error handling improvements |
| **#152** | Fix: prevent crashes when HuggingFace unreachable | Medium | Monkey-patches HF hub; solves real offline bug (#150, #151) |
| **#218** | fix: unify qwen tts cache dir on Windows | Low | Windows-specific path fix |
| **#214** | fix: panic on launch from tokio::spawn | Low | Rust-side Tauri fix |
| **#210** | fix: Linux NVIDIA GBM buffer crash | Low | Linux-specific, narrowly scoped |
| **#88** | security: restrict CORS to known local origins | Low | Security hardening |
| **#133** | feat: network access toggle | Low | Wires up existing plumbing |
### Significant Feature PRs
| PR | Title | Complexity | Dependencies | Notes |
|----|-------|-----------|--------------|-------|
| **#97** | fix: pass language parameter to TTS models | Medium | None | **Critical bug** — language param was silently dropped. Adds `LANGUAGE_CODE_TO_NAME` mapping to both backends. Should be high priority. |
| **#133** | feat: network access toggle | Low | None | Wires up existing plumbing (`--host 0.0.0.0`). Clean, small. |
| **#238** | download cancel/clear UI + error panel | Medium | None | Adds cancel buttons, VS Code-style Problems panel, fixes whisper-large repo. Quality-of-life win. |
| **#99** | feat: chunked TTS with quality selector | Medium | None | Solves the 500-char/2048-token limit. Sentence-aware splitting, crossfade concat, 44.1kHz upsampling. Addresses #191, #203, #69, #111. |
| **#154** | feat: Audiobook tab | Medium | Depends on #99 concepts | Full audiobook workflow — chunked gen, preview, auto-save to Stories. New route + tab. |
| **#91** | fix: CoreAudio device enumeration | Medium | None | macOS audio device handling. |
| PR | Title | Complexity | Notes |
|----|-------|-----------|-------|
| **#253** | Enhance speech tokenizer with 48kHz version | Medium | Qwen tokenizer upgrade |
| **#97** | fix: pass language parameter to TTS models | Medium | May be partially obsoleted by multi-engine work — needs review |
| **#99** | feat: chunked TTS with quality selector | Medium | Solves 500-char limit. Addresses #191, #203, #69, #111. |
| **#154** | feat: Audiobook tab | Medium | Full audiobook workflow. Depends on #99 concepts. |
| **#91** | fix: CoreAudio device enumeration | Medium | macOS audio device handling |
### Architectural PRs (Need Careful Review)
| PR | Title | Complexity | Notes |
|----|-------|-----------|-------|
| **#33** | CUDA GPU Support — External Provider Binaries | **Very High** | The big one. Splits monolithic backend into main app + downloadable provider executables (PyTorch CPU, CUDA). New provider management system, CI/CD for R2 uploads, provider settings UI. Created Feb 1, significant codebase. **This is the foundation for multi-model support** but is currently Qwen-only. |
| **#225** | feat: custom HuggingFace model support | High | Adds `custom_models.py`, `custom:<slug>` model IDs, frontend model grouping (Built-in vs Custom). **Takes a different approach than #33** — keeps single backend but allows arbitrary HF repos. These two PRs may conflict architecturally. |
| **#194** | feat: Hebrew + Chatterbox TTS | High | **First non-Qwen TTS model.** Adds `ChatterboxTTSBackend` alongside existing backends. Routes by language (`he` → Chatterbox, else → Qwen). Adds Hebrew Whisper models. Includes a lot of cleanup. Important precedent for multi-model. |
| **#195** | feat: per-profile LoRA fine-tuning | **Very High** | Depends on #194. Training pipeline, adapter management, SSE progress, 15 new API endpoints. New DB tables. Forces PyTorch even on MLX systems for adapter inference. |
| **#161** | feat: Docker + web deployment | High | 3-stage Dockerfile, SPA serving from FastAPI, docker-compose. Implements the Docker deployment plan. |
| **#124** | Add Dockerfiles + docker-compose + docs | Medium | Earlier, simpler Docker attempt. Overlaps with #161. |
| **#123** | added docker | Low | Minimal Docker PR. Overlaps with #161 and #124. |
| **#227** | fix: harden input validation & file safety | Medium | Follow-up to #225. Atomic writes, threading locks, input validation. Good hardening but coupled to the custom models feature. |
| **#225** | feat: custom HuggingFace model support | High | Arbitrary HF repo loading. May need rework given multi-engine arch is now shipped. |
| **#194** | feat: Hebrew + Chatterbox TTS | High | **Superseded** by PR #257 which shipped Chatterbox multilingual (23 langs incl. Hebrew). May be closeable. |
| **#195** | feat: per-profile LoRA fine-tuning | Very High | Training pipeline, adapter management, 15 new endpoints. Depends on #194 (now superseded). |
| **#161** | feat: Docker + web deployment | High | 3-stage Dockerfile, SPA serving. Independent of TTS engine work. |
| **#124** / **#123** | Docker (simpler attempts) | Low-Medium | Overlap with #161 |
| **#227** | fix: harden input validation & file safety | Medium | Coupled to #225 (custom models) |
### PRs That Need Author Action / Are Stale
| PR | Title | Notes |
|----|-------|-------|
| **#237** | fix: bundle qwen_tts source files in PyInstaller | Solves #212 but needs review for build system impact |
| **#237** | fix: bundle qwen_tts source files in PyInstaller | Build system, needs review |
| **#215** | Update prerequisites with Tauri deps | Branch is `main` — will have conflicts |
| **#89** | Linux Support | Branch is `main` — will have conflicts. Broad scope. |
| **#83** | Update download links for v0.1.12 | Outdated (we're on v0.1.13) |
### PRs Likely Superseded
| PR | Superseded By | Notes |
|----|--------------|-------|
| **#194** (Hebrew + Chatterbox) | PR #257 (merged) | #257 ships Chatterbox multilingual with 23 languages including Hebrew. #194 took a different approach (route by language). Can likely be closed. |
| **#33** (External provider binaries) | PR #252 (merged) | #252 shipped CUDA backend swap. #33's broader provider architecture may still have value but needs reassessment. |
---
## Open Issues — Categorized
@@ -186,15 +239,15 @@ The single most reported category. Users on Windows with NVIDIA GPUs frequently
**Key issues:** #239, #222, #220, #217, #208, #198, #192, #167, #164, #141, #130, #127
**Fix path:** PR #33 (external provider binaries) is designed to solve this. Ship a small main app, let users download the CUDA provider separately.
**Fix path:** PR #252 (CUDA backend swap) is now merged. Users can download the CUDA binary separately from the GPU acceleration settings. Many of these issues may now be resolvable — needs triage to confirm.
### Model Downloads (20 issues)
Second most reported. Users get stuck downloads, can't resume, no cancel button, no offline fallback.
Second most reported. Users get stuck downloads, can't resume, no offline fallback.
**Key issues:** #249, #240, #221, #216, #212, #181, #180, #159, #150, #149, #145, #143, #135, #134
**Fix path:** PR #238 (cancel/clear UI), PR #152 (offline crash fix). Resume support not yet addressed.
**Fix path:** PR #238 (cancel/clear UI) is now merged. PR #152 (offline crash fix) still open. Inline progress bars now show for all engines. Resume support not yet addressed.
### Language Requests (18 issues)
@@ -202,7 +255,7 @@ Strong demand for: Hindi (#245), Indonesian (#247), Dutch (#236), Hebrew (#199),
**Key issues:** #247, #245, #236, #211, #205, #199, #189, #188, #187, #183, #179, #162
**Fix path:** PR #97 (pass language param — currently silently dropped!) is the prerequisite. Qwen3-TTS already supports many languages; the bug is that the language code isn't forwarded. Multi-model (#194 Chatterbox for Hebrew) expands coverage further.
**Fix path:** Chatterbox Multilingual (merged via #257) now supports 23 languages including many of the requested ones: Arabic, Danish, German, Greek, Finnish, Hebrew, Hindi, Dutch, Norwegian, Polish, Swedish, Swahili, Turkish. Per-engine language filtering (PR #258) ensures the UI shows correct options. Several of these issues may be closeable.
### New Model Requests (5 explicit issues)
@@ -214,7 +267,7 @@ Strong demand for: Hindi (#245), Indonesian (#247), Dutch (#236), Hebrew (#199),
| #132 | LavaSR (transcription) |
| #76 | (General model expansion) |
Community is also vocally requesting: LuxTTS, Chatterbox, XTTS-v2, Fish Speech, CosyVoice, Kokoro on social media and in issue comments.
Community also requests: XTTS-v2, Fish Speech, CosyVoice, Kokoro. The multi-engine architecture is now in place, making new model integration significantly easier.
### Long-Form / Chunking (5 issues)
@@ -255,153 +308,128 @@ Notable requests:
| Document | Target Version | Status | Relevance |
|----------|---------------|--------|-----------|
| `TTS_PROVIDER_ARCHITECTURE.md` | v0.1.13 | **Partially implemented** in PR #33 | Core architecture for multi-model + CUDA distribution |
| `EXTERNAL_PROVIDERS.md` | v0.2.0 | **Not started** | Remote server support. API path inconsistency with provider arch doc (`/v1/` vs `/tts/`) |
| `MLX_AUDIO.md` | — | **Shipped** (the only one) | MLX backend is live. 0.6B MLX model still missing. |
| `DOCKER_DEPLOYMENT.md` | v0.2.0 | **PR exists** (#161) | Waiting on review. No official images published. |
| `OPENAI_SUPPORT.md` | v0.2.0 | **Not started** | OpenAI-compatible API layer. Linked to issue #10. Low complexity. |
### Cross-Document Conflicts
1. **API path inconsistency:** Provider arch uses `/tts/generate`, External providers uses `/v1/generate`, OpenAI compat uses `/v1/audio/speech`. Need to reconcile.
2. **Docker vs. Provider split:** Docker doc assumes monolithic backend. Provider arch splits into separate binaries. Need to decide: does Docker run the monolith or individual providers?
3. **Version targeting:** Provider arch targets v0.1.13 (current!) but isn't merged. Everything else targets v0.2.0.
| `TTS_PROVIDER_ARCHITECTURE.md` | v0.1.13 | **Partially superseded** by multi-engine arch + CUDA swap | Core concepts implemented differently than planned |
| `CUDA_BACKEND_SWAP.md` | — | **Shipped** (PR #252) | CUDA binary download + backend restart |
| `CUDA_BACKEND_SWAP_FINAL.md` | — | **Shipped** (PR #252) | Final implementation plan |
| `EXTERNAL_PROVIDERS.md` | v0.2.0 | **Not started** | Remote server support |
| `MLX_AUDIO.md` | — | **Shipped** | MLX backend is live |
| `DOCKER_DEPLOYMENT.md` | v0.2.0 | **PR exists** (#161) | Waiting on review |
| `OPENAI_SUPPORT.md` | v0.2.0 | **Not started** | OpenAI-compatible API layer |
| `PR33_CUDA_PROVIDER_REVIEW.md` | — | **Reference** | Analysis of the original provider approach |
---
## New Model Integration — Landscape
### Models Worth Supporting (2026 SOTA)
### Models Worth Supporting (2026 SOTA — updated March 13)
| Model | Cloning | Speed | Sample Rate | Languages | VRAM | Integration Ease | Repo |
|-------|---------|-------|-------------|-----------|------|-----------------|------|
| **LuxTTS** | 3s zero-shot | 150x RT, CPU ok | 48 kHz | English-first | <1 GB | Easy | `ysharma3501/LuxTTS` |
| **Chatterbox** | 5s zero-shot | Sub-200ms streaming | 24-48 kHz | 23+ | Low | Medium | `resemble-ai/chatterbox` |
| **XTTS-v2** | 6s zero-shot | Fast mid-GPU | 24 kHz | 17+ | Medium | Medium | `coqui/XTTS-v2` |
| **Fish Speech** | 10-30s few-shot | Real-time | 24-44 kHz | 50+ | Medium | Medium | `fishaudio/fish-speech` |
| **CosyVoice2-0.5B** | 3-10s zero-shot | Very fast | 24 kHz | Multilingual | Low | Easy | Alibaba HF org |
| **Kokoro-82M** | 3s instant | CPU realtime | 24 kHz | English | Tiny | Medium | Kokoro repo |
| Model | Cloning | Speed | Sample Rate | Languages | VRAM | Integration Ease | Status |
|-------|---------|-------|-------------|-----------|------|-----------------|--------|
| **Qwen3-TTS** | 10s zero-shot | Medium | 24 kHz | 10 | Medium | **Shipped** | v0.1.13 |
| **LuxTTS** | 3s zero-shot | 150x RT, CPU ok | 48 kHz | English | <1 GB | **Shipped** | PR #254 |
| **Chatterbox MTL** | 5s zero-shot | Medium | 24 kHz | 23 | Medium | **Shipped** | PR #257 |
| **Chatterbox Turbo** | 5s zero-shot | Fast | 24 kHz | English | Low | **PR #258** | In review |
| **HumeAI TADA 1B/3B** | Zero-shot | 5× faster than LLM-TTS | — | EN (1B), Multilingual (3B) | Medium | Needs vetting | MIT, 700s+ coherent, synced transcript output |
| **MOSS-TTS Family** | Zero-shot | — | — | Multilingual | Medium | Needs vetting | Apache 2.0, multi-speaker dialogue, text-to-voice design (no ref audio) |
| **VoxCPM 1.5** | Zero-shot (seconds) | ~0.15 RTF streaming | — | Bilingual (EN/ZH) | Medium | Needs vetting | Apache 2.0, tokenizer-free continuous diffusion, LoRA-friendly |
| **Pocket TTS** | Zero-shot + streaming | >1× RT on CPU | — | English | ~100M params, CPU-first | Needs vetting | MIT, Kyutai Labs, no GPU required |
| **Kokoro-82M** | 3s instant | CPU realtime | 24 kHz | English | Tiny (82M) | Ready | Apache 2.0, multi-engine arch in place |
| **XTTS-v2** | 6s zero-shot | Mid-GPU | 24 kHz | 17+ | Medium | Ready | Multi-engine arch in place |
| **Fish Speech** | 10-30s few-shot | Real-time | 24-44 kHz | 50+ | Medium | Ready | Multi-engine arch in place |
| **CosyVoice2-0.5B** | 3-10s zero-shot | Very fast | 24 kHz | Multilingual | Low | Ready | Multi-engine arch in place |
### What's Needed Architecturally for Multi-Model
#### Notes on New Candidates (March 2026)
The current codebase assumes one TTS model family (Qwen3-TTS). Adding any new model requires:
- **HumeAI TADA** — Text-Audio Dual Alignment arch. Near-zero hallucinations/drift, free synced transcript. 700+ seconds coherent audio. Best candidate for Stories long-form reliability. [HF: HumeAI/tada-1b](https://huggingface.co/HumeAI/tada-1b) | [GitHub: HumeAI/tada](https://github.com/HumeAI/tada)
- **MOSS-TTS** — Modular suite: flagship cloning, MOSS-TTSD (multi-speaker dialogue), MOSS-VoiceGenerator (create voices from text descriptions, no ref audio). Unique UX for Stories voice design. [GitHub: OpenMOSS/MOSS-TTS](https://github.com/OpenMOSS/MOSS-TTS)
- **VoxCPM 1.5** — Tokenizer-free continuous diffusion + autoregressive. No discrete token artifacts. Context-aware prosody/emotion, real-time streaming, LoRA fine-tuning. Trained on 1.8M+ hours. [GitHub: OpenBMB/VoxCPM](https://github.com/OpenBMB/VoxCPM)
- **Pocket TTS** — 100M param CPU-first model from Kyutai Labs (Moshi team). Runs >1× realtime without GPU. Broadens hardware support significantly. [GitHub: kyutai-labs/pocket-tts](https://github.com/kyutai-labs/pocket-tts)
- **Watch list:** MioTTS-2.6B (fast LLM-based EN/JP, vLLM compatible), Oolel-Voices (Soynade Research, expressive modular control)
- **Skipped:** Fish Audio S2 — restrictive research license (commercial use requires approval), despite strong features
1. **Model type concept** — A `model_type` field (e.g. `qwen`, `luxtts`, `chatterbox`) alongside `model_size`. The `GenerationRequest` schema, frontend form, and all model config dicts need updating.
### Adding a New Engine (Now Straightforward)
2. **Multiple backend instances** — The singleton `get_tts_backend()` needs to become a registry. Different models have different voice prompt formats, different inference APIs, different sample rates.
With the multi-engine architecture shipped, adding a new TTS engine requires:
3. **Voice prompt format abstraction** — Qwen uses `torch.save()`-serialized tensors. LuxTTS uses `encode_prompt()` returning its own format. Chatterbox uses audio-path-based cloning. The cache system (`backend/utils/cache.py`) needs to handle heterogeneous formats.
1. **Create `backend/backends/<engine>_backend.py`** — implement `TTSBackend` protocol (~200-300 lines)
2. **Register in `backend/backends/__init__.py`** — add to `TTS_ENGINES` dict + factory function
3. **Update `backend/models.py`** — add engine name to regex
4. **Update `backend/main.py`** — add engine cases in generate, stream, model-status, download, delete (5 dispatch points)
5. **Update frontend** — add to engine union type, form schema, model dropdown, language map (5-6 files)
4. **Sample rate normalization** — Qwen outputs 24 kHz. LuxTTS outputs 48 kHz. The Stories editor and audio pipeline need to handle mixed rates.
5. **Per-model capabilities** — Not all models support `instruct` (delivery instructions), not all support streaming, not all support the same languages. The UI needs to adapt.
### PR #194 as Precedent
The Hebrew/Chatterbox PR (#194) is the first attempt at multi-model. It takes a pragmatic approach: route by language (`he` → Chatterbox, else → Qwen). This works for one extra model but doesn't scale — what happens when you want Chatterbox for English too?
### PR #225 as Alternative Approach
The custom HuggingFace models PR (#225) takes a different angle: let users register arbitrary HF repos and attempt to load them through the existing Qwen backend. This is flexible but fragile — it assumes all models have the same API as Qwen3-TTS.
### PR #33 as Foundation
The external provider binaries PR (#33) has the most robust architecture for multi-model, since each provider is a separate process with its own dependencies. But it's complex, currently Qwen-only, and has been stale since early February.
Total effort: **~1 day** for a well-documented model with a PyPI package.
---
## Architectural Bottlenecks
### 1. Single Backend Singleton
### ~~1. Single Backend Singleton~~ — RESOLVED
**File:** `backend/backends/__init__.py:118-137`
The singleton TTS backend was replaced with a thread-safe per-engine registry in PR #254. Multiple engines can now be loaded simultaneously.
The entire TTS system runs through one global `_tts_backend` instance. You literally cannot have two models loaded. This is the #1 blocker for multi-model support.
### 2. `main.py` is 2100+ Lines
### 2. `main.py` is 1700+ Lines
All API routes, all model configs, all business logic in one file. Five separate dispatch points for each engine. Any new engine touches this file in 5 places. A model config registry pattern would reduce duplication.
All API routes, all model configs, all business logic in one file. Three separate hardcoded model config dicts that must stay in sync. Any multi-model change touches this file heavily.
### 3. Model Config is Scattered (Improved)
### 3. Model Config is Scattered
Model identifiers, HF repo IDs, display names, and download logic are duplicated across:
- `main.py` (3 separate dicts)
- `pytorch_backend.py` (HF repo map)
- `mlx_backend.py` (MLX repo map)
- `GenerationForm.tsx` (UI labels)
- `useGenerationForm.ts` (validation schema)
- `ModelManagement.tsx` (prefix filters)
There is no single source of truth for "what models does Voicebox support."
Model identifiers are still duplicated across `main.py` (3 dicts), backend files, frontend components, and the languages constant. However, the pattern is now consistent and well-understood. A centralized model registry would help but isn't blocking.
### 4. Voice Prompt Cache Assumes PyTorch Tensors
`backend/utils/cache.py` uses `torch.save()` / `torch.load()` for caching voice prompts. Models that don't use PyTorch tensors (LuxTTS, MLX-native models) can't use this cache.
`backend/utils/cache.py` uses `torch.save()` / `torch.load()`. LuxTTS and Chatterbox backends work around this by storing reference audio paths instead of tensors in their voice prompt dicts. Not ideal but functional.
### 5. Frontend Assumes Qwen Model Sizes
### 5. ~~Frontend Assumes Qwen Model Sizes~~ — RESOLVED
The generation form schema (`useGenerationForm.ts:17`) validates `model_size` as `'1.7B' | '0.6B'`. The model management UI filters by string prefix `qwen-tts`. Adding any model requires touching 3-4 frontend files.
The generation form now uses a flat model dropdown with engine-based routing. Per-engine language filtering is in place. Model size is only sent for Qwen.
---
## Recommended Priorities
### Tier 1 — Ship Now (Bug Fixes & Critical Improvements)
### Tier 1 — Ship Now (Low Risk)
These PRs fix real user pain with low risk. Can be reviewed and merged quickly.
| Priority | PR/Item | Impact | Effort |
|----------|---------|--------|--------|
| 1 | **#258** — Chatterbox Turbo + per-engine languages | Paralinguistic tags, proper language filtering | Review only |
| 2 | **#152** — Offline mode crash fix | Fixes #150, #151 | Low |
| 3 | **#99** — Chunked TTS + quality selector | Removes 500-char limit, addresses 5 issues | Medium |
| 4 | **#218** — Windows HF cache dir fix | Windows-specific pain | Low |
| 5 | **#178** — Generation error handling | Error UX | Low |
| 6 | **#230** — Docs fixes | Zero risk | None |
| 7 | **#133** — Network access toggle | Wires up existing code | Low |
| 8 | **#88** — CORS restriction | Security improvement | Low |
| 9 | **#214** — Tauri window close panic fix | Stability | Low |
| 10 | Triage GPU issues | Many may be resolved by CUDA swap (#252) | Low |
| 11 | Close superseded PRs | #194 (superseded by #257), #83 (outdated) | None |
| Priority | PR | Impact | Effort |
|----------|-----|--------|--------|
| 1 | **#97** — Pass language param to TTS | Fixes all non-English generation (18 language issues) | Low |
| 2 | **#238** — Download cancel/clear UI | Addresses 20 download-related issues | Low |
| 3 | **#152** — Offline mode crash fix | Fixes #150, #151 | Low |
| 4 | **#99** — Chunked TTS + quality selector | Removes 500-char limit, addresses 5 issues | Medium |
| 5 | **#218** — Windows HF cache dir fix | Windows-specific pain | Low |
| 6 | **#175, #178** — Profile validation + error handling | Small fixes | Low |
| 7 | **#250, #230** — Docs fixes | Zero risk | None |
| 8 | **#133** — Network access toggle | Wires up existing code | Low |
| 9 | **#88** — CORS restriction | Security improvement | Low |
| 10 | **#214** — Tauri window close panic fix | Stability | Low |
### Tier 2 — Next Release (v0.2.0)
### Tier 2 — Next Release (v0.2.0 Foundations)
These require more review but unlock major capabilities.
| Priority | Item | Impact | Effort | Dependencies |
|----------|------|--------|--------|-------------|
| 1 | **PR #33** — External provider binaries | Solves GPU distribution (19 issues), foundation for multi-model | Very High | Needs rebase, thorough review |
| 2 | **Multi-model abstraction layer** | Required before adding LuxTTS/Chatterbox/etc. | High | Informed by #33, #194, #225 |
| 3 | **PR #161** — Docker deployment | Server/headless users | Medium | Independent of #33 |
| 4 | **PR #194** — Hebrew + Chatterbox | First non-Qwen model, language expansion | High | Should align with multi-model abstraction |
| 5 | **PR #154** — Audiobook tab | Significant feature for long-form users | Medium | Benefits from #99 (chunking) |
| Priority | Item | Impact | Effort |
|----------|------|--------|--------|
| 1 | **#253** — 48kHz speech tokenizer | Quality improvement | Medium |
| 2 | **#161** — Docker deployment | Server/headless users | Medium |
| 3 | **#154** — Audiobook tab | Long-form users | Medium |
| 4 | **Model config registry** | Reduce 5-dispatch-point duplication in main.py | Medium |
| 5 | **#225** — Custom HuggingFace models | User-supplied models | High (needs rework for multi-engine) |
### Tier 3 — Future (v0.3.0+)
| Item | Notes |
|------|-------|
| LuxTTS integration | 48 kHz, low VRAM, but needs multi-model arch first |
| XTTS-v2 / Fish Speech | Multilingual powerhouses |
| OpenAI-compatible API (plan doc exists) | Low effort once API is stable |
| LoRA fine-tuning (PR #195) | Complex, depends on #194 |
| External/remote providers (plan doc exists) | Depends on provider architecture |
| GGUF support (#226) | Depends on model ecosystem maturity |
| Queue system (#234) | Batch generation |
| Real-time streaming synthesis | MLX-only currently, needs PyTorch path |
### Decision Point: Multi-Model Architecture
Before adding any new TTS model, a decision is needed on *how*:
**Option A — Provider Binary Split (PR #33 approach)**
Each model family is a separate executable/process. Most isolated, most flexible, but most complex. Solves the CUDA distribution problem simultaneously.
**Option B — In-Process Model Registry**
Keep everything in one process but replace the singleton with a registry that can instantiate multiple `TTSBackend` implementations. Simpler, but doesn't solve binary size / CUDA distribution.
**Option C — Hybrid (Recommended)**
Use Option B for lightweight models (LuxTTS, Kokoro — small, CPU-friendly) that can coexist in-process. Use Option A for heavy models (CUDA Qwen3-TTS, Fish Speech) that need their own process/dependencies. The provider architecture from PR #33 becomes the escape hatch for heavy models, while light models are built-in.
This matches how PR #194 already works (Chatterbox loaded in-process alongside Qwen) while keeping the door open for PR #33's provider split.
| Priority | Item | Notes |
|----------|------|-------|
| 1 | **HumeAI TADA** | Long-form reliability for Stories, synced transcripts. Addresses #234, #203, #191, #111, #69. Needs API vetting. |
| 2 | **Pocket TTS** (Kyutai) | CPU-first 100M model, broadens hardware support. Kyutai ships clean code. Needs API vetting. |
| 3 | **MOSS-TTS** | Text-to-voice design (no ref audio) is unique. Multi-speaker dialogue for Stories. Needs thorough API vetting. |
| 4 | **Kokoro-82M** | 82M params, CPU realtime, Apache 2.0. Easy win. |
| 5 | **Model config registry refactor** | Reduce 5-dispatch-point duplication in main.py — do before adding 3+ more engines |
| 6 | XTTS-v2 / Fish Speech / CosyVoice | Multi-engine arch is ready; just needs backend implementation |
| 7 | **VoxCPM 1.5** | Tokenizer-free streaming, interesting but uncertain integration surface |
| 8 | OpenAI-compatible API (plan doc exists) | Low effort once API is stable |
| 9 | LoRA fine-tuning (PR #195) | Complex, needs rework for multi-engine |
| 10 | External/remote providers | Depends on use case demand |
| 11 | GGUF support (#226) | Depends on model ecosystem maturity |
| 12 | Queue system (#234) | Batch generation |
| 13 | Streaming for non-MLX engines | Currently MLX-only |
---
@@ -409,24 +437,20 @@ This matches how PR #194 already works (Chatterbox loaded in-process alongside Q
| Branch | PR | Status | Notes |
|--------|-----|--------|-------|
| `external-provider-binaries` | #33 | Open, stale | Major architecture work |
| `feat/dual-server-binaries` | — | No PR | Related to provider split? |
| `feat/chatterbox-turbo` | #258 | Open | Chatterbox Turbo + per-engine languages |
| `feat/chatterbox` | #257 | **Merged** | Chatterbox Multilingual |
| `feat/luxtts` | #254 | **Merged** | LuxTTS + multi-engine arch |
| `external-provider-binaries` | #33 | Superseded by #252 | Original CUDA provider approach |
| `feat/dual-server-binaries` | — | No PR | Related to provider split |
| `fix-multi-sample` | — | No PR | Voice profile multi-sample fix |
| `fix-dl-notification-...` | — | No PR | Model download UX |
| `improvements` | — | No PR | Unknown scope |
| `stories` | — | No PR | Stories editor work? |
| `windows-server-shutdown` | — | No PR | Windows lifecycle |
| `model-dl-fix` | — | No PR | Model download fix |
| `channels` | — | No PR | Audio channels |
| `audio-export-entitlement-fix` | — | No PR | macOS entitlements |
| `better-docs` | — | No PR | Documentation |
---
## Quick Reference: API Endpoints
<details>
<summary>All current endpoints (v0.1.13)</summary>
<summary>All current endpoints</summary>
| Endpoint | Method | Purpose |
|----------|--------|---------|
@@ -437,20 +461,21 @@ This matches how PR #194 already works (Chatterbox loaded in-process alongside Q
| `/profiles/{id}/avatar` | POST, GET, DELETE | Avatar management |
| `/profiles/{id}/export` | GET | Export profile as ZIP |
| `/profiles/import` | POST | Import profile from ZIP |
| `/generate` | POST | Generate speech |
| `/generate/stream` | POST | Stream speech (SSE) |
| `/generate` | POST | Generate speech (engine param selects TTS backend) |
| `/generate/stream` | POST | Stream speech (MLX only) |
| `/history` | GET | List generation history |
| `/history/{id}` | GET, DELETE | Get/delete generation |
| `/history/{id}/export` | GET | Export generation ZIP |
| `/history/{id}/export-audio` | GET | Export audio only |
| `/transcribe` | POST | Transcribe audio (Whisper) |
| `/models/status` | GET | All model statuses |
| `/models/status` | GET | All model statuses (Qwen, LuxTTS, Chatterbox, Chatterbox Turbo, Whisper) |
| `/models/download` | POST | Trigger model download |
| `/models/download/cancel` | POST | Cancel/dismiss download |
| `/models/{name}` | DELETE | Delete downloaded model |
| `/models/load` | POST | Load model into memory |
| `/models/unload` | POST | Unload model |
| `/models/progress/{name}` | GET | SSE download progress |
| `/tasks/active` | GET | Active downloads/generations |
| `/tasks/active` | GET | Active downloads/generations (with inline progress) |
| `/stories` | POST, GET | Create/list stories |
| `/stories/{id}` | GET, PUT, DELETE | Story CRUD |
| `/stories/{id}/items` | POST, GET | Story items CRUD |
@@ -458,5 +483,8 @@ This matches how PR #194 already works (Chatterbox loaded in-process alongside Q
| `/channels` | POST, GET | Audio channel CRUD |
| `/channels/{id}` | PUT, DELETE | Channel update/delete |
| `/cache/clear` | POST | Clear voice prompt cache |
| `/server/cuda/status` | GET | CUDA binary availability |
| `/server/cuda/download` | POST | Download CUDA binary |
| `/server/cuda/switch` | POST | Switch to CUDA backend |
</details>

Some files were not shown because too many files have changed in this diff Show More