Compare commits

..
Author SHA1 Message Date
Jamie Pine 1622e08372 format 2026-02-03 06:45:35 -08:00
Jamie Pine dab3344fb5 Enhance StoryList component with dynamic bottom padding
- Introduced useMemo to calculate bottom padding based on the visibility of the StoryTrackEditor and FloatingGenerateBox.
- Updated imports to include useStory hook for fetching the current story data.
- Adjusted the StoryList layout to accommodate the new padding logic, improving UI consistency.
2026-02-03 06:36:09 -08:00
Jamie Pine 66cdb307e9 remove redundant file 2026-02-03 06:33:24 -08:00
Jamie Pine 5d66970fe9 Refactor FloatingGenerateBox and HistoryTable components for improved UI consistency
- Removed unnecessary class from the FloatingGenerateBox button for cleaner styling.
- Updated HistoryTable to remove redundant class from the delete button.
- Reorganized imports in dropdown-menu for better readability.
- Enhanced useAutoUpdater hook with improved dependency management and updated icon usage.
- Deleted obsolete useAutoUpdater.ts file to streamline codebase.
2026-02-03 06:25:27 -08:00
Jamie Pine 637e0b4585 Enhance ProviderSettings component for platform-specific functionality
- Added platform detection for macOS and Windows to improve user experience.
- Updated UI to conditionally disable options and provide clearer guidance based on installed providers.
- Refactored button states and labels for PyTorch CUDA and CPU to reflect availability and download status accurately.
2026-02-03 06:07:19 -08:00
Jamie Pine d21954358d Merge branch 'main' into external-provider-binaries 2026-02-03 05:44:44 -08:00
Jamie Pine 3f633f4a02 Update ProviderSettings component for improved clarity and user guidance
- Enhanced text descriptions for TTS providers, clarifying the functionality of PyTorch CUDA and Apple MLX.
- Removed outdated references to availability and added notes regarding model version differences.
- Improved UI structure by ensuring consistent labeling and disabling options based on system compatibility.
2026-02-03 05:40:37 -08:00
Jamie Pine 43241b85cf Update Dockerfile to install Python 3.12 venv and bootstrap pip
- Replaced python3-pip with python3.12-venv in the installation process.
- Added commands to ensure pip is upgraded and installed after setting Python 3.12 as the default.
- Streamlined the installation of PyTorch by removing redundant pip upgrade command.
2026-02-03 04:11:27 -08:00
Jamie Pine 793e392e56 Add DataFolders component to ServerSettings for managing system folder paths
- Introduced a new DataFolders component to display and manage paths for application data, models, and providers.
- Implemented a FolderRow component for individual folder display, including loading states and open folder functionality.
- Added API client methods and hooks to fetch system folder paths from the backend.
- Updated ServerTab to include the new DataFolders component, enhancing server settings management.
2026-02-03 00:32:18 -08:00
Jamie Pine 73d9b5d70e Refactor ProviderSettings component for improved readability and maintainability
- Removed unnecessary line breaks in the rendering of radio buttons and labels for a cleaner code structure.
- Added missing imports for numpy and scipy in build_binary.py to ensure proper functionality.
- Updated binary Assets.car file to reflect recent changes.
2026-02-02 20:08:07 -08:00
Jamie Pine af7e9814db Remove 'Active' badge for inactive PyTorch providers in ProviderSettings component 2026-02-02 05:23:15 -08:00
Jamie Pine 409ec2dbb1 Enhance ProviderSettings component with loading state and improved UI interactions
- Introduced a loading state to indicate when a provider is starting, enhancing user experience.
- Disabled radio buttons and action buttons during the loading state to prevent user interaction.
- Updated UI elements to reflect the loading state, including a spinner and appropriate cursor styles.
2026-02-02 05:16:17 -08:00
Jamie Pine 3ffbdeed89 Update Dockerfiles to prevent interactive prompts and include timezone data
- Added environment variables to Dockerfiles to set non-interactive mode and configure the timezone to UTC.
- Included installation of `tzdata` in both Dockerfiles to support timezone configuration during the build process.
2026-02-02 05:12:11 -08:00
Jamie Pine 61dabe7382 Enhance provider packaging and CUDA download functionality
- Added support for packaging provider archives in the release workflow, creating platform-specific zip and tar.gz files for distribution.
- Updated the `.gitignore` to exclude `.spec` files.
- Introduced a new `CudaDownloadSection` component to manage CUDA downloads, including progress tracking and error handling.
- Refactored provider download logic to handle archive extraction and cleanup after download.
- Improved subprocess output handling in the provider manager for better logging and error reporting.
2026-02-02 04:06:21 -08:00
Jamie Pine 732d35ca89 Update README and documentation for Linux support and Docker usage
- Revised README to include links for downloading the latest releases for macOS, Windows, and Linux.
- Added detailed instructions for running Voicebox with Docker, including Docker Compose usage.
- Updated installation documentation to reflect Linux availability and provide specific download options for AppImage and Deb packages.
- Enhanced clarity in Docker documentation regarding accessing the web UI.
2026-02-02 03:34:20 -08:00
Jamie Pine 53b1e8868c Enhance provider logging and error handling
- Added functionality to create log files for provider output, improving debugging on Windows.
- Updated error handling to read from log files instead of using subprocess output directly.
- Enhanced logging messages to include log file locations for easier troubleshooting.
2026-02-02 02:19:11 -08:00
Jamie Pine f090759d8f Add Docker support and update dependencies
- Introduced Docker support with CPU-only and GPU-enabled configurations via Dockerfiles and docker-compose files.
- Added a .dockerignore file to exclude unnecessary files from Docker images.
- Updated bun.lock and package.json to include new dependencies for icon handling.
- Enhanced README with Docker usage instructions and deployment options.
- Refactored components to utilize new icon libraries for improved UI consistency.
2026-02-02 02:19:05 -08:00
Jamie Pine 4c4b3e5463 Update dependencies and enhance project configuration
- Removed `lucide-react` version 0.454.0 and downgraded to version 0.316.0 in `bun.lock`.
- Added `@tailwindcss/vite` and `tailwindcss` as development dependencies in `package.json`.
- Updated Vite configuration to include Tailwind CSS plugin.
- Set the HTML document to use a dark theme by adding the `class="dark"` attribute to the `<html>` tag.
2026-02-02 00:36:24 -08:00
Jamie Pine e4f3647f9a Add CPU-only PyTorch installation for Linux in release workflow
- Included a conditional installation of CPU-only PyTorch packages for Ubuntu 22.04 to reduce unnecessary CUDA dependencies.
- Updated the release workflow to ensure compatibility with CPU-focused builds on Linux.
2026-02-02 00:17:57 -08:00
Jamie Pine 9b07a8480d Update Windows configuration in release workflow to use PyTorch backend
- Changed the backend setting for Windows from "none" to "pytorch" to ensure compatibility with bundled PyTorch CPU providers.
- Updated comments for clarity regarding the Windows setup.
2026-02-01 23:25:03 -08:00
Jamie Pine be30a0ac6b Update provider documentation and enhance build configurations
- Clarified the bundling of PyTorch CPU providers for Windows and macOS Intel builds in documentation.
- Improved handling of platform-specific dependencies in the build process, including asyncio support for PyInstaller.
- Updated backend logic to gracefully handle missing dependencies and provide clearer error messages.
- Enhanced progress management to ensure compatibility with PyInstaller's async handling.
- Removed unnecessary exclusions from the build scripts for PyTorch providers to streamline the build process.
2026-02-01 23:21:27 -08:00
Jamie Pine 595747c3d0 Refactor provider health status and update build configurations
- Updated the `ProviderSettings` component to log the current active provider.
- Changed the provider health status to use specific names for MLX and PyTorch backends.
- Removed unnecessary exclusions from the build scripts for both PyTorch CPU and CUDA providers.
- Ensured consistency in the `.spec` files for PyTorch providers by aligning exclusion lists.
2026-02-01 02:29:40 -08:00
Jamie Pine 580179eba3 Refactor TTS provider management and enhance documentation
- Renamed `bundled-mlx` to `apple-mlx` for clarity in provider types.
- Updated the ProviderSettings component to reflect the new provider naming.
- Improved logging for provider startup and error handling in the backend.
- Added scripts for building and installing PyTorch CPU and CUDA providers locally.
- Enhanced the documentation to include details on TTS provider architecture and development setup.
2026-02-01 01:23:20 -08:00
Jamie Pine 3b14f81741 Enhance release workflow and update provider settings
- Added macOS support for PyTorch CPU providers in the release workflow.
- Updated the ProviderSettings component to handle macOS-specific conditions and improve UI interactions.
- Refactored the radio group component styles for better accessibility and visual consistency.
- Improved provider management logic to ensure proper handling of available providers across different platforms.
2026-02-01 00:00:47 -08:00
Jamie Pine 6dd5bb2311 hugeicons 2026-01-31 22:18:01 -08:00
Jamie Pine dcbdf3e89b Bump version: 0.1.12 → 0.1.13 2026-01-31 20:18:32 -08:00
Jamie Pine 942064912a Update TTS provider methods and dependencies
- Renamed `load_model` to `load_model_async` in TTS provider classes for clarity and consistency.
- Added compatibility alias for `load_model` to maintain existing functionality.
- Enhanced `get_model_status` to handle both synchronous and asynchronous check functions.
- Updated version numbers in `bun.lock` and `Cargo.lock` to 0.1.12, reflecting recent changes.
2026-01-31 20:18:27 -08:00
Jamie Pine a52ff7d950 Add Linux audio capture module with unsupported functionality
- Introduced a new `linux.rs` module for audio capture, indicating that audio capture is not supported on Linux at this time.
- Updated `mod.rs` to include the Linux module conditionally based on the target OS.
2026-01-31 05:10:27 -08:00
Jamie Pine ab10c26ce4 Update release workflow to include libasound2-dev dependency for Ubuntu 22.04 2026-01-31 04:11:30 -08:00
Jamie Pine ce4269ffa5 Refactor build scripts and update release workflow
- Commented out the PyTorch CPU configuration in the release workflow for Ubuntu 22.04.
- Updated TTS provider documentation to clarify options for Windows and Linux users.
- Enhanced build scripts for both CPU and CUDA providers by excluding large unused modules to reduce binary size.
2026-01-31 03:45:41 -08:00
Jamie Pine ec0fb60197 Enhance release workflow and add radio group component
- Updated the release workflow to include a new configuration for the Ubuntu 22.04 platform without TTS bundled.
- Added the @radix-ui/react-radio-group dependency to package.json.
- Implemented a new RadioGroup component for better UI handling of radio inputs.
2026-01-31 03:40:27 -08:00
Jamie Pine ec9402c568 Update qwen-tts version in requirements for CPU and CUDA providers from 0.1.0 to 0.0.5 2026-01-31 03:11:34 -08:00
Jamie Pine d89521559a Update LocalProvider to manage model size dynamically and clean up build scripts
- Introduced a new attribute `_current_model_size` in `LocalProvider` to store the current model size, allowing for dynamic configuration during generation.
- Updated the `generate` method to use the current model size instead of a hardcoded value.
- Modified the `load_model` method to track the requested model size.
- Removed platform-specific extension handling from the build scripts for both CPU and CUDA providers to streamline the build process.
2026-01-31 03:09:49 -08:00
Jamie Pine 80689ad8ce Implement TTS provider management system and update release workflow
- Added support for TTS providers in the backend, including endpoints for listing, starting, stopping, and downloading providers.
- Enhanced the release workflow to build and upload TTS provider binaries for both Windows and Linux platforms.
- Updated the architecture documentation to reflect the new provider system and its benefits for modularity and user experience.
- Introduced a new `ProviderSettings` component in the frontend for managing provider configurations.
2026-01-31 03:05:50 -08:00
367 changed files with 20975 additions and 24756 deletions
+1 -1
View File
@@ -1,5 +1,5 @@
[bumpversion]
current_version = 0.2.3
current_version = 0.1.13
commit = True
tag = True
tag_name = v{new_version}
+41 -32
View File
@@ -1,46 +1,55 @@
# Version control
.git
.github
.gitignore
# Desktop-only (not needed in web container)
tauri/
landing/
docs/
mlx-test/
scripts/
# Dependencies & build artifacts (rebuilt in Docker)
# Dependencies
node_modules/
__pycache__/
*.pyc
*.pyo
*.egg-info/
dist/
*.pyd
.Python
env/
venv/
# Build outputs
build/
*.spec
*.egg-info/
.eggs/
target/
# Data (will be bind-mounted)
data/
backend/data/
# Keep web/dist for the Docker image
!web/dist
# IDE & OS
# Development
.git/
.github/
.vscode/
.idea/
*.swp
*.swo
*~
# Data and logs
data/
*.log
*.sqlite
*.db
# OS
.DS_Store
Thumbs.db
# Config files not needed in container
biome.json
.biomeignore
.bumpversion.cfg
.npmrc
Makefile
CHANGELOG.md
CONTRIBUTING.md
SECURITY.md
LICENSE
README.md
backend/README.md
# Documentation
docs/
landing/
mlx-test/
# Test files
*.test.ts
*.test.tsx
*.spec.ts
*.spec.tsx
# Keep these out
.env
.env.local
*.pem
*.key
credentials.json
-63
View File
@@ -1,63 +0,0 @@
name: Build Windows
on:
workflow_dispatch:
jobs:
build-windows:
permissions:
contents: write
runs-on: windows-latest
steps:
- uses: actions/checkout@v4
- name: Setup Python
uses: actions/setup-python@v5
with:
python-version: "3.12"
cache: "pip"
- name: Install Python dependencies
run: |
python -m pip install --upgrade pip
pip install pyinstaller
pip install -r backend/requirements.txt
- name: Build Python server
shell: bash
run: |
cd backend
python build_binary.py
PLATFORM=$(rustc --print host-tuple)
mkdir -p ../tauri/src-tauri/binaries
cp dist/voicebox-server.exe ../tauri/src-tauri/binaries/voicebox-server-${PLATFORM}.exe
echo "Built voicebox-server-${PLATFORM}.exe"
- name: Setup Bun
uses: oven-sh/setup-bun@v2
- name: Install Rust stable
uses: dtolnay/rust-toolchain@stable
- name: Rust cache
uses: swatinem/rust-cache@v2
with:
workspaces: "./tauri/src-tauri -> target"
- name: Install dependencies
run: bun install
- uses: tauri-apps/tauri-action@v0
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
projectPath: tauri
tagName: v__VERSION__
releaseName: "voicebox v__VERSION__ (test build)"
releaseBody: "Test build for audio export fix"
releaseDraft: true
prerelease: true
args: ""
includeUpdaterJson: false
+225 -59
View File
@@ -6,7 +6,151 @@ on:
tags:
- "v*"
env:
PROVIDER_VERSION: "1.0.0"
jobs:
# ============================================
# Build TTS Providers (uploaded to R2, not GitHub)
# ============================================
build-providers:
runs-on: ${{ matrix.platform }}
strategy:
fail-fast: false
matrix:
include:
# PyTorch CPU provider (Windows)
- platform: "windows-latest"
provider: "pytorch-cpu"
python-version: "3.12"
# PyTorch CUDA provider (Windows) - large binary, uploaded to R2
- platform: "windows-latest"
provider: "pytorch-cuda"
python-version: "3.12"
# PyTorch CPU provider (Linux)
- platform: "ubuntu-22.04"
provider: "pytorch-cpu"
python-version: "3.12"
# PyTorch CUDA provider (Linux) - large binary, uploaded to R2
- platform: "ubuntu-22.04"
provider: "pytorch-cuda"
python-version: "3.12"
# PyTorch CPU provider (macOS Apple Silicon)
- platform: "macos-latest"
provider: "pytorch-cpu"
python-version: "3.12"
# PyTorch CPU provider (macOS Intel)
- platform: "macos-15-intel"
provider: "pytorch-cpu"
python-version: "3.12"
steps:
- uses: actions/checkout@v4
- name: Install dependencies (ubuntu only)
if: matrix.platform == 'ubuntu-22.04'
run: |
sudo apt-get update
sudo apt-get install -y llvm-dev
- name: Setup Python
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
cache: "pip"
- name: Install CPU-only torch (Linux)
if: matrix.provider == 'pytorch-cpu' && matrix.platform == 'ubuntu-22.04'
run: |
python -m pip install --upgrade pip
pip install pyinstaller
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cpu
pip install -r providers/pytorch-cpu/requirements.txt
pip install -r backend/requirements.txt
- name: Install Python dependencies (CPU - non-Linux)
if: matrix.provider == 'pytorch-cpu' && matrix.platform != 'ubuntu-22.04'
run: |
python -m pip install --upgrade pip
pip install pyinstaller
pip install -r providers/pytorch-cpu/requirements.txt
pip install -r backend/requirements.txt
- name: Install Python dependencies (CUDA)
if: matrix.provider == 'pytorch-cuda'
run: |
python -m pip install --upgrade pip
pip install pyinstaller
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121
pip install -r providers/pytorch-cuda/requirements.txt
pip install -r backend/requirements.txt
- name: Build provider binary
shell: bash
run: |
cd providers/${{ matrix.provider }}
python build.py
- name: Package provider for distribution
shell: bash
run: |
cd providers/${{ matrix.provider }}/dist
# Add platform suffix for archive name
if [ "${{ matrix.platform }}" == "windows-latest" ]; then
ARCHIVE_NAME="tts-provider-${{ matrix.provider }}-windows.zip"
# On Windows, zip the directory
powershell Compress-Archive -Path "tts-provider-${{ matrix.provider }}/*" -DestinationPath "$ARCHIVE_NAME"
elif [ "${{ matrix.platform }}" == "macos-latest" ]; then
ARCHIVE_NAME="tts-provider-${{ matrix.provider }}-macos-arm64.tar.gz"
tar -czf "$ARCHIVE_NAME" tts-provider-${{ matrix.provider }}/
elif [ "${{ matrix.platform }}" == "macos-15-intel" ]; then
ARCHIVE_NAME="tts-provider-${{ matrix.provider }}-macos-x64.tar.gz"
tar -czf "$ARCHIVE_NAME" tts-provider-${{ matrix.provider }}/
else
ARCHIVE_NAME="tts-provider-${{ matrix.provider }}-linux.tar.gz"
tar -czf "$ARCHIVE_NAME" tts-provider-${{ matrix.provider }}/
fi
echo "Created archive: $ARCHIVE_NAME"
ls -lh "$ARCHIVE_NAME"
- name: Upload provider to R2
shell: bash
env:
R2_ACCESS_KEY_ID: ${{ secrets.R2_ACCESS_KEY_ID }}
R2_SECRET_ACCESS_KEY: ${{ secrets.R2_SECRET_ACCESS_KEY }}
R2_ENDPOINT: ${{ secrets.R2_ENDPOINT }}
run: |
# Install AWS CLI (compatible with R2)
pip install awscli
# Configure AWS CLI for R2
aws configure set aws_access_key_id $R2_ACCESS_KEY_ID
aws configure set aws_secret_access_key $R2_SECRET_ACCESS_KEY
aws configure set region auto
# Determine archive name based on platform
if [ "${{ matrix.platform }}" == "windows-latest" ]; then
ARCHIVE_NAME="tts-provider-${{ matrix.provider }}-windows.zip"
elif [ "${{ matrix.platform }}" == "macos-latest" ]; then
ARCHIVE_NAME="tts-provider-${{ matrix.provider }}-macos-arm64.tar.gz"
elif [ "${{ matrix.platform }}" == "macos-15-intel" ]; then
ARCHIVE_NAME="tts-provider-${{ matrix.provider }}-macos-x64.tar.gz"
else
ARCHIVE_NAME="tts-provider-${{ matrix.provider }}-linux.tar.gz"
fi
# Upload to R2 (bucket: voicebox)
aws s3 cp "providers/${{ matrix.provider }}/dist/$ARCHIVE_NAME" \
"s3://voicebox/providers/v${{ env.PROVIDER_VERSION }}/$ARCHIVE_NAME" \
--endpoint-url "$R2_ENDPOINT"
echo "Uploaded $ARCHIVE_NAME to R2"
# ============================================
# Build Main App (without bundled TTS on Win/Linux)
# ============================================
release:
permissions:
contents: write
@@ -14,14 +158,22 @@ jobs:
fail-fast: false
matrix:
include:
# macOS Apple Silicon - MLX bundled (works out of the box)
- platform: "macos-latest"
args: "--target aarch64-apple-darwin"
python-version: "3.12"
backend: "mlx"
# macOS Intel - PyTorch bundled (smaller user base, keep simple)
- platform: "macos-15-intel"
args: "--target x86_64-apple-darwin"
python-version: "3.12"
backend: "pytorch"
# Linux - No TTS bundled, providers downloaded separately
- platform: "ubuntu-22.04"
args: ""
python-version: "3.12"
backend: "none"
# Windows - PyTorch CPU bundled (works out of the box)
- platform: "windows-latest"
args: ""
python-version: "3.12"
@@ -33,7 +185,7 @@ jobs:
- uses: actions/checkout@v4
- name: Install dependencies (ubuntu only)
if: contains(matrix.platform, 'ubuntu') || contains(matrix.platform, 'namespace')
if: matrix.platform == 'ubuntu-22.04'
run: |
sudo apt-get update
sudo apt-get install -y libwebkit2gtk-4.1-dev libappindicator3-dev librsvg2-dev patchelf llvm-dev libasound2-dev
@@ -51,17 +203,21 @@ jobs:
python-version: ${{ matrix.python-version }}
cache: "pip"
- name: Install CPU-only PyTorch (Linux)
if: contains(matrix.platform, 'ubuntu') || contains(matrix.platform, 'namespace')
run: |
pip install torch torchaudio --index-url https://download.pytorch.org/whl/cpu
- name: Install Python dependencies
- name: Install Python dependencies (with TTS)
if: matrix.backend != 'none'
run: |
python -m pip install --upgrade pip
pip install pyinstaller
pip install -r backend/requirements.txt
pip install --no-deps chatterbox-tts
- name: Install Python dependencies (without TTS)
if: matrix.backend == 'none'
run: |
python -m pip install --upgrade pip
pip install pyinstaller
# Install base requirements without PyTorch/Qwen-TTS
pip install fastapi uvicorn sqlalchemy librosa soundfile numpy httpx
pip install huggingface_hub # For Whisper downloads
- name: Install MLX dependencies (Apple Silicon only)
if: matrix.backend == 'mlx'
@@ -123,7 +279,7 @@ jobs:
p12-file-base64: ${{ secrets.APPLE_CERTIFICATE }}
p12-password: ${{ secrets.APPLE_CERTIFICATE_PASSWORD }}
- uses: tauri-apps/tauri-action@v0.6
- uses: tauri-apps/tauri-action@v0
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
TAURI_SIGNING_PRIVATE_KEY: ${{ secrets.TAURI_SIGNING_PRIVATE_KEY }}
@@ -144,10 +300,15 @@ jobs:
See the assets below to download and install this version.
### Installation
- **macOS (Apple Silicon)**: Download the `aarch64.dmg` file - uses MLX for fast native inference
- **macOS (Apple Silicon)**: Download the `aarch64.dmg` file - uses MLX for fast native inference (works out of the box)
- **macOS (Intel)**: Download the `x64.dmg` file - uses PyTorch
- **Windows**: Download the `.msi` installer
- **Linux**: Compile from source (see README)
- **Windows**: Download the `.msi` installer - requires downloading a TTS provider on first use
- **Linux**: Download the `.AppImage` or `.deb` package - requires downloading a TTS provider on first use
### TTS Providers
Windows and Linux users will be prompted to download a TTS provider on first launch:
- **Windows**: PyTorch CPU (~300MB) or PyTorch CUDA (~2.4GB for NVIDIA GPUs)
- **Linux**: PyTorch CUDA (~2.4GB) - requires NVIDIA GPU
The app includes automatic updates - future updates will be installed automatically.
releaseDraft: true
@@ -155,63 +316,68 @@ jobs:
args: ${{ matrix.args }}
includeUpdaterJson: true
build-cuda-windows:
runs-on: windows-latest
# ============================================
# Build and Push Docker Images
# ============================================
docker:
runs-on: ubuntu-latest
permissions:
contents: write
contents: read
packages: write
steps:
- uses: actions/checkout@v4
- name: Setup Python
uses: actions/setup-python@v5
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Login to GitHub Container Registry
uses: docker/login-action@v3
with:
python-version: "3.12"
cache: "pip"
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Install Python dependencies
- name: Setup Bun
uses: oven-sh/setup-bun@v2
- name: Install dependencies and build web UI
run: |
python -m pip install --upgrade pip
pip install pyinstaller
pip install -r backend/requirements.txt
pip install --no-deps chatterbox-tts
bun install
cd web
bun run build
- name: Install PyTorch with CUDA 12.1
- name: Extract version from tag
id: version
run: |
pip install torch --index-url https://download.pytorch.org/whl/cu121 --force-reinstall --no-deps
pip install torchaudio --index-url https://download.pytorch.org/whl/cu121
if [[ $GITHUB_REF == refs/tags/v* ]]; then
VERSION=${GITHUB_REF#refs/tags/v}
else
VERSION="dev"
fi
echo "version=$VERSION" >> $GITHUB_OUTPUT
- name: Verify CUDA support in torch
run: |
python -c "import torch; print(f'CUDA available in build: {torch.cuda.is_available()}'); print(f'CUDA version: {torch.version.cuda}')"
- name: Build CUDA server binary
shell: bash
working-directory: backend
run: python build_binary.py --cuda
- name: Split binary for GitHub Releases
shell: bash
run: |
python scripts/split_binary.py \
backend/dist/voicebox-server-cuda.exe \
--output release-assets/
- name: Upload split parts to GitHub Release
if: startsWith(github.ref, 'refs/tags/')
uses: softprops/action-gh-release@v1
- name: Build and push CPU image
uses: docker/build-push-action@v5
with:
files: |
release-assets/voicebox-server-cuda.part*.exe
release-assets/voicebox-server-cuda.sha256
release-assets/voicebox-server-cuda.manifest
draft: true
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
context: .
file: ./Dockerfile
platforms: linux/amd64,linux/arm64
push: true
tags: |
ghcr.io/jamiepine/voicebox:latest
ghcr.io/jamiepine/voicebox:${{ steps.version.outputs.version }}
cache-from: type=gha
cache-to: type=gha,mode=max
- name: Upload binary as workflow artifact
uses: actions/upload-artifact@v4
- name: Build and push CUDA image
uses: docker/build-push-action@v5
with:
name: voicebox-server-cuda-windows
path: backend/dist/voicebox-server-cuda.exe
retention-days: 7
context: .
file: ./Dockerfile.cuda
platforms: linux/amd64
push: true
tags: |
ghcr.io/jamiepine/voicebox:latest-cuda
ghcr.io/jamiepine/voicebox:${{ steps.version.outputs.version }}-cuda
cache-from: type=gha
cache-to: type=gha,mode=max
+5 -2
View File
@@ -15,6 +15,7 @@ dist/
build/
*.egg-info/
*.egg
*.spec
target/
*.app
*.dmg
@@ -35,7 +36,10 @@ target/
Thumbs.db
# Data (user-generated)
data/
data/profiles/*
data/generations/*
data/projects/*
data/voicebox.db
!data/.gitkeep
# Logs
@@ -49,7 +53,6 @@ logs/
# Generated files
app/openapi.json
tauri/src-tauri/binaries/*
tauri/src-tauri/gen/Assets.car
# Temporary
tmp/
+6 -17
View File
@@ -5,14 +5,6 @@ All notable changes to Voicebox will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [Unreleased]
### Fixed
- **Profile Name Validation** - Added proper validation to prevent duplicate profile names ([#134](https://github.com/jamiepine/voicebox/issues/134))
- Users now receive clear error messages when attempting to create or update profiles with duplicate names
- Improved error handling in create and update profile API endpoints
- Added comprehensive test suite for duplicate name validation
## [0.1.0] - 2026-01-25
### Added
@@ -63,19 +55,16 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Fixed
- Audio export failing when Tauri save dialog returns object instead of string path
- OpenAPI client generator script now documents the local backend port and avoids an unused loop variable warning
### Added
- **justfile** - Comprehensive development workflow automation with commands for setup, development, building, testing, and code quality checks
- Cross-platform support (macOS, Linux, Windows)
- Python version detection and compatibility warnings
- Self-documenting help system with `just --list`
- **Makefile** - Comprehensive development workflow automation with commands for setup, development, building, testing, and code quality checks
- Includes Python version detection and compatibility warnings
- Self-documenting help system with `make help`
- Colored output for better readability
- Supports parallel development server execution
### Changed
- **README** - Updated Quick Start with justfile-based setup instructions
### Removed
- **Makefile** - Replaced by justfile (cross-platform, simpler syntax)
- **README** - Added Makefile reference and updated Quick Start with Makefile-based setup instructions alongside manual setup
---
+230 -47
View File
@@ -14,129 +14,308 @@ Thank you for your interest in contributing to Voicebox! This document provides
### Prerequisites
- **[Bun](https://bun.sh)** - Fast JavaScript runtime and package manager
```bash
curl -fsSL https://bun.sh/install | bash
```
- **[Python 3.11+](https://python.org)** - For backend development
```bash
python --version # Should be 3.11 or higher
```
- **[Rust](https://rustup.rs)** - For Tauri desktop app (installed automatically by Tauri CLI)
```bash
rustc --version # Check if installed
```
- **[Tauri Prerequisites](https://v2.tauri.app/start/prerequisites)** - Tauri-specific system dependencies (varies by OS).
- **Git** - Version control
### Development Setup
Install [just](https://github.com/casey/just) (`brew install just`, `cargo install just`, or `winget install Casey.Just`), then:
**Using the Makefile (recommended for macOS/Linux):** Run `make setup` to install all dependencies, then `make dev` to start development servers. See `make help` for all available commands.
```bash
git clone https://github.com/YOUR_USERNAME/voicebox.git
cd voicebox
**Manual setup (required for Windows):**
just setup # creates venv, installs Python + JS deps
just dev # starts backend + desktop app
```
1. **Fork and clone the repository**
`just setup` handles everything automatically, including:
- Creating a Python virtual environment
- Installing Python dependencies (with CUDA PyTorch on Windows if an NVIDIA GPU is detected)
- Installing MLX dependencies on Apple Silicon
- Installing JavaScript dependencies
```bash
git clone https://github.com/YOUR_USERNAME/voicebox.git
cd voicebox
```
`just dev` starts the backend and desktop app together. If a backend is already running (e.g. from `just dev-backend` in another terminal), it detects it and only starts the frontend.
2. **Install JavaScript dependencies**
Other useful commands:
```bash
bun install
```
```bash
just dev-web # backend + web app (no Tauri/Rust build)
just dev-backend # backend only
just dev-frontend # Tauri app only (backend must be running)
just kill # stop all dev processes
just clean-all # nuke everything and start fresh
just --list # see all available commands
```
This installs dependencies for:
> **Note:** In dev mode, the app connects to a manually-started Python server.
> The bundled server binary is only used in production builds.
- `app/` - Shared React frontend
- `tauri/` - Tauri desktop wrapper
- `web/` - Web deployment wrapper
#### Windows Notes
3. **Set up Python backend**
The justfile works natively on Windows via PowerShell. No WSL or Git Bash required. On Windows with an NVIDIA GPU, `just setup` automatically installs CUDA-enabled PyTorch for GPU acceleration.
```bash
cd backend
# Create virtual environment
python -m venv venv
# Activate virtual environment
source venv/bin/activate # On macOS/Linux
# or
venv\Scripts\activate # On Windows
# Install Python dependencies
pip install -r requirements.txt
# Install MLX dependencies (Apple Silicon only - for faster inference)
# On Apple Silicon, this enables native Metal acceleration
if [[ $(uname -m) == "arm64" ]]; then
pip install -r requirements-mlx.txt
fi
# Install Qwen3-TTS (required for voice synthesis)
pip install git+https://github.com/QwenLM/Qwen3-TTS.git
```
4. **Start development servers**
Development requires two terminals: one for the Python backend, one for the Tauri app.
**Terminal 1: Backend server** (start this first)
```bash
cd backend
source venv/bin/activate # Activate venv if not already active
bun run dev:server
# Or manually: uvicorn main:app --reload --port 17493
```
Backend will be available at `http://localhost:17493`
**Terminal 2: Desktop app**
```bash
bun run dev
```
This will:
- Create a placeholder sidecar binary (for Tauri compilation)
- Start Vite dev server on port 5173
- Launch Tauri window pointing to localhost:5173
- Connect to the Python server you started in Terminal 1
- Enable hot reload
> **Note:** In dev mode, the app connects to your manually-started Python server.
> The bundled server binary is only used in production builds.
**Optional: Web app**
```bash
bun run dev:web
```
Web app will be available at `http://localhost:5174`
### Model Downloads
Models are automatically downloaded from HuggingFace Hub on first use:
- **Whisper** (transcription): Auto-downloads on first transcription
- **Qwen3-TTS** (voice cloning): Auto-downloads on first generation (~2-4GB)
First-time usage will be slower due to model downloads, but subsequent runs will use cached models.
### TTS Provider Development
Voicebox uses a modular provider system to support different inference backends. Understanding this architecture is important when working on TTS features.
#### Provider Types
**Bundled Providers** — Included with the app binary:
- `apple-mlx` — Bundled with macOS Apple Silicon builds (`.dmg` for aarch64)
- Uses MLX for native Metal acceleration
- Configured in `.github/workflows/release.yml` with `backend: "mlx"`
**Hybrid Provider:**
- `pytorch-cpu` — Can be bundled OR downloaded depending on platform
- **Bundled** with Windows and macOS Intel builds
- macOS Intel: `.dmg` for x64 with `backend: "pytorch"`
- Windows: `.exe` installer with PyTorch CPU included
- **Downloaded** on first use for Linux builds (~300MB)
- Falls back to bundled version if external binary not found
**External-Only Providers:**
- `pytorch-cuda` — NVIDIA GPU-accelerated provider (~2.4GB)
- Windows/Linux only (no NVIDIA GPUs on macOS)
- Downloaded on demand, not bundled
- Optional for users with CUDA-capable GPUs
#### Provider Architecture
```
backend/providers/
├── __init__.py # ProviderManager - lifecycle management
├── base.py # TTSProvider protocol
├── bundled.py # BundledProvider - wraps built-in backends
├── local.py # LocalProvider - wraps external subprocess
├── installer.py # Download and install external providers
└── types.py # Provider type definitions
providers/
├── pytorch-cpu/ # External PyTorch CPU provider
│ ├── main.py # FastAPI server
│ ├── build.py # PyInstaller build script
│ └── build_and_install.py # Build and install locally
└── pytorch-cuda/ # External PyTorch CUDA provider
├── main.py
├── build.py
└── build_and_install.py
```
**How it works:**
1. **Bundled providers** run in-process within the main backend
2. **External providers** run as separate subprocess servers
3. **LocalProvider** communicates with external providers via HTTP
4. **ProviderManager** handles starting/stopping and health checks
#### Building Providers Locally
When developing provider features, you'll need to build and test external providers:
**Build a single provider:**
```bash
cd providers/pytorch-cpu
python build_and_install.py
```
**Build all providers:**
```bash
bun run build:providers
```
This script:
- Builds the provider binary with PyInstaller
- Detects your platform (Windows/macOS/Linux)
- Copies to the correct location:
- macOS: `~/Library/Application Support/voicebox/providers/`
- Windows: `%APPDATA%\voicebox\providers\`
- Linux: `~/.local/share/voicebox/providers/`
- Sets executable permissions on Unix
**Testing provider changes:**
1. Make changes to `providers/pytorch-cpu/main.py`
2. Run `bun run build:providers`
3. Restart the Voicebox app
4. Select the provider in Settings → TTS Provider
#### Provider Binary Distribution
For production releases, provider binaries are:
1. Built by GitHub Actions for all platforms
2. Uploaded to Cloudflare R2 at `downloads.voicebox.sh/providers/v{VERSION}/`
3. Downloaded on-demand by users based on their platform and GPU
See `.github/workflows/release.yml` for the build matrix.
### Building
**Build production app:**
**Build everything (recommended):**
```bash
just build # Build CPU server binary + Tauri installer
bun run build
```
On Windows, to build with CUDA support for local testing:
This automatically:
```bash
just build-local # Build CPU + CUDA server binaries + Tauri installer
```
This builds the CPU sidecar (bundled with the app), the CUDA binary (placed in `%APPDATA%/com.voicebox.app/backends/` for runtime GPU switching), and the installable Tauri app.
1. Builds the Python server binary (`./scripts/build-server.sh`)
2. Builds the Tauri desktop app (`cd tauri && bun run tauri build`)
Creates platform-specific installers (`.dmg`, `.msi`, `.AppImage`) in `tauri/src-tauri/target/release/bundle/`.
**Individual build targets:**
**Note:** The build process detects your platform and includes the appropriate backend (MLX for Apple Silicon, PyTorch for others).
**Build server binary only:**
```bash
just build-server # CPU server binary only
just build-server-cuda # CUDA server binary only (Windows)
just build-tauri # Tauri desktop app only
just build-web # Web app only
bun run build:server
# or
./scripts/build-server.sh
```
Creates platform-specific binary in `tauri/src-tauri/binaries/`
**Build provider binaries (for development):**
```bash
bun run build:providers
```
Builds all external provider binaries and installs them to the system provider directory. See [TTS Provider Development](#tts-provider-development) for details.
**Building with local Qwen3-TTS development version:**
If you're actively developing or modifying the Qwen3-TTS library, set the `QWEN_TTS_PATH` environment variable to point to your local clone:
```bash
export QWEN_TTS_PATH=~/path/to/your/Qwen3-TTS
just build-server
bun run build:server
```
This makes PyInstaller use your local qwen-tts version instead of the pip-installed package.
This makes PyInstaller use your local qwen-tts version instead of the pip-installed package. Useful when testing changes to the TTS library before they're published to PyPI or when using an editable install (`pip install -e`).
**Build web app:**
```bash
cd web
bun run build
```
Output in `web/dist/`
### Generate OpenAPI Client
After starting the backend server:
```bash
./scripts/generate-api.sh
```
This downloads the OpenAPI schema and generates the TypeScript client in `app/src/lib/api/`
### Convert Assets to Web Formats
To optimize images and videos for the web, run:
```bash
bun run convert:assets
```
This script:
- Converts PNG → WebP (better compression, same quality)
- Converts MOV → WebM (VP9 codec, smaller file size)
- Processes files in `landing/public/` and `docs/public/`
- **Deletes original files** after successful conversion
**Requirements:** Install `webp` and `ffmpeg`:
```bash
brew install webp ffmpeg
```
@@ -183,6 +362,7 @@ git push origin feature/your-feature-name
```
Then create a pull request on GitHub with:
- Clear description of changes
- Screenshots (for UI changes)
- Reference to related issues
@@ -328,21 +508,23 @@ Currently, testing is primarily manual. When adding tests:
Releases are managed by maintainers:
1. **Bump version using bumpversion:**
```bash
# Install bumpversion (if not already installed)
pip install bumpversion
# Bump patch version (0.1.0 -> 0.1.1)
bumpversion patch
# Or bump minor version (0.1.0 -> 0.2.0)
bumpversion minor
# Or bump major version (0.1.0 -> 1.0.0)
bumpversion major
```
This automatically:
- Updates version numbers in all files (`tauri.conf.json`, `Cargo.toml`, all `package.json` files, `backend/main.py`)
- Creates a git commit with the version bump
- Creates a git tag (e.g., `v0.1.1`, `v0.2.0`)
@@ -350,6 +532,7 @@ Releases are managed by maintainers:
2. **Update CHANGELOG.md** with release notes
3. **Push commits and tags:**
```bash
git push
git push --tags
@@ -365,7 +548,7 @@ See [docs/TROUBLESHOOTING.md](docs/TROUBLESHOOTING.md) for common issues and sol
- **Backend won't start:** Check Python version (3.11+), ensure venv is activated, install dependencies
- **Tauri build fails:** Ensure Rust is installed, clean build with `cd tauri/src-tauri && cargo clean`
- **OpenAPI client generation fails:** Ensure backend is running, check `curl http://localhost:17493/openapi.json`
- **OpenAPI client generation fails:** Ensure backend is running, check `curl http://localhost:8000/openapi.json`
## Questions?
+28 -64
View File
@@ -1,79 +1,43 @@
# ============================================================
# Voicebox — Local TTS Server with Web UI (CPU)
# 3-stage build: Frontend → Python deps → Runtime
# ============================================================
# Base Dockerfile for Voicebox (CPU-only)
# For GPU support, use Dockerfile.cuda
# === Stage 1: Build frontend ===
FROM oven/bun:1 AS frontend
FROM python:3.12-slim
WORKDIR /build
# Copy workspace config and frontend source
COPY package.json bun.lock ./
COPY app/ ./app/
COPY web/ ./web/
# Strip workspaces not needed for web build, and fix trailing comma
RUN sed -i '/"tauri"/d; /"landing"/d' package.json && \
sed -i -z 's/,\n ]/\n ]/' package.json
RUN bun install --no-save
# Build frontend (skip tsc — upstream has pre-existing type errors)
RUN cd web && bunx --bun vite build
# === Stage 2: Build Python dependencies ===
FROM python:3.11-slim AS backend-builder
WORKDIR /build
RUN apt-get update && apt-get install -y --no-install-recommends \
git \
build-essential \
&& rm -rf /var/lib/apt/lists/*
COPY backend/requirements.txt .
RUN pip install --no-cache-dir --prefix=/install -r requirements.txt
RUN pip install --no-cache-dir --prefix=/install \
git+https://github.com/QwenLM/Qwen3-TTS.git
# === Stage 3: Runtime ===
FROM python:3.11-slim
# Create non-root user for security
RUN groupadd -r voicebox && \
useradd -r -g voicebox -m -s /bin/bash voicebox
# Prevent interactive prompts during build
ENV DEBIAN_FRONTEND=noninteractive
ENV TZ=UTC
WORKDIR /app
# Install only runtime system dependencies
RUN apt-get update && apt-get install -y --no-install-recommends \
# Install system dependencies
RUN apt-get update && apt-get install -y \
ffmpeg \
curl \
tzdata \
&& rm -rf /var/lib/apt/lists/*
# Copy installed Python packages from builder stage
COPY --from=backend-builder /install /usr/local
# Copy backend
COPY backend/ /app/backend/
COPY providers/ /app/providers/
# Copy backend application code
COPY --chown=voicebox:voicebox backend/ /app/backend/
# Copy pre-built web UI
COPY web/dist/ /app/web/dist/
# Copy built frontend from frontend stage
COPY --from=frontend --chown=voicebox:voicebox /build/web/dist /app/frontend/
# Install Python dependencies (without PyTorch - will be downloaded via provider system)
RUN python -m pip install --upgrade pip && \
pip install --no-cache-dir \
fastapi uvicorn[standard] pydantic sqlalchemy alembic \
librosa soundfile numpy python-multipart Pillow \
huggingface_hub transformers accelerate
# Create data directories owned by non-root user
RUN mkdir -p /app/data/generations /app/data/profiles /app/data/cache \
&& chown -R voicebox:voicebox /app/data
# Create data directory for profiles/generations
RUN mkdir -p /app/data
# Switch to non-root user
USER voicebox
# Health check
HEALTHCHECK --interval=30s --timeout=10s --retries=3 --start-period=40s \
CMD curl -f http://localhost:8000/health || exit 1
# Expose the API port
EXPOSE 17493
EXPOSE 8000
# Health check — auto-restart if the server hangs
HEALTHCHECK --interval=30s --timeout=10s --retries=3 --start-period=60s \
CMD curl -f http://localhost:17493/health || exit 1
# Start the FastAPI server
CMD ["uvicorn", "backend.main:app", "--host", "0.0.0.0", "--port", "17493"]
# Run server with web UI
CMD ["uvicorn", "backend.main:app", "--host", "0.0.0.0", "--port", "8000"]
+58
View File
@@ -0,0 +1,58 @@
# Dockerfile for Voicebox with NVIDIA GPU support (CUDA)
FROM nvidia/cuda:12.1.1-runtime-ubuntu22.04
# Prevent interactive prompts during build
ENV DEBIAN_FRONTEND=noninteractive
ENV TZ=UTC
WORKDIR /app
# Install Python 3.12
RUN apt-get update && apt-get install -y \
software-properties-common \
&& add-apt-repository ppa:deadsnakes/ppa \
&& apt-get update && apt-get install -y \
python3.12 \
python3.12-dev \
python3.12-venv \
ffmpeg \
curl \
tzdata \
&& rm -rf /var/lib/apt/lists/*
# Set Python 3.12 as default and bootstrap pip
RUN update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.12 1 && \
update-alternatives --install /usr/bin/python python /usr/bin/python3.12 1 && \
python3.12 -m ensurepip --upgrade && \
python3.12 -m pip install --upgrade pip
# Copy backend
COPY backend/ /app/backend/
COPY providers/ /app/providers/
# Copy pre-built web UI
COPY web/dist/ /app/web/dist/
# Install PyTorch with CUDA support first
RUN pip install --no-cache-dir \
torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121
# Install remaining dependencies
RUN pip install --no-cache-dir \
fastapi uvicorn[standard] pydantic sqlalchemy alembic \
transformers accelerate huggingface_hub \
librosa soundfile numpy python-multipart Pillow \
qwen-tts
# Create data directory
RUN mkdir -p /app/data
# Health check
HEALTHCHECK --interval=30s --timeout=10s --retries=3 --start-period=40s \
CMD curl -f http://localhost:8000/health || exit 1
EXPOSE 8000
# Run server with web UI
CMD ["uvicorn", "backend.main:app", "--host", "0.0.0.0", "--port", "8000"]
+245
View File
@@ -0,0 +1,245 @@
# Voicebox Makefile
# Unix-only (macOS/Linux). Windows users should use WSL.
SHELL := /bin/bash
.DEFAULT_GOAL := help
# Directories
BACKEND_DIR := backend
TAURI_DIR := tauri
WEB_DIR := web
APP_DIR := app
# Python (prefer 3.12, fallback to 3.13, then python3)
PYTHON := $(shell command -v python3.12 2>/dev/null || command -v python3.13 2>/dev/null || echo python3)
VENV := $(CURDIR)/$(BACKEND_DIR)/venv
VENV_BIN := $(VENV)/bin
PIP := $(VENV_BIN)/pip
PYTHON_VENV := $(VENV_BIN)/python
# Colors for output
BLUE := \033[0;34m
GREEN := \033[0;32m
YELLOW := \033[0;33m
NC := \033[0m # No Color
.PHONY: help
help: ## Show this help message
@echo -e "$(BLUE)Voicebox$(NC) - Development Commands"
@echo ""
@grep -E '^[a-zA-Z_-]+:.*?## .*$$' $(MAKEFILE_LIST) | sort | \
awk 'BEGIN {FS = ":.*?## "}; {printf " $(GREEN)%-20s$(NC) %s\n", $$1, $$2}'
# =============================================================================
# SETUP
# =============================================================================
.PHONY: setup setup-js setup-python setup-rust
setup: setup-js setup-python ## Full project setup (all dependencies)
@echo -e "$(GREEN)✓ Setup complete!$(NC)"
@echo -e " Run $(YELLOW)make dev$(NC) to start development servers"
setup-js: ## Install JavaScript dependencies (bun)
@echo -e "$(BLUE)Installing JavaScript dependencies...$(NC)"
bun install
setup-python: $(VENV)/bin/activate ## Set up Python virtual environment and dependencies
@echo -e "$(BLUE)Installing Python dependencies...$(NC)"
$(PIP) install --upgrade pip
$(PIP) install -r $(BACKEND_DIR)/requirements.txt
@if [ "$$(uname -m)" = "arm64" ] && [ "$$(uname)" = "Darwin" ]; then \
echo -e "$(BLUE)Detected Apple Silicon - installing MLX dependencies...$(NC)"; \
$(PIP) install -r $(BACKEND_DIR)/requirements-mlx.txt; \
echo -e "$(GREEN)✓ MLX backend enabled (native Metal acceleration)$(NC)"; \
fi
$(PIP) install git+https://github.com/QwenLM/Qwen3-TTS.git
@echo -e "$(GREEN)✓ Python environment ready$(NC)"
$(VENV)/bin/activate:
@echo -e "$(BLUE)Creating Python virtual environment...$(NC)"
@PY_MINOR=$$($(PYTHON) -c "import sys; print(sys.version_info[1])"); \
if [ "$$PY_MINOR" -gt 13 ]; then \
echo -e "$(YELLOW)Warning: Python 3.$$PY_MINOR detected. ML packages may not be compatible.$(NC)"; \
echo -e "$(YELLOW)Recommended: Use Python 3.12 or 3.13 (brew install [email protected])$(NC)"; \
fi
$(PYTHON) -m venv $(VENV)
setup-rust: ## Install Rust toolchain (if not present)
@command -v rustc >/dev/null 2>&1 || curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
# =============================================================================
# DEVELOPMENT
# =============================================================================
.PHONY: dev dev-backend dev-frontend dev-web kill-dev
dev: ## Start backend + desktop app (parallel)
@echo -e "$(BLUE)Starting development servers...$(NC)"
@echo -e "$(YELLOW)Note: If Tauri fails, run 'make build-server' first or use separate terminals$(NC)"
@trap 'kill 0' EXIT; \
$(MAKE) dev-backend & \
sleep 2 && $(MAKE) dev-frontend & \
wait
dev-backend: ## Start FastAPI backend server
@echo -e "$(BLUE)Starting backend server on http://localhost:17493$(NC)"
$(VENV_BIN)/uvicorn backend.main:app --reload --port 17493
dev-frontend: ## Start Tauri desktop app
@echo -e "$(BLUE)Starting Tauri desktop app...$(NC)"
bun run dev
dev-web: ## Start backend + web app (parallel)
@echo -e "$(BLUE)Starting web development servers...$(NC)"
@trap 'kill 0' EXIT; \
$(MAKE) dev-backend & \
sleep 2 && cd $(WEB_DIR) && bun run dev & \
wait
kill-dev: ## Kill all development processes
@echo -e "$(YELLOW)Killing development processes...$(NC)"
-pkill -f "uvicorn main:app" 2>/dev/null || true
-pkill -f "vite" 2>/dev/null || true
@echo -e "$(GREEN)✓ Processes killed$(NC)"
# =============================================================================
# BUILD
# =============================================================================
.PHONY: build build-server build-tauri build-web
build: build-server build-tauri ## Build everything (server binary + desktop app)
@echo -e "$(GREEN)✓ Build complete!$(NC)"
build-server: ## Build Python server binary
@echo -e "$(BLUE)Building server binary...$(NC)"
PATH="$(VENV_BIN):$$PATH" ./scripts/build-server.sh
build-tauri: ## Build Tauri desktop app
@echo -e "$(BLUE)Building Tauri desktop app...$(NC)"
cd $(TAURI_DIR) && bun run tauri build
build-web: ## Build web app
@echo -e "$(BLUE)Building web app...$(NC)"
cd $(WEB_DIR) && bun run build
@echo -e "$(GREEN)✓ Web build output in $(WEB_DIR)/dist/$(NC)"
# =============================================================================
# DATABASE & API
# =============================================================================
.PHONY: db-init db-reset generate-api
db-init: $(VENV)/bin/activate ## Initialize SQLite database
@echo -e "$(BLUE)Initializing database...$(NC)"
cd $(BACKEND_DIR) && $(PYTHON_VENV) -c "from database import init_db; init_db()"
@echo -e "$(GREEN)✓ Database created at $(BACKEND_DIR)/data/voicebox.db$(NC)"
db-reset: ## Reset database (delete and reinitialize)
@echo -e "$(YELLOW)Resetting database...$(NC)"
rm -f $(BACKEND_DIR)/data/voicebox.db
$(MAKE) db-init
generate-api: ## Generate TypeScript API client from OpenAPI schema
@echo -e "$(BLUE)Generating API client...$(NC)"
@echo -e "$(YELLOW)Note: Backend must be running (make dev-backend)$(NC)"
./scripts/generate-api.sh
@echo -e "$(GREEN)✓ API client generated in $(APP_DIR)/src/lib/api/$(NC)"
# =============================================================================
# CODE QUALITY
# =============================================================================
.PHONY: lint format typecheck check
lint: ## Run linter (Biome)
@echo -e "$(BLUE)Linting...$(NC)"
bun run lint
format: ## Format code (Biome)
@echo -e "$(BLUE)Formatting...$(NC)"
bun run format
typecheck: ## Run TypeScript type checking
@echo -e "$(BLUE)Type checking...$(NC)"
bun run tsc --noEmit
check: ## Run all checks (Biome lint + format + type check)
@echo -e "$(BLUE)Running all checks...$(NC)"
bun run check
@echo -e "$(GREEN)✓ All checks passed$(NC)"
# =============================================================================
# TESTING
# =============================================================================
.PHONY: test test-backend test-frontend
test: test-backend test-frontend ## Run all tests
@echo -e "$(GREEN)✓ All tests passed$(NC)"
test-backend: ## Run Python backend tests (requires pytest)
@echo -e "$(BLUE)Running backend tests...$(NC)"
@if [ -f "$(VENV_BIN)/pytest" ]; then \
cd $(BACKEND_DIR) && $(VENV_BIN)/pytest -v; \
else \
echo -e "$(YELLOW)pytest not installed. Run: $(PIP) install pytest$(NC)"; \
exit 1; \
fi
test-frontend: ## Run frontend tests (requires test script in package.json)
@echo -e "$(BLUE)Running frontend tests...$(NC)"
@if bun run test --help >/dev/null 2>&1; then \
bun run test; \
else \
echo -e "$(YELLOW)No test script configured$(NC)"; \
exit 1; \
fi
# =============================================================================
# LOGS & DEBUGGING
# =============================================================================
.PHONY: logs docs
logs: ## Tail backend logs
@echo -e "$(BLUE)Tailing logs (Ctrl+C to stop)...$(NC)"
tail -f $(BACKEND_DIR)/logs/*.log 2>/dev/null || echo "No log files found"
docs: ## Open API documentation (backend must be running)
@echo -e "$(BLUE)Opening API docs...$(NC)"
open http://localhost:17493/docs 2>/dev/null || xdg-open http://localhost:17493/docs
# =============================================================================
# CLEAN
# =============================================================================
.PHONY: clean clean-python clean-build clean-all
clean: ## Clean build artifacts
@echo -e "$(BLUE)Cleaning build artifacts...$(NC)"
rm -rf $(TAURI_DIR)/src-tauri/target/release
rm -rf $(WEB_DIR)/dist
rm -rf $(APP_DIR)/dist
@echo -e "$(GREEN)✓ Build artifacts cleaned$(NC)"
clean-python: ## Clean Python cache and virtual environment
@echo -e "$(BLUE)Cleaning Python files...$(NC)"
rm -rf $(VENV)
find $(BACKEND_DIR) -type d -name "__pycache__" -exec rm -rf {} + 2>/dev/null || true
find $(BACKEND_DIR) -type f -name "*.pyc" -delete 2>/dev/null || true
@echo -e "$(GREEN)✓ Python environment cleaned$(NC)"
clean-build: ## Clean Rust/Tauri build cache
@echo -e "$(BLUE)Cleaning Rust build cache...$(NC)"
cd $(TAURI_DIR)/src-tauri && cargo clean
@echo -e "$(GREEN)✓ Rust cache cleaned$(NC)"
clean-all: clean clean-python clean-build ## Nuclear clean (everything)
@echo -e "$(BLUE)Cleaning node_modules...$(NC)"
rm -rf node_modules
rm -rf $(APP_DIR)/node_modules
rm -rf $(TAURI_DIR)/node_modules
rm -rf $(WEB_DIR)/node_modules
@echo -e "$(GREEN)✓ Full clean complete$(NC)"
-58
View File
@@ -1,58 +0,0 @@
# Voicebox Offline Mode Fix
## Problem
Voicebox crashes when generating speech if HuggingFace is unreachable, even when models are fully cached locally.
**Root Cause:**
- Voicebox downloads `mlx-community/Qwen3-TTS-12Hz-1.7B-Base-bf16` (MLX optimized version)
- But `mlx_audio.tts.load()` tries to fetch `config.json` from original repo `Qwen/Qwen3-TTS-12Hz-1.7B-Base`
- This network request fails → server crashes with `RemoteDisconnected`
**Related Issues:**
- Issue #150: "Internet connection required, even though models are downloaded?"
- Issue #151: "API Stability Issues: Model Loading Hangs and Server Crashes"
## Solution
Two-part fix:
### 1. Monkey-patch huggingface_hub (`backend/utils/hf_offline_patch.py`)
- Intercepts cache lookup functions
- Forces offline mode early (before mlx_audio imports)
- Adds debug logging for cache hits/misses
### 2. Symlink original repo to MLX version (`ensure_original_qwen_config_cached()`)
- When original `Qwen/Qwen3-TTS-12Hz-1.7B-Base` cache doesn't exist
- But MLX `mlx-community/Qwen3-TTS-12Hz-1.7B-Base-bf16` does exist
- Creates a symlink so cache lookups succeed
## Files Changed
- `backend/backends/mlx_backend.py` - Added patch imports at top
- `backend/utils/hf_offline_patch.py` - New patch module
## Testing
To test this fix:
1. Build Voicebox from source: `just build`
2. Disconnect from internet
3. Try generating speech
4. Should work without network requests
## Build Instructions
```bash
# Install dependencies
just setup
# Build the app
just build
# Or build just the server
just build-server
```
## Notes
- The patch is applied automatically when `mlx_backend.py` is imported
- Set `VOICEBOX_OFFLINE_PATCH=0` to disable the patch
- The symlink approach works because the config.json is compatible between versions
---
*Patch contributed by community*
+175 -140
View File
@@ -6,7 +6,7 @@
<p align="center">
<strong>The open-source voice synthesis studio.</strong><br/>
Clone voices. Generate speech. Apply effects. Build voice-powered apps.<br/>
Clone voices. Generate speech. Build voice-powered apps.<br/>
All running locally on your machine.
</p>
@@ -59,195 +59,209 @@
## What is Voicebox?
Voicebox is a **local-first voice cloning studio** — a free and open-source alternative to ElevenLabs. Clone voices from a few seconds of audio, generate speech in 23 languages across 4 TTS engines, apply post-processing effects, and compose multi-voice projects with a timeline editor.
Voicebox is a **local-first voice cloning studio** with DAW-like features for professional voice synthesis. Think of it as a **local, free and open-source alternative to ElevenLabs** — download models, clone voices, and generate speech entirely on your machine.
Unlike cloud services that lock your voice data behind subscriptions, Voicebox gives you:
- **Complete privacy** — models and voice data stay on your machine
- **4 TTS engines** — Qwen3-TTS, LuxTTS, Chatterbox Multilingual, and Chatterbox Turbo
- **23 languages** — from English to Arabic, Japanese, Hindi, Swahili, and more
- **Post-processing effects** — pitch shift, reverb, delay, chorus, compression, and filters
- **Expressive speech** — paralinguistic tags like `[laugh]`, `[sigh]`, `[gasp]` via Chatterbox Turbo
- **Unlimited length** — auto-chunking with crossfade for scripts, articles, and chapters
- **Stories editor** — multi-track timeline for conversations, podcasts, and narratives
- **API-first** — REST API for integrating voice synthesis into your own projects
- **Professional tools** — multi-track timeline editor, audio trimming, conversation mixing
- **Model flexibility** — currently powered by Qwen3-TTS, with support for XTTS, Bark, and other models coming soon
- **API-first** — use the desktop app or integrate voice synthesis into your own projects
- **Native performance** — built with Tauri (Rust), not Electron
- **Runs everywhere** — macOS (MLX/Metal), Windows (CUDA), Linux, AMD ROCm, Intel Arc, Docker
- **Super fast on Mac** — MLX backend with native Metal acceleration for 4-5x faster inference on Apple Silicon
Download a voice model, clone any voice from a few seconds of audio, and compose multi-voice projects with studio-grade editing tools. No Python install required, no cloud dependency, no limits.
---
## Download
| Platform | Download |
|----------|----------|
| macOS (Apple Silicon) | [Download DMG](https://voicebox.sh/download/mac-arm) |
| macOS (Intel) | [Download DMG](https://voicebox.sh/download/mac-intel) |
| Windows | [Download MSI](https://voicebox.sh/download/windows) |
| Docker | `docker compose up` |
### Desktop App
> **[View all binaries →](https://github.com/jamiepine/voicebox/releases/latest)**
| Platform | Download |
| --------------------- | --------------------------------------------------------------------------------------------------------------------------- |
| macOS (Apple Silicon) | [Download latest release](https://github.com/jamiepine/voicebox/releases/latest) |
| macOS (Intel) | [Download latest release](https://github.com/jamiepine/voicebox/releases/latest) |
| Windows (MSI) | [Download latest release](https://github.com/jamiepine/voicebox/releases/latest) |
| Windows (Setup) | [Download latest release](https://github.com/jamiepine/voicebox/releases/latest) |
| Linux (AppImage) | [Download latest release](https://github.com/jamiepine/voicebox/releases/latest) |
| Linux (Deb) | [Download latest release](https://github.com/jamiepine/voicebox/releases/latest) |
> **Linux** — Pre-built binaries are not yet available. See [voicebox.sh/linux-install](https://voicebox.sh/linux-install) for build-from-source instructions.
### Docker
Run Voicebox with the web UI in Docker - perfect for servers and headless deployments:
```bash
# CPU-only (supports amd64 and arm64)
docker run -p 8000:8000 -v voicebox-data:/app/data \
ghcr.io/jamiepine/voicebox:latest
# NVIDIA GPU (recommended for performance)
docker run --gpus all -p 8000:8000 -v voicebox-data:/app/data \
ghcr.io/jamiepine/voicebox:latest-cuda
```
Or use Docker Compose:
```bash
docker compose up -d
```
Open http://localhost:8000 to access the web UI.
See [Docker Deployment Guide](docs/overview/docker.mdx) for cloud deployments, GPU setup, and more.
---
## Features
### Multi-Engine Voice Cloning
### Voice Cloning with Qwen3-TTS
Four TTS engines with different strengths, switchable per-generation:
Powered by Alibaba's **Qwen3-TTS** — a breakthrough model that achieves near-perfect voice cloning from just a few seconds of audio.
| Engine | Languages | Strengths |
|--------|-----------|-----------|
| **Qwen3-TTS** (0.6B / 1.7B) | 10 | High-quality multilingual cloning, delivery instructions ("speak slowly", "whisper") |
| **LuxTTS** | English | Lightweight (~1GB VRAM), 48kHz output, 150x realtime on CPU |
| **Chatterbox Multilingual** | 23 | Broadest language coverage — Arabic, Danish, Finnish, Greek, Hebrew, Hindi, Malay, Norwegian, Polish, Swahili, Swedish, Turkish and more |
| **Chatterbox Turbo** | English | Fast 350M model with paralinguistic emotion/sound tags |
### Emotions & Paralinguistic Tags
Type `/` in the text input to insert expressive tags that the model synthesizes inline with speech (Chatterbox Turbo):
`[laugh]` `[chuckle]` `[gasp]` `[cough]` `[sigh]` `[groan]` `[sniff]` `[shush]` `[clear throat]`
### Post-Processing Effects
8 audio effects powered by Spotify's `pedalboard` library. Apply after generation, preview in real time, build reusable presets.
| Effect | Description |
|--------|-------------|
| Pitch Shift | Up or down by up to 12 semitones |
| Reverb | Configurable room size, damping, wet/dry mix |
| Delay | Echo with adjustable time, feedback, and mix |
| Chorus / Flanger | Modulated delay for metallic or lush textures |
| Compressor | Dynamic range compression |
| Gain | Volume adjustment (-40 to +40 dB) |
| High-Pass Filter | Remove low frequencies |
| Low-Pass Filter | Remove high frequencies |
Ships with 4 built-in presets (Robotic, Radio, Echo Chamber, Deep Voice) and supports custom presets. Effects can be assigned per-profile as defaults.
### Unlimited Generation Length
Text is automatically split at sentence boundaries and each chunk is generated independently, then crossfaded together. Works with all engines.
- Configurable auto-chunking limit (100–5,000 chars)
- Crossfade slider (0–200ms) for smooth transitions
- Max text length: 50,000 characters
- Smart splitting respects abbreviations, CJK punctuation, and `[tags]`
### Generation Versions
Every generation supports multiple versions with provenance tracking:
- **Original** — clean TTS output, always preserved
- **Effects versions** — apply different effects chains from any source version
- **Takes** — regenerate with a new seed for variation
- **Source tracking** — each version records its lineage
- **Favorites** — star generations for quick access
### Async Generation Queue
Generation is non-blocking. Submit and immediately start typing the next one.
- Serial execution queue prevents GPU contention
- Real-time SSE status streaming
- Failed generations can be retried
- Stale generations from crashes auto-recover on startup
- **Instant cloning** — Upload a sample, get a voice profile
- **High fidelity** — Natural prosody, emotion, and cadence
- **Multi-language** — English, Chinese, and more coming
- **Lightning fast on Mac** — MLX backend leverages Apple Silicon's Neural Engine for super fast generation
### Voice Profile Management
- Create profiles from audio files or record directly in-app
- Import/export profiles to share or back up
- Multi-sample support for higher quality cloning
- Per-profile default effects chains
- Organize with descriptions and language tags
- **Create profiles** from audio files or record directly in-app
- **Import/Export** profiles to share or backup
- **Multi-sample support** — combine multiple samples for higher quality cloning
- **Organize** with descriptions and language tags
### Speech Generation
- **Text-to-speech** with any cloned voice
- **Batch generation** for long-form content
- **Smart caching** — regenerate instantly with voice prompt caching
### Stories Editor
Multi-voice timeline editor for conversations, podcasts, and narratives.
Create multi-voice narratives, podcasts, and conversations with a timeline-based editor.
- Multi-track composition with drag-and-drop
- Inline audio trimming and splitting
- Auto-playback with synchronized playhead
- Version pinning per track clip
- **Multi-track composition** — arrange multiple voice tracks in a single project
- **Inline audio editing** — trim and split clips directly in the timeline
- **Auto-playback** — preview stories with synchronized playhead
- **Voice mixing** — build conversations with multiple participants
### Recording & Transcription
- In-app recording with waveform visualization
- System audio capture (macOS and Windows)
- Automatic transcription powered by Whisper (including Whisper Turbo)
- Export recordings in multiple formats
- **In-app recording** with waveform visualization
- **System audio capture** — record desktop audio on macOS and Windows
- **Automatic transcription** powered by Whisper
- **Export recordings** in multiple formats
### Model Management
### Generation History
- Per-model unload to free GPU memory without deleting downloads
- Custom models directory via `VOICEBOX_MODELS_DIR`
- Model folder migration with progress tracking
- Download cancel/clear UI
- **Full history** of all generated audio
- **Search & filter** by voice, text, or date
- **Re-generate** any past generation with one click
### GPU Support
### Flexible Deployment
| Platform | Backend | Notes |
|----------|---------|-------|
| macOS (Apple Silicon) | MLX (Metal) | 4-5x faster via Neural Engine |
| Windows / Linux (NVIDIA) | PyTorch (CUDA) | Auto-downloads CUDA binary from within the app |
| Linux (AMD) | PyTorch (ROCm) | Auto-configures HSA_OVERRIDE_GFX_VERSION |
| Windows (any GPU) | DirectML | Universal Windows GPU support |
| Intel Arc | IPEX/XPU | Intel discrete GPU acceleration |
| Any | CPU | Works everywhere, just slower |
- **Desktop app** — Native apps for macOS, Windows, and Linux
- **Docker** — Deploy to servers with the web UI included
- **Remote mode** — Connect desktop app to a remote GPU server
- **Cloud ready** — Deploy to AWS, GCP, DigitalOcean, or any cloud provider
---
## API
Voicebox exposes a full REST API for integrating voice synthesis into your own apps.
Voicebox exposes a full REST API, so you can integrate voice synthesis into your own apps.
```bash
# Generate speech
curl -X POST http://localhost:17493/generate \
curl -X POST http://localhost:8000/generate \
-H "Content-Type: application/json" \
-d '{"text": "Hello world", "profile_id": "abc123", "language": "en"}'
# List voice profiles
curl http://localhost:17493/profiles
curl http://localhost:8000/profiles
# Create a profile
curl -X POST http://localhost:17493/profiles \
curl -X POST http://localhost:8000/profiles \
-H "Content-Type: application/json" \
-d '{"name": "My Voice", "language": "en"}'
```
**Use cases:** game dialogue, podcast production, accessibility tools, voice assistants, content automation.
**Use cases:**
Full API documentation available at `http://localhost:17493/docs`.
- Game dialogue systems
- Podcast/video production pipelines
- Accessibility tools
- Voice assistants
- Content creation automation
Full API documentation available at `http://localhost:8000/docs` when running.
---
## Tech Stack
| Layer | Technology |
|-------|------------|
| Desktop App | Tauri (Rust) |
| Frontend | React, TypeScript, Tailwind CSS |
| State | Zustand, React Query |
| Backend | FastAPI (Python) |
| TTS Engines | Qwen3-TTS, LuxTTS, Chatterbox, Chatterbox Turbo |
| Effects | Pedalboard (Spotify) |
| Transcription | Whisper / Whisper Turbo (PyTorch or MLX) |
| Inference | MLX (Apple Silicon) / PyTorch (CUDA/ROCm/XPU/CPU) |
| Database | SQLite |
| Audio | WaveSurfer.js, librosa |
| Layer | Technology |
| ---------------- | --------------------------------------------------- |
| Desktop App | Tauri (Rust) |
| Frontend | React, TypeScript, Tailwind CSS |
| State | Zustand, React Query |
| Backend | FastAPI (Python) |
| Voice Model | Qwen3-TTS (PyTorch or MLX) |
| Transcription | Whisper (PyTorch or MLX) |
| Inference Engine | MLX (Apple Silicon) / PyTorch (Windows/Linux/Intel) |
| Database | SQLite |
| Audio | WaveSurfer.js, librosa |
**Why this stack?**
- **Tauri over Electron** — 10x smaller bundle, native performance, lower memory
- **FastAPI** — Async Python with automatic OpenAPI schema generation
- **Type-safe end-to-end** — Generated TypeScript client from OpenAPI spec
### TTS Provider Architecture
Voicebox uses a modular provider system to support different inference backends:
- **`apple-mlx`** — Bundled with macOS Apple Silicon builds
- Uses MLX with native Metal acceleration (4-5x faster)
- Works out of the box, no download required
- **`pytorch-cpu`** — Universal CPU provider (bundled or downloaded)
- Bundled with Windows and macOS Intel builds
- Downloaded on first use for Linux (~300MB)
- **`pytorch-cuda`** — Optional NVIDIA GPU-accelerated provider
- Windows/Linux only (~2.4GB)
- 4-5x faster inference on CUDA-capable GPUs
macOS and Windows builds work out of the box with bundled providers. Linux users download a provider on first launch. The app automatically detects your hardware and recommends the best option. All downloadable providers are distributed via Cloudflare R2 for fast, global delivery.
---
## Roadmap
| Feature | Description |
|---------|-------------|
| **Real-time Streaming** | Stream audio as it generates, word by word |
| **Voice Design** | Create new voices from text descriptions |
| **More Models** | XTTS, Bark, and other open-source voice models |
| **Plugin Architecture** | Extend with custom models and effects |
| **Mobile Companion** | Control Voicebox from your phone |
Voicebox is the beginning of something bigger. Here's what's coming:
### Coming Soon
| Feature | Description |
| ----------------------- | -------------------------------------------------- |
| **Real-time Synthesis** | Stream audio as it generates, word by word |
| **Conversation Mode** | Multi-speaker dialogues with automatic turn-taking |
| **Voice Effects** | Pitch shift, reverb, M3GAN-style effects |
| **Timeline Editor** | Audio studio with word-level precision editing |
| **More Models** | XTTS, Bark, and other open-source voice models |
### Future Vision
- **Voice Design** — Create new voices from text descriptions
- **Project System** — Save and load complex multi-voice sessions
- **Plugin Architecture** — Extend with custom models and effects
- **Mobile Companion** — Control Voicebox from your phone
Voicebox aims to be the **one-stop shop for everything voice** — cloning, synthesis, editing, effects, and beyond.
---
@@ -255,27 +269,48 @@ Full API documentation available at `http://localhost:17493/docs`.
See [CONTRIBUTING.md](CONTRIBUTING.md) for detailed setup and contribution guidelines.
**Using the Makefile (recommended):** Run `make help` to see all available commands for setup, development, building, and testing.
### Quick Start
**With Makefile (Unix/macOS/Linux):**
```bash
git clone https://github.com/jamiepine/voicebox.git
# Clone the repo
git clone https://github.com/voicebox-sh/voicebox.git
cd voicebox
just setup # creates Python venv, installs all deps
just dev # starts backend + desktop app
# Setup everything
make setup
# Start development
make dev
```
Install [just](https://github.com/casey/just): `brew install just` or `cargo install just`. Run `just --list` to see all commands.
**Prerequisites:** [Bun](https://bun.sh), [Rust](https://rustup.rs), [Python 3.11+](https://python.org), [Tauri Prerequisites](https://v2.tauri.app/start/prerequisites/), and [Xcode](https://developer.apple.com/xcode/) on macOS.
### Building Locally
**Manual setup (all platforms):**
```bash
just build # Build CPU server binary + Tauri app
just build-local # (Windows) Build CPU + CUDA server binaries + Tauri app
# Clone the repo
git clone https://github.com/voicebox-sh/voicebox.git
cd voicebox
# Install dependencies
bun install
# Install Python dependencies
cd backend && pip install -r requirements.txt && cd ..
# Start development
bun run dev
```
**Prerequisites:** [Bun](https://bun.sh), [Rust](https://rustup.rs), [Python 3.11+](https://python.org).
**Performance:**
- **Apple Silicon (M1/M2/M3)**: Uses MLX backend with native Metal acceleration for 4-5x faster inference
- **Windows/Linux/Intel Mac**: Uses PyTorch backend (CUDA GPU recommended, CPU supported but slower)
### Project Structure
```
+6 -2
View File
@@ -1,6 +1,6 @@
{
"name": "@voicebox/app",
"version": "0.2.3",
"version": "0.1.13",
"private": true,
"type": "module",
"scripts": {
@@ -17,6 +17,10 @@
"@dnd-kit/sortable": "^10.0.0",
"@dnd-kit/utilities": "^3.2.2",
"@hookform/resolvers": "^3.9.0",
"@hugeicons/core-free-icons": "^3.1.1",
"@hugeicons/react": "^1.1.4",
"@iconify-json/svg-spinners": "^1.2.4",
"@iconify/react": "^6.0.2",
"@radix-ui/react-alert-dialog": "^1.1.1",
"@radix-ui/react-avatar": "^1.1.0",
"@radix-ui/react-dialog": "^1.1.1",
@@ -24,6 +28,7 @@
"@radix-ui/react-label": "^2.1.0",
"@radix-ui/react-popover": "^1.1.1",
"@radix-ui/react-progress": "^1.1.0",
"@radix-ui/react-radio-group": "^1.2.0",
"@radix-ui/react-scroll-area": "^1.1.0",
"@radix-ui/react-select": "^2.1.1",
"@radix-ui/react-separator": "^1.1.0",
@@ -43,7 +48,6 @@
"clsx": "^2.1.1",
"date-fns": "^3.6.0",
"framer-motion": "^12.29.0",
"lucide-react": "^0.454.0",
"motion": "^12.29.0",
"react": "^18.3.0",
"react-dom": "^18.3.0",
+6 -11
View File
@@ -40,7 +40,7 @@ function App() {
const serverStartingRef = useRef(false);
// Automatically check for app updates on startup and show toast notifications
useAutoUpdater({ checkOnMount: true, showToast: true });
useAutoUpdater(true);
// Sync stored setting to Rust on startup
useEffect(() => {
@@ -82,8 +82,7 @@ function App() {
console.log('Dev mode: Skipping auto-start of server (run it separately)');
setServerReady(true); // Mark as ready so UI doesn't show loading screen
// Mark that server was not started by app (so we don't try to stop it on close)
// @ts-expect-error - adding property to window
window.__voiceboxServerStartedByApp = false;
(window as any).__voiceboxServerStartedByApp = false;
return;
}
@@ -93,26 +92,22 @@ function App() {
}
serverStartingRef.current = true;
const isRemote = useServerStore.getState().mode === 'remote';
const customModelsDir = useServerStore.getState().customModelsDir;
console.log(`Production mode: Starting bundled server... (remote: ${isRemote})`);
console.log('Production mode: Starting bundled server...');
platform.lifecycle
.startServer(isRemote, customModelsDir)
.startServer(false)
.then((serverUrl) => {
console.log('Server is ready at:', serverUrl);
// Update the server URL in the store with the dynamically assigned port
useServerStore.getState().setServerUrl(serverUrl);
setServerReady(true);
// Mark that we started the server (so we know to stop it on close)
// @ts-expect-error - adding property to window
window.__voiceboxServerStartedByApp = true;
(window as any).__voiceboxServerStartedByApp = true;
})
.catch((error) => {
console.error('Failed to auto-start server:', error);
serverStartingRef.current = false;
// @ts-expect-error - adding property to window
window.__voiceboxServerStartedByApp = false;
(window as any).__voiceboxServerStartedByApp = false;
});
// Cleanup: stop server on actual unmount (not StrictMode remount)
+147 -77
View File
@@ -1,18 +1,18 @@
import { useQuery } from '@tanstack/react-query';
import { Pause, Play, Repeat, Volume2, VolumeX, X } from 'lucide-react';
import { useEffect, useId, useMemo, useRef, useState } from 'react';
import { HugeiconsIcon } from '@hugeicons/react';
import { PauseIcon, PlayIcon, RepeatIcon, VolumeHighIcon, VolumeMuteIcon, Cancel01Icon } from '@hugeicons/core-free-icons';
import { useEffect, useMemo, useRef, useState } from 'react';
import WaveSurfer from 'wavesurfer.js';
import { Button } from '@/components/ui/button';
import { Slider } from '@/components/ui/slider';
import { apiClient } from '@/lib/api/client';
import { formatAudioDuration } from '@/lib/utils/audio';
import { debug } from '@/lib/utils/debug';
import { usePlatform } from '@/platform/PlatformContext';
import { usePlayerStore } from '@/stores/playerStore';
import { usePlatform } from '@/platform/PlatformContext';
export function AudioPlayer() {
const platform = usePlatform();
const volumeLabelId = useId();
const {
audioUrl,
audioId,
@@ -139,11 +139,7 @@ export function AudioPlayer() {
barRadius: 2,
height: 80,
normalize: true,
// Use MediaElement backend (default). Unlike the WebAudio backend,
// MediaElement uses a standard <audio> element for playback which
// benefits from the browser/webview's built-in audio session recovery.
// This prevents audio loss when another app steals audio output or
// the system audio session is interrupted.
backend: 'WebAudio',
interact: true, // Enable interaction (click to seek)
mediaControls: false, // Don't show native controls
});
@@ -161,21 +157,8 @@ export function AudioPlayer() {
const wavesurfer = wavesurferRef.current;
if (!wavesurfer) return;
// Update store when time changes, stop if past duration
// Update store when time changes
wavesurfer.on('timeupdate', (time) => {
const dur = usePlayerStore.getState().duration;
if (dur > 0 && time >= dur) {
setCurrentTime(dur);
const loop = usePlayerStore.getState().isLooping;
if (loop) {
wavesurfer.seekTo(0);
wavesurfer.play();
} else {
wavesurfer.pause();
setIsPlaying(false);
}
return;
}
setCurrentTime(time);
});
@@ -193,6 +176,15 @@ export function AudioPlayer() {
const currentVolume = usePlayerStore.getState().volume;
wavesurfer.setVolume(currentVolume);
// Get the underlying audio element and ensure it's not muted
// (unless we're using native playback, which will be set later)
const mediaElement = wavesurfer.getMediaElement();
if (mediaElement && !isUsingNativePlaybackRef.current) {
mediaElement.volume = currentVolume;
mediaElement.muted = false;
debug.log('Audio element volume:', mediaElement.volume, 'muted:', mediaElement.muted);
}
// Auto-play when ready - check if we should use native playback
// Get current values from the store and queries at runtime (not captured closure values)
const currentAudioUrl = usePlayerStore.getState().audioUrl;
@@ -259,8 +251,21 @@ export function AudioPlayer() {
debug.log('Should use native playback:', shouldUseNative);
if (!shouldUseNative) {
debug.log('No custom devices assigned, using standard playback');
debug.log('No custom devices assigned, falling back to WaveSurfer');
// Reset native playback flag and unmute WaveSurfer
isUsingNativePlaybackRef.current = false;
const mediaElement = wavesurfer.getMediaElement();
if (mediaElement) {
const currentVolume = usePlayerStore.getState().volume;
mediaElement.volume = currentVolume;
mediaElement.muted = false;
debug.log(
'WaveSurfer unmuted for normal playback - volume:',
mediaElement.volume,
'muted:',
mediaElement.muted,
);
}
} else {
const deviceIds = assignedChannels.flatMap((ch: any) => ch.device_ids);
debug.log('Device IDs to play to:', deviceIds);
@@ -281,10 +286,19 @@ export function AudioPlayer() {
// Mark that we're using native playback
isUsingNativePlaybackRef.current = true;
// Mute WaveSurfer's audio output — native handles the actual sound
// Keep WaveSurfer running for waveform visualization
wavesurfer.setVolume(0);
wavesurfer.setMuted(true);
// Mute WaveSurfer's audio element to prevent UI audio output
// Keep WaveSurfer running for visualization
const mediaElement = wavesurfer.getMediaElement();
if (mediaElement) {
mediaElement.volume = 0;
mediaElement.muted = true;
debug.log(
'WaveSurfer muted for native playback - volume:',
mediaElement.volume,
'muted:',
mediaElement.muted,
);
}
// Start WaveSurfer playback for visualization (muted)
wavesurfer.play().catch((error) => {
@@ -307,15 +321,38 @@ export function AudioPlayer() {
'Native playback failed during auto-play, falling back to WaveSurfer:',
error,
);
// Reset native playback flag and unmute WaveSurfer
isUsingNativePlaybackRef.current = false;
const mediaElement = wavesurfer.getMediaElement();
if (mediaElement) {
const currentVolume = usePlayerStore.getState().volume;
mediaElement.volume = currentVolume;
mediaElement.muted = false;
debug.log(
'WaveSurfer unmuted after native playback failure - volume:',
mediaElement.volume,
'muted:',
mediaElement.muted,
);
}
// Fall through to WaveSurfer playback
}
}
// Standard playback path — ensure WaveSurfer is unmuted
if (!isUsingNativePlaybackRef.current) {
wavesurfer.setMuted(false);
wavesurfer.setVolume(usePlayerStore.getState().volume);
} else {
debug.log('Not using native playback, using WaveSurfer');
// Reset native playback flag and unmute WaveSurfer
isUsingNativePlaybackRef.current = false;
const mediaElement = wavesurfer.getMediaElement();
if (mediaElement) {
const currentVolume = usePlayerStore.getState().volume;
mediaElement.volume = currentVolume;
mediaElement.muted = false;
debug.log(
'WaveSurfer unmuted for normal playback - volume:',
mediaElement.volume,
'muted:',
mediaElement.muted,
);
}
}
// Only auto-play if shouldAutoPlay flag is set (user explicitly clicked to play)
@@ -323,7 +360,7 @@ export function AudioPlayer() {
if (shouldAutoPlayNow) {
// Clear the flag first
usePlayerStore.getState().clearAutoPlayFlag();
// Use a small delay to ensure audio element is fully ready
setTimeout(() => {
wavesurfer.play().catch((error) => {
@@ -339,6 +376,28 @@ export function AudioPlayer() {
// Handle play/pause
wavesurfer.on('play', () => {
setIsPlaying(true);
// Ensure audio element volume is set correctly
const mediaElement = wavesurfer.getMediaElement();
if (mediaElement) {
// Double-check: if using native playback, keep WaveSurfer muted
// Otherwise, ensure it's unmuted
if (isUsingNativePlaybackRef.current) {
mediaElement.volume = 0;
mediaElement.muted = true;
debug.log('Playing (native mode) - WaveSurfer muted for visualization only');
} else {
// Ensure WaveSurfer is unmuted for normal playback
const currentVolume = usePlayerStore.getState().volume;
mediaElement.volume = currentVolume;
mediaElement.muted = false;
debug.log(
'Playing (normal mode) - volume:',
mediaElement.volume,
'muted:',
mediaElement.muted,
);
}
}
});
wavesurfer.on('pause', () => setIsPlaying(false));
wavesurfer.on('finish', () => {
@@ -401,7 +460,7 @@ export function AudioPlayer() {
// Use double requestAnimationFrame to ensure DOM is fully rendered
let rafId1: number;
let rafId2: number;
let timeoutId: number | null = null;
let timeoutId: ReturnType<typeof setTimeout> | null = null;
rafId1 = requestAnimationFrame(() => {
rafId2 = requestAnimationFrame(() => {
@@ -420,6 +479,11 @@ export function AudioPlayer() {
if (wavesurferRef.current) {
debug.log('Destroying WaveSurfer instance');
try {
const mediaElement = wavesurferRef.current.getMediaElement();
if (mediaElement) {
mediaElement.pause();
mediaElement.src = '';
}
wavesurferRef.current.destroy();
} catch (error) {
debug.error('Error destroying WaveSurfer:', error);
@@ -460,10 +524,13 @@ export function AudioPlayer() {
}
// Reset native playback flag when loading new audio
// Unmute WaveSurfer if it was muted for native playback
// Also unmute WaveSurfer if it was muted
if (isUsingNativePlaybackRef.current) {
wavesurfer.setMuted(false);
wavesurfer.setVolume(usePlayerStore.getState().volume);
const mediaElement = wavesurfer.getMediaElement();
if (mediaElement) {
mediaElement.muted = false;
mediaElement.volume = usePlayerStore.getState().volume;
}
}
isUsingNativePlaybackRef.current = false;
@@ -479,7 +546,16 @@ export function AudioPlayer() {
wavesurfer.pause();
}
// Use empty() to completely destroy the waveform and reset media
// Stop the media element explicitly
const mediaElement = wavesurfer.getMediaElement();
if (mediaElement) {
debug.log('Stopping media element');
mediaElement.pause();
mediaElement.currentTime = 0;
mediaElement.src = '';
}
// Use empty() to completely destroy the waveform and media element
debug.log('Calling wavesurfer.empty() to destroy audio');
wavesurfer.empty();
} catch (error) {
@@ -534,13 +610,20 @@ export function AudioPlayer() {
// Sync volume
useEffect(() => {
if (wavesurferRef.current) {
// If using native playback, keep WaveSurfer muted regardless of volume setting
if (isUsingNativePlaybackRef.current) {
wavesurferRef.current.setVolume(0);
debug.log('Volume sync: Using native playback, keeping WaveSurfer muted');
} else {
wavesurferRef.current.setVolume(volume);
debug.log('Volume synced:', volume);
wavesurferRef.current.setVolume(volume);
// Also ensure the underlying audio element volume is set
const mediaElement = wavesurferRef.current.getMediaElement();
if (mediaElement) {
// If using native playback, keep WaveSurfer muted regardless of volume setting
if (isUsingNativePlaybackRef.current) {
mediaElement.volume = 0;
mediaElement.muted = true;
debug.log('Volume sync: Using native playback, keeping WaveSurfer muted');
} else {
mediaElement.volume = volume;
mediaElement.muted = volume === 0;
debug.log('Volume synced:', volume, 'muted:', mediaElement.muted);
}
}
}
}, [volume]);
@@ -582,7 +665,7 @@ export function AudioPlayer() {
// Handle shouldAutoPlay flag - for story mode auto-advance
const shouldAutoPlay = usePlayerStore((state) => state.shouldAutoPlay);
const clearAutoPlayFlag = usePlayerStore((state) => state.clearAutoPlayFlag);
useEffect(() => {
const wavesurfer = wavesurferRef.current;
if (!wavesurfer || !shouldAutoPlay || duration === 0) {
@@ -661,8 +744,11 @@ export function AudioPlayer() {
isUsingNativePlaybackRef.current = true;
// Mute WaveSurfer and start it for visualization
wavesurferRef.current.setVolume(0);
wavesurferRef.current.setMuted(true);
const mediaElement = wavesurferRef.current.getMediaElement();
if (mediaElement) {
mediaElement.volume = 0;
mediaElement.muted = true;
}
// Start WaveSurfer for visualization (muted)
wavesurferRef.current.play().catch((error) => {
@@ -686,8 +772,11 @@ export function AudioPlayer() {
} else {
// Ensure WaveSurfer is not muted if not using native playback
if (!isUsingNativePlaybackRef.current) {
wavesurferRef.current.setMuted(false);
wavesurferRef.current.setVolume(volume);
const mediaElement = wavesurferRef.current.getMediaElement();
if (mediaElement) {
mediaElement.muted = false;
mediaElement.volume = volume;
}
}
wavesurferRef.current.play().catch((error) => {
@@ -743,11 +832,8 @@ export function AudioPlayer() {
disabled={isLoading || duration === 0}
className="shrink-0"
title={duration === 0 && !isLoading ? 'Audio not loaded' : ''}
aria-label={
duration === 0 && !isLoading ? 'Audio not loaded' : isPlaying ? 'Pause' : 'Play'
}
>
{isPlaying ? <Pause className="h-5 w-5" /> : <Play className="h-5 w-5" />}
{isPlaying ? <HugeiconsIcon icon={PauseIcon} size={20} className="h-5 w-5" /> : <HugeiconsIcon icon={PlayIcon} size={20} className="h-5 w-5" />}
</Button>
{/* Waveform */}
@@ -760,8 +846,6 @@ export function AudioPlayer() {
max={100}
step={0.1}
className="w-full"
aria-label="Playback position"
aria-valuetext={`${formatAudioDuration(currentTime)} of ${formatAudioDuration(duration)}`}
/>
)}
{isLoading && (
@@ -779,9 +863,7 @@ export function AudioPlayer() {
{/* Title */}
{title && (
<div className="text-sm font-medium truncate max-w-[200px] shrink-0 hidden lg:block">
{title}
</div>
<div className="text-sm font-medium truncate max-w-[200px] shrink-0">{title}</div>
)}
{/* Loop Button */}
@@ -791,37 +873,26 @@ export function AudioPlayer() {
onClick={toggleLoop}
className={isLooping ? 'text-primary' : ''}
title="Toggle loop"
aria-label={isLooping ? 'Stop looping' : 'Loop'}
>
<Repeat className="h-4 w-4" />
<HugeiconsIcon icon={RepeatIcon} size={16} className="h-4 w-4" />
</Button>
{/* Volume Control */}
<div
className="flex items-center gap-2 shrink-0 w-[120px]"
role="group"
aria-label="Volume"
>
<div className="flex items-center gap-2 shrink-0 w-[120px]">
<Button
variant="ghost"
size="icon"
onClick={() => setVolume(volume > 0 ? 0 : 1)}
className="h-8 w-8"
aria-label={volume > 0 ? 'Mute' : 'Unmute'}
>
{volume > 0 ? <Volume2 className="h-4 w-4" /> : <VolumeX className="h-4 w-4" />}
{volume > 0 ? <HugeiconsIcon icon={VolumeHighIcon} size={16} className="h-4 w-4" /> : <HugeiconsIcon icon={VolumeMuteIcon} size={16} className="h-4 w-4" />}
</Button>
<span id={volumeLabelId} className="sr-only">
Volume level, {Math.round(volume * 100)}%
</span>
<Slider
value={[volume * 100]}
onValueChange={handleVolumeChange}
max={100}
step={1}
className="flex-1"
aria-labelledby={volumeLabelId}
aria-valuetext={`${Math.round(volume * 100)}%`}
/>
</div>
@@ -832,9 +903,8 @@ export function AudioPlayer() {
onClick={handleClose}
className="shrink-0"
title="Close player"
aria-label="Close player"
>
<X className="h-5 w-5" />
<HugeiconsIcon icon={Cancel01Icon} size={20} className="h-5 w-5" />
</Button>
</div>
</div>
+23 -26
View File
@@ -1,5 +1,6 @@
import { useMutation, useQuery, useQueryClient } from '@tanstack/react-query';
import { Check, CheckCircle2, Edit, Plus, Speaker, Trash2 } from 'lucide-react';
import { HugeiconsIcon } from '@hugeicons/react';
import { CheckmarkCircle01Icon, CheckmarkCircle02Icon, Edit01Icon, Add01Icon, SpeakerIcon, Delete01Icon } from '@hugeicons/core-free-icons';
import { useState } from 'react';
import { Badge } from '@/components/ui/badge';
import { Button } from '@/components/ui/button';
@@ -23,8 +24,8 @@ import {
import { apiClient } from '@/lib/api/client';
import { BOTTOM_SAFE_AREA_PADDING } from '@/lib/constants/ui';
import { cn } from '@/lib/utils/cn';
import { usePlatform } from '@/platform/PlatformContext';
import { usePlayerStore } from '@/stores/playerStore';
import { usePlatform } from '@/platform/PlatformContext';
interface AudioDevice {
id: string;
@@ -124,13 +125,6 @@ export function AudioTab() {
);
}
const handleChannelDelete = async (e, channelId) => {
e.stopPropagation();
if (await confirm('Delete this channel?')) {
deleteChannel.mutate(channelId);
}
};
const allChannels = channels || [];
const allDevices = devices || [];
const selectedChannel = selectedChannelId
@@ -142,7 +136,7 @@ export function AudioTab() {
<div className="flex items-center justify-between mb-6 shrink-0">
<h2 className="text-2xl font-bold">Audio Channels</h2>
<Button onClick={() => setCreateDialogOpen(true)}>
<Plus className="h-4 w-4 mr-2" />
<HugeiconsIcon icon={Add01Icon} size={16} className="h-4 w-4 mr-2" />
New Channel
</Button>
</div>
@@ -157,18 +151,18 @@ export function AudioTab() {
>
{allChannels.length === 0 ? (
<div className="flex flex-col items-center justify-center py-12 border-2 border-dashed border-muted rounded-md">
<Speaker className="h-12 w-12 text-muted-foreground mb-4" />
<HugeiconsIcon icon={SpeakerIcon} size={48} className="h-12 w-12 text-muted-foreground mb-4" />
<p className="text-muted-foreground mb-4">
No audio channels yet. Create your first channel to route voices to specific
devices.
</p>
<Button onClick={() => setCreateDialogOpen(true)}>
<Plus className="h-4 w-4 mr-2" />
<HugeiconsIcon icon={Add01Icon} size={16} className="h-4 w-4 mr-2" />
Create Channel
</Button>
</div>
) : (
<div className="space-y-3">
<div className="space-y-3 p-2">
{allChannels.map((channel) => {
const isSelected = selectedChannelId === channel.id;
return (
@@ -185,7 +179,7 @@ export function AudioTab() {
<div className="flex-1 min-w-0">
<div className="flex items-center gap-2 mb-3">
<div className="h-8 w-8 rounded-lg bg-muted flex items-center justify-center shrink-0">
<Speaker className="h-4 w-4 text-muted-foreground" />
<HugeiconsIcon icon={SpeakerIcon} size={16} className="h-4 w-4 text-muted-foreground" />
</div>
<div className="flex items-center gap-2 min-w-0">
<h3 className="font-semibold text-base truncate">{channel.name}</h3>
@@ -242,15 +236,20 @@ export function AudioTab() {
setEditingChannel(channel.id);
}}
>
<Edit className="h-4 w-4" />
<HugeiconsIcon icon={Edit01Icon} size={16} className="h-4 w-4" />
</Button>
<Button
variant="ghost"
size="sm"
className="h-8 w-8 p-0"
onClick={(e) => handleChannelDelete(e, channel.id)}
onClick={(e) => {
e.stopPropagation();
if (confirm('Delete this channel?')) {
deleteChannel.mutate(channel.id);
}
}}
>
<Trash2 className="h-4 w-4" />
<HugeiconsIcon icon={Delete01Icon} size={16} className="h-4 w-4" />
</Button>
</div>
)}
@@ -327,10 +326,10 @@ export function AudioTab() {
isConnected ? 'bg-accent border-accent' : 'border-muted-foreground/30',
)}
>
{isConnected && <Check className="h-3 w-3 text-accent-foreground" />}
{isConnected && <HugeiconsIcon icon={CheckmarkCircle01Icon} size={12} className="h-3 w-3 text-accent-foreground" />}
</div>
) : device.is_default ? (
<CheckCircle2 className="h-4 w-4 text-primary shrink-0" />
<HugeiconsIcon icon={CheckmarkCircle02Icon} size={16} className="h-4 w-4 text-primary shrink-0" />
) : null}
<span className={cn('truncate flex-1', device.is_default && 'font-medium')}>
{device.name}
@@ -341,11 +340,9 @@ export function AudioTab() {
</div>
) : (
<div className="flex flex-col items-center justify-center py-12 border-2 border-dashed border-muted rounded-md">
<CheckCircle2 className="h-12 w-12 text-muted-foreground mb-4" />
<HugeiconsIcon icon={CheckmarkCircle02Icon} size={48} className="h-12 w-12 text-muted-foreground mb-4" />
<p className="text-muted-foreground text-center">
{platform.metadata.isTauri
? 'No audio devices found'
: 'Audio device selection requires Tauri'}
{platform.metadata.isTauri ? 'No audio devices found' : 'Audio device selection requires Tauri'}
</p>
</div>
)}
@@ -498,7 +495,7 @@ function CreateChannelDialog({ open, onOpenChange, devices, onCreate }: CreateCh
setSelectedDevices(selectedDevices.filter((id) => id !== deviceId))
}
>
<Trash2 className="h-3 w-3" />
<HugeiconsIcon icon={Delete01Icon} size={12} className="h-3 w-3" />
</Button>
</div>
);
@@ -606,7 +603,7 @@ function EditChannelDialog({
setSelectedDevices(selectedDevices.filter((id) => id !== deviceId))
}
>
<Trash2 className="h-3 w-3" />
<HugeiconsIcon icon={Delete01Icon} size={12} className="h-3 w-3" />
</Button>
</div>
);
@@ -652,7 +649,7 @@ function EditChannelDialog({
setSelectedVoices(selectedVoices.filter((id) => id !== profileId))
}
>
<Trash2 className="h-3 w-3" />
<HugeiconsIcon icon={Delete01Icon} size={12} className="h-3 w-3" />
</Button>
</div>
);
@@ -1,377 +0,0 @@
import {
closestCenter,
DndContext,
type DragEndEvent,
KeyboardSensor,
PointerSensor,
useSensor,
useSensors,
} from '@dnd-kit/core';
import {
arrayMove,
SortableContext,
sortableKeyboardCoordinates,
useSortable,
verticalListSortingStrategy,
} from '@dnd-kit/sortable';
import { CSS } from '@dnd-kit/utilities';
import { useQuery } from '@tanstack/react-query';
import { ChevronDown, ChevronRight, GripVertical, Plus, Power, Trash2 } from 'lucide-react';
import { useCallback, useMemo, useRef, useState } from 'react';
import { Button } from '@/components/ui/button';
import { Label } from '@/components/ui/label';
import {
Select,
SelectContent,
SelectItem,
SelectTrigger,
SelectValue,
} from '@/components/ui/select';
import { Slider } from '@/components/ui/slider';
import { apiClient } from '@/lib/api/client';
import type { AvailableEffect, EffectConfig, EffectPresetResponse } from '@/lib/api/types';
import { cn } from '@/lib/utils/cn';
// Each effect in the chain gets a stable ID for dnd-kit
interface EffectWithId extends EffectConfig {
_id: string;
}
let nextId = 0;
function makeId() {
return `fx-${++nextId}`;
}
interface EffectsChainEditorProps {
value: EffectConfig[];
onChange: (chain: EffectConfig[]) => void;
compact?: boolean;
showPresets?: boolean;
}
export function EffectsChainEditor({
value,
onChange,
compact = false,
showPresets = true,
}: EffectsChainEditorProps) {
const [expandedId, setExpandedId] = useState<string | null>(null);
// Maintain stable IDs for each effect across renders.
// We use a ref to map value items to IDs, rebuilding when length changes.
const idsRef = useRef<string[]>([]);
const items: EffectWithId[] = useMemo(() => {
// Grow ID array if effects were added
while (idsRef.current.length < value.length) {
idsRef.current.push(makeId());
}
// Shrink if effects were removed
if (idsRef.current.length > value.length) {
idsRef.current = idsRef.current.slice(0, value.length);
}
return value.map((e, i) => ({ ...e, _id: idsRef.current[i] }));
}, [value]);
const sensors = useSensors(
useSensor(PointerSensor, { activationConstraint: { distance: 5 } }),
useSensor(KeyboardSensor, { coordinateGetter: sortableKeyboardCoordinates }),
);
const { data: availableEffects } = useQuery({
queryKey: ['available-effects'],
queryFn: () => apiClient.getAvailableEffects(),
staleTime: Infinity,
});
const { data: presets } = useQuery({
queryKey: ['effect-presets'],
queryFn: () => apiClient.listEffectPresets(),
staleTime: 30_000,
});
const effectsMap = useMemo(() => {
const m = new Map<string, AvailableEffect>();
if (availableEffects) {
for (const e of availableEffects.effects) {
m.set(e.type, e);
}
}
return m;
}, [availableEffects]);
function addEffect(type: string) {
const def = effectsMap.get(type);
if (!def) return;
const params: Record<string, number> = {};
for (const [key, p] of Object.entries(def.params)) {
params[key] = p.default;
}
const newEffect: EffectConfig = { type, enabled: true, params };
const newId = makeId();
idsRef.current = [...idsRef.current, newId];
onChange([...value, newEffect]);
setExpandedId(newId);
}
const removeEffect = useCallback(
(index: number) => {
const removedId = idsRef.current[index];
idsRef.current = idsRef.current.filter((_, i) => i !== index);
onChange(value.filter((_, i) => i !== index));
if (expandedId === removedId) setExpandedId(null);
},
[value, onChange, expandedId],
);
const toggleEnabled = useCallback(
(index: number) => {
onChange(value.map((e, i) => (i === index ? { ...e, enabled: !e.enabled } : e)));
},
[value, onChange],
);
const updateParam = useCallback(
(index: number, paramName: string, paramValue: number) => {
onChange(
value.map((e, i) =>
i === index ? { ...e, params: { ...e.params, [paramName]: paramValue } } : e,
),
);
},
[value, onChange],
);
function loadPreset(preset: EffectPresetResponse) {
idsRef.current = preset.effects_chain.map(() => makeId());
onChange(preset.effects_chain);
setExpandedId(null);
}
function clearAll() {
idsRef.current = [];
onChange([]);
setExpandedId(null);
}
function handleDragEnd(event: DragEndEvent) {
const { active, over } = event;
if (!over || active.id === over.id) return;
const oldIndex = idsRef.current.indexOf(active.id as string);
const newIndex = idsRef.current.indexOf(over.id as string);
if (oldIndex === -1 || newIndex === -1) return;
idsRef.current = arrayMove(idsRef.current, oldIndex, newIndex);
onChange(arrayMove([...value], oldIndex, newIndex));
}
return (
<div className={cn('space-y-2', compact && 'text-sm')}>
{/* Preset selector row */}
{showPresets && (
<div className="flex items-center gap-2">
<Select
onValueChange={(id) => {
const preset = presets?.find((p) => p.id === id);
if (preset) loadPreset(preset);
}}
>
<SelectTrigger className="h-8 flex-1 text-xs focus:ring-0 focus:ring-offset-0">
<SelectValue placeholder="Load preset..." />
</SelectTrigger>
<SelectContent>
{presets?.map((p) => (
<SelectItem key={p.id} value={p.id}>
{p.name}
{p.description && (
<span className="ml-1 text-muted-foreground">- {p.description}</span>
)}
</SelectItem>
))}
</SelectContent>
</Select>
{value.length > 0 && (
<Button
variant="ghost"
size="sm"
className="h-8 px-2 text-xs text-muted-foreground"
onClick={clearAll}
>
Clear
</Button>
)}
</div>
)}
{/* Sortable effects chain */}
<DndContext sensors={sensors} collisionDetection={closestCenter} onDragEnd={handleDragEnd}>
<SortableContext items={items.map((i) => i._id)} strategy={verticalListSortingStrategy}>
{items.map((effect, index) => (
<SortableEffectItem
key={effect._id}
id={effect._id}
effect={effect}
index={index}
effectDef={effectsMap.get(effect.type)}
isExpanded={expandedId === effect._id}
onToggleExpand={() => setExpandedId(expandedId === effect._id ? null : effect._id)}
onRemove={() => removeEffect(index)}
onToggleEnabled={() => toggleEnabled(index)}
onUpdateParam={(paramName, paramValue) => updateParam(index, paramName, paramValue)}
/>
))}
</SortableContext>
</DndContext>
{/* Add effect */}
{availableEffects && (
<Select onValueChange={addEffect}>
<SelectTrigger className="h-8 border-dashed text-xs text-muted-foreground focus:ring-0 focus:ring-offset-0">
<Plus className="mr-1 h-3.5 w-3.5" />
<SelectValue placeholder="Add effect..." />
</SelectTrigger>
<SelectContent>
{availableEffects.effects.map((e) => (
<SelectItem key={e.type} value={e.type}>
{e.label}
</SelectItem>
))}
</SelectContent>
</Select>
)}
</div>
);
}
// ---------------------------------------------------------------------------
// Sortable effect item
// ---------------------------------------------------------------------------
interface SortableEffectItemProps {
id: string;
effect: EffectConfig;
index: number;
effectDef?: AvailableEffect;
isExpanded: boolean;
onToggleExpand: () => void;
onRemove: () => void;
onToggleEnabled: () => void;
onUpdateParam: (paramName: string, paramValue: number) => void;
}
function SortableEffectItem({
id,
effect,
effectDef,
isExpanded,
onToggleExpand,
onRemove,
onToggleEnabled,
onUpdateParam,
}: SortableEffectItemProps) {
const { attributes, listeners, setNodeRef, transform, transition, isDragging } = useSortable({
id,
});
const style = {
transform: CSS.Transform.toString(transform),
transition,
zIndex: isDragging ? 10 : undefined,
};
const label = effectDef?.label ?? effect.type;
return (
<div
ref={setNodeRef}
style={style}
className={cn(
'rounded-md border',
effect.enabled ? 'border-border bg-card' : 'border-border/50 bg-muted/30',
isDragging && 'opacity-80 shadow-lg',
)}
>
{/* Header */}
<div className="flex items-center gap-1 px-2 py-1.5">
<button
type="button"
className="p-0.5 text-muted-foreground hover:text-foreground"
onClick={onToggleExpand}
>
{isExpanded ? (
<ChevronDown className="h-3.5 w-3.5" />
) : (
<ChevronRight className="h-3.5 w-3.5" />
)}
</button>
<button
type="button"
className="p-0.5 text-muted-foreground/50 hover:text-muted-foreground cursor-grab active:cursor-grabbing touch-none"
{...attributes}
{...listeners}
>
<GripVertical className="h-3.5 w-3.5" />
</button>
<span
className={cn('flex-1 text-xs font-medium', !effect.enabled && 'text-muted-foreground')}
>
{label}
</span>
<button
type="button"
className={cn(
'p-0.5 transition-colors',
effect.enabled ? 'text-primary' : 'text-muted-foreground hover:text-foreground',
)}
onClick={onToggleEnabled}
title={effect.enabled ? 'Disable' : 'Enable'}
>
<Power className="h-3.5 w-3.5" />
</button>
<button
type="button"
className="p-0.5 text-muted-foreground hover:text-destructive"
onClick={onRemove}
title="Remove"
>
<Trash2 className="h-3.5 w-3.5" />
</button>
</div>
{/* Params */}
{isExpanded && effectDef && (
<div className="space-y-3 border-t px-3 py-2.5">
{Object.entries(effectDef.params).map(([paramName, paramDef]) => {
const currentValue = effect.params[paramName] ?? paramDef.default;
return (
<div key={paramName} className="space-y-1">
<div className="flex items-center justify-between">
<Label className="text-[11px] text-muted-foreground">
{paramDef.description}
</Label>
<span className="text-[11px] font-mono tabular-nums text-foreground">
{currentValue.toFixed(
paramDef.step < 1 ? Math.max(1, -Math.floor(Math.log10(paramDef.step))) : 0,
)}
</span>
</div>
<Slider
min={paramDef.min}
max={paramDef.max}
step={paramDef.step}
value={[currentValue]}
onValueChange={([v]) => onUpdateParam(paramName, v)}
/>
</div>
);
})}
</div>
)}
</div>
);
}
@@ -1,103 +0,0 @@
import { ChevronDown, Search } from 'lucide-react';
import { useMemo, useState } from 'react';
import { Button } from '@/components/ui/button';
import { Input } from '@/components/ui/input';
import { Popover, PopoverContent, PopoverTrigger } from '@/components/ui/popover';
import type { HistoryResponse } from '@/lib/api/types';
import { useHistory } from '@/lib/hooks/useHistory';
import { cn } from '@/lib/utils/cn';
interface GenerationPickerProps {
selectedId: string | null;
onSelect: (generation: HistoryResponse) => void;
className?: string;
}
export function GenerationPicker({ selectedId, onSelect, className }: GenerationPickerProps) {
const [open, setOpen] = useState(false);
const [searchQuery, setSearchQuery] = useState('');
const { data: historyData } = useHistory({ limit: 50 });
const completedGenerations = useMemo(() => {
if (!historyData?.items) return [];
return historyData.items.filter((gen) => gen.status === 'completed');
}, [historyData]);
const filtered = useMemo(() => {
if (!searchQuery) return completedGenerations;
const q = searchQuery.toLowerCase();
return completedGenerations.filter(
(gen) => gen.text.toLowerCase().includes(q) || gen.profile_name.toLowerCase().includes(q),
);
}, [completedGenerations, searchQuery]);
const selectedGeneration = completedGenerations.find((g) => g.id === selectedId);
return (
<Popover open={open} onOpenChange={setOpen}>
<PopoverTrigger asChild>
<Button
variant="outline"
size="sm"
className={cn('h-8 justify-between gap-2 text-xs font-normal', className)}
>
{selectedGeneration ? (
<span className="truncate">
<span className="font-medium">{selectedGeneration.profile_name}</span>
<span className="text-muted-foreground ml-1.5">
{selectedGeneration.text.length > 30
? `${selectedGeneration.text.substring(0, 30)}...`
: selectedGeneration.text}
</span>
</span>
) : (
<span className="text-muted-foreground">Select a generation...</span>
)}
<ChevronDown className="h-3.5 w-3.5 shrink-0 opacity-50" />
</Button>
</PopoverTrigger>
<PopoverContent className="w-80 p-0" align="start">
<div className="p-2 border-b">
<div className="relative">
<Search className="absolute left-2 top-1/2 -translate-y-1/2 h-3.5 w-3.5 text-muted-foreground" />
<Input
placeholder="Search by voice or text..."
value={searchQuery}
onChange={(e) => setSearchQuery(e.target.value)}
className="h-8 pl-7 text-xs"
/>
</div>
</div>
<div className="max-h-60 overflow-y-auto">
{filtered.length === 0 ? (
<div className="p-4 text-center text-xs text-muted-foreground">
No generations found
</div>
) : (
filtered.map((gen) => (
<button
key={gen.id}
type="button"
className={cn(
'w-full text-left px-3 py-2 hover:bg-muted/50 transition-colors border-b border-border/30 last:border-0',
gen.id === selectedId && 'bg-accent/10',
)}
onClick={() => {
onSelect(gen);
setOpen(false);
setSearchQuery('');
}}
>
<div className="font-medium text-sm">{gen.profile_name}</div>
<div className="text-xs text-muted-foreground truncate">
{gen.text.length > 60 ? `${gen.text.substring(0, 60)}...` : gen.text}
</div>
</button>
))
)}
</div>
</PopoverContent>
</Popover>
);
}
@@ -1,422 +0,0 @@
import { useQuery, useQueryClient } from '@tanstack/react-query';
import { Loader2, Play, Save, Trash2, Wand2 } from 'lucide-react';
import { useEffect, useRef, useState } from 'react';
import { EffectsChainEditor } from '@/components/Effects/EffectsChainEditor';
import { GenerationPicker } from '@/components/Effects/GenerationPicker';
import { Button } from '@/components/ui/button';
import {
Dialog,
DialogContent,
DialogDescription,
DialogFooter,
DialogHeader,
DialogTitle,
} from '@/components/ui/dialog';
import { Input } from '@/components/ui/input';
import { Label } from '@/components/ui/label';
import { Separator } from '@/components/ui/separator';
import { Textarea } from '@/components/ui/textarea';
import { useToast } from '@/components/ui/use-toast';
import { apiClient } from '@/lib/api/client';
import type { HistoryResponse } from '@/lib/api/types';
import { useHistory } from '@/lib/hooks/useHistory';
import { useEffectsStore } from '@/stores/effectsStore';
import { usePlayerStore } from '@/stores/playerStore';
export function EffectsDetail() {
const selectedPresetId = useEffectsStore((s) => s.selectedPresetId);
const isCreatingNew = useEffectsStore((s) => s.isCreatingNew);
const workingChain = useEffectsStore((s) => s.workingChain);
const setWorkingChain = useEffectsStore((s) => s.setWorkingChain);
const setSelectedPresetId = useEffectsStore((s) => s.setSelectedPresetId);
const setIsCreatingNew = useEffectsStore((s) => s.setIsCreatingNew);
const [name, setName] = useState('');
const [description, setDescription] = useState('');
const [saving, setSaving] = useState(false);
const [deleting, setDeleting] = useState(false);
// "Save as Custom" dialog state
const [saveAsDialogOpen, setSaveAsDialogOpen] = useState(false);
const [saveAsName, setSaveAsName] = useState('');
const [saveAsDescription, setSaveAsDescription] = useState('');
// Preview state
const [previewGenId, setPreviewGenId] = useState<string | null>(null);
const [previewLoading, setPreviewLoading] = useState(false);
const blobUrlRef = useRef<string | null>(null);
const setAudioWithAutoPlay = usePlayerStore((s) => s.setAudioWithAutoPlay);
const { toast } = useToast();
const queryClient = useQueryClient();
// Auto-select the most recent generation as preview source
const { data: historyData } = useHistory({ limit: 1 });
useEffect(() => {
if (!previewGenId && historyData?.items?.length) {
const first = historyData.items.find((g) => g.status === 'completed');
if (first) setPreviewGenId(first.id);
}
}, [historyData, previewGenId]);
const { data: preset } = useQuery({
queryKey: ['effect-preset', selectedPresetId],
queryFn: () =>
selectedPresetId
? apiClient
.listEffectPresets()
.then((all) => all.find((p) => p.id === selectedPresetId) ?? null)
: null,
enabled: !!selectedPresetId,
staleTime: 30_000,
});
// Sync name/description when selecting a preset
useEffect(() => {
if (preset) {
setName(preset.name);
setDescription(preset.description ?? '');
} else if (isCreatingNew) {
setName('');
setDescription('');
}
}, [preset, isCreatingNew]);
// Cleanup blob URL on unmount
useEffect(() => {
return () => {
if (blobUrlRef.current) {
URL.revokeObjectURL(blobUrlRef.current);
blobUrlRef.current = null;
}
};
}, []);
const isEditing = !!selectedPresetId || isCreatingNew;
const isBuiltIn = preset?.is_builtin ?? false;
async function handlePreview() {
if (!previewGenId || workingChain.length === 0) return;
setPreviewLoading(true);
try {
const blob = await apiClient.previewEffects(previewGenId, workingChain);
// Revoke old blob URL
if (blobUrlRef.current) {
URL.revokeObjectURL(blobUrlRef.current);
}
const url = URL.createObjectURL(blob);
blobUrlRef.current = url;
// Play through the main audio player
setAudioWithAutoPlay(url, `preview-${Date.now()}`, null, 'Effects Preview');
} catch (error) {
toast({
title: 'Preview failed',
description: error instanceof Error ? error.message : 'Unknown error',
variant: 'destructive',
});
} finally {
setPreviewLoading(false);
}
}
function handleSelectGeneration(gen: HistoryResponse) {
setPreviewGenId(gen.id);
}
async function handleSaveNew() {
if (!name.trim()) {
toast({ title: 'Name required', variant: 'destructive' });
return;
}
setSaving(true);
try {
const created = await apiClient.createEffectPreset({
name: name.trim(),
description: description.trim() || undefined,
effects_chain: workingChain,
});
queryClient.invalidateQueries({ queryKey: ['effect-presets'] });
setIsCreatingNew(false);
setSelectedPresetId(created.id);
toast({ title: 'Preset saved', description: `"${created.name}" has been created.` });
} catch (error) {
toast({
title: 'Failed to save',
description: error instanceof Error ? error.message : 'Unknown error',
variant: 'destructive',
});
} finally {
setSaving(false);
}
}
async function handleSaveExisting() {
if (!selectedPresetId || !name.trim()) return;
setSaving(true);
try {
await apiClient.updateEffectPreset(selectedPresetId, {
name: name.trim(),
description: description.trim() || undefined,
effects_chain: workingChain,
});
queryClient.invalidateQueries({ queryKey: ['effect-presets'] });
queryClient.invalidateQueries({ queryKey: ['effect-preset', selectedPresetId] });
toast({ title: 'Preset updated' });
} catch (error) {
toast({
title: 'Failed to save',
description: error instanceof Error ? error.message : 'Unknown error',
variant: 'destructive',
});
} finally {
setSaving(false);
}
}
function handleSaveAsNew() {
// Open the dialog with a suggested name based on the current preset
setSaveAsName(`${name} (Copy)`);
setSaveAsDescription(description);
setSaveAsDialogOpen(true);
}
async function handleSaveAsConfirm() {
if (!saveAsName.trim()) {
toast({ title: 'Name required', variant: 'destructive' });
return;
}
setSaving(true);
try {
const created = await apiClient.createEffectPreset({
name: saveAsName.trim(),
description: saveAsDescription.trim() || undefined,
effects_chain: workingChain,
});
queryClient.invalidateQueries({ queryKey: ['effect-presets'] });
setSaveAsDialogOpen(false);
setSelectedPresetId(created.id);
toast({ title: 'Preset saved', description: `"${created.name}" has been created.` });
} catch (error) {
toast({
title: 'Failed to save',
description: error instanceof Error ? error.message : 'Unknown error',
variant: 'destructive',
});
} finally {
setSaving(false);
}
}
async function handleDelete() {
if (!selectedPresetId) return;
setDeleting(true);
try {
await apiClient.deleteEffectPreset(selectedPresetId);
queryClient.invalidateQueries({ queryKey: ['effect-presets'] });
setSelectedPresetId(null);
setWorkingChain([]);
toast({ title: 'Preset deleted' });
} catch (error) {
toast({
title: 'Failed to delete',
description: error instanceof Error ? error.message : 'Unknown error',
variant: 'destructive',
});
} finally {
setDeleting(false);
}
}
if (!isEditing) {
return (
<div className="flex-1 flex items-center justify-center text-muted-foreground">
<div className="text-center space-y-2">
<Wand2 className="h-10 w-10 mx-auto opacity-30" />
<p className="text-sm">Select a preset or create a new one</p>
</div>
</div>
);
}
return (
<div className="flex flex-col h-full min-h-0">
{/* Header */}
<div className="flex items-center justify-between mb-4">
<h2 className="text-lg font-semibold">
{isCreatingNew ? 'New Preset' : isBuiltIn ? preset?.name : 'Edit Preset'}
</h2>
<div className="flex items-center gap-2">
{!isBuiltIn && !isCreatingNew && (
<>
<Button
variant="ghost"
size="sm"
className="h-8 text-destructive hover:text-destructive gap-1.5"
onClick={handleDelete}
disabled={deleting}
>
<Trash2 className="h-3.5 w-3.5" />
{deleting ? 'Deleting...' : 'Delete'}
</Button>
<Button
size="sm"
className="h-8 gap-1.5"
onClick={handleSaveExisting}
disabled={saving || workingChain.length === 0}
>
<Save className="h-3.5 w-3.5" />
{saving ? 'Saving...' : 'Save'}
</Button>
</>
)}
{isCreatingNew && (
<Button
size="sm"
className="h-8 gap-1.5"
onClick={handleSaveNew}
disabled={saving || workingChain.length === 0}
>
<Save className="h-3.5 w-3.5" />
{saving ? 'Saving...' : 'Save Preset'}
</Button>
)}
{isBuiltIn && (
<Button
size="sm"
variant="outline"
className="h-8 gap-1.5"
onClick={handleSaveAsNew}
disabled={saving}
>
<Save className="h-3.5 w-3.5" />
{saving ? 'Saving...' : 'Save as Custom'}
</Button>
)}
</div>
</div>
{/* Scrollable content */}
<div className="flex-1 min-h-0 overflow-y-auto space-y-5 pr-1">
{/* Name & description */}
{(isCreatingNew || !isBuiltIn) && (
<div className="space-y-3">
<div className="space-y-1.5">
<Label className="text-xs">Name</Label>
<Input
value={name}
onChange={(e) => setName(e.target.value)}
placeholder="My preset..."
className="h-9"
/>
</div>
<div className="space-y-1.5">
<Label className="text-xs">Description</Label>
<Textarea
value={description}
onChange={(e) => setDescription(e.target.value)}
placeholder="Describe what this preset does..."
className="min-h-[60px] resize-none"
/>
</div>
</div>
)}
{/* Built-in description (read-only) */}
{isBuiltIn && preset?.description && (
<p className="text-sm text-muted-foreground">{preset.description}</p>
)}
{/* Effects chain editor */}
<EffectsChainEditor value={workingChain} onChange={setWorkingChain} showPresets={false} />
<Separator />
{/* Preview section */}
<div className="space-y-3">
<Label className="text-xs">Preview</Label>
<div className="flex items-center gap-2">
<GenerationPicker
selectedId={previewGenId}
onSelect={handleSelectGeneration}
className="flex-1"
/>
<Button
variant="outline"
size="sm"
className="h-8 gap-1.5 shrink-0"
onClick={handlePreview}
disabled={!previewGenId || workingChain.length === 0 || previewLoading}
>
{previewLoading ? (
<>
<Loader2 className="h-3.5 w-3.5 animate-spin" />
Processing...
</>
) : (
<>
<Play className="h-3.5 w-3.5" />
Preview
</>
)}
</Button>
</div>
<p className="text-[11px] text-muted-foreground">
Preview applies effects to the clean version without saving.
</p>
</div>
</div>
{/* Save as Custom dialog */}
<Dialog open={saveAsDialogOpen} onOpenChange={setSaveAsDialogOpen}>
<DialogContent className="sm:max-w-md">
<DialogHeader>
<DialogTitle>Save as Custom Preset</DialogTitle>
<DialogDescription>
Create a new custom preset based on the current effects chain.
</DialogDescription>
</DialogHeader>
<div className="space-y-3 py-2">
<div className="space-y-1.5">
<Label className="text-xs">Name</Label>
<Input
value={saveAsName}
onChange={(e) => setSaveAsName(e.target.value)}
placeholder="My preset..."
className="h-9"
autoFocus
onKeyDown={(e) => {
if (e.key === 'Enter' && saveAsName.trim()) {
handleSaveAsConfirm();
}
}}
/>
</div>
<div className="space-y-1.5">
<Label className="text-xs">Description</Label>
<Textarea
value={saveAsDescription}
onChange={(e) => setSaveAsDescription(e.target.value)}
placeholder="Describe what this preset does..."
className="min-h-[60px] resize-none"
/>
</div>
</div>
<DialogFooter>
<Button variant="outline" onClick={() => setSaveAsDialogOpen(false)} disabled={saving}>
Cancel
</Button>
<Button onClick={handleSaveAsConfirm} disabled={saving || !saveAsName.trim()}>
<Save className="h-3.5 w-3.5 mr-1.5" />
{saving ? 'Saving...' : 'Save'}
</Button>
</DialogFooter>
</DialogContent>
</Dialog>
</div>
);
}
@@ -1,165 +0,0 @@
import { useQuery } from '@tanstack/react-query';
import { Loader2, Plus, Sparkles, Wand2 } from 'lucide-react';
import { Button } from '@/components/ui/button';
import { apiClient } from '@/lib/api/client';
import type { EffectPresetResponse } from '@/lib/api/types';
import { cn } from '@/lib/utils/cn';
import { useEffectsStore } from '@/stores/effectsStore';
export function EffectsList() {
const selectedPresetId = useEffectsStore((s) => s.selectedPresetId);
const setSelectedPresetId = useEffectsStore((s) => s.setSelectedPresetId);
const setWorkingChain = useEffectsStore((s) => s.setWorkingChain);
const setIsCreatingNew = useEffectsStore((s) => s.setIsCreatingNew);
const isCreatingNew = useEffectsStore((s) => s.isCreatingNew);
const { data: presets, isLoading } = useQuery({
queryKey: ['effect-presets'],
queryFn: () => apiClient.listEffectPresets(),
staleTime: 30_000,
});
const builtIn = presets?.filter((p) => p.is_builtin) ?? [];
const userPresets = presets?.filter((p) => !p.is_builtin) ?? [];
function handleSelect(preset: EffectPresetResponse) {
setSelectedPresetId(preset.id);
setWorkingChain(preset.effects_chain);
}
function handleCreateNew() {
setIsCreatingNew(true);
setWorkingChain([]);
}
if (isLoading) {
return (
<div className="flex items-center justify-center h-full">
<Loader2 className="h-6 w-6 animate-spin text-muted-foreground" />
</div>
);
}
return (
<div className="flex flex-col h-full min-h-0">
{/* Header */}
<div className="flex items-center justify-between mb-4">
<h2 className="text-lg font-semibold">Effects</h2>
<Button variant="outline" size="sm" className="h-8 gap-1.5" onClick={handleCreateNew}>
<Plus className="h-3.5 w-3.5" />
New Preset
</Button>
</div>
{/* Scrollable list */}
<div className="flex-1 min-h-0 overflow-y-auto space-y-4">
{/* Built-in presets */}
{builtIn.length > 0 && (
<div>
<div className="text-[11px] text-muted-foreground font-medium uppercase tracking-wider mb-2 px-1">
Built-in
</div>
<div className="space-y-1.5">
{builtIn.map((preset) => (
<PresetCard
key={preset.id}
preset={preset}
isSelected={selectedPresetId === preset.id && !isCreatingNew}
onSelect={() => handleSelect(preset)}
/>
))}
</div>
</div>
)}
{/* User presets */}
{userPresets.length > 0 && (
<div>
<div className="text-[11px] text-muted-foreground font-medium uppercase tracking-wider mb-2 px-1">
Custom
</div>
<div className="space-y-1.5">
{userPresets.map((preset) => (
<PresetCard
key={preset.id}
preset={preset}
isSelected={selectedPresetId === preset.id && !isCreatingNew}
onSelect={() => handleSelect(preset)}
/>
))}
</div>
</div>
)}
{/* New preset placeholder */}
{isCreatingNew && (
<div>
<div className="text-[11px] text-muted-foreground font-medium uppercase tracking-wider mb-2 px-1">
New
</div>
<div className="rounded-xl border-2 border-accent/40 bg-accent/5 p-3">
<div className="flex items-center gap-2">
<Sparkles className="h-4 w-4 text-accent" />
<span className="text-sm font-medium">Unsaved Preset</span>
</div>
<p className="text-xs text-muted-foreground mt-1">
Configure effects in the panel on the right.
</p>
</div>
</div>
)}
</div>
</div>
);
}
function PresetCard({
preset,
isSelected,
onSelect,
}: {
preset: EffectPresetResponse;
isSelected: boolean;
onSelect: () => void;
}) {
const effectCount = preset.effects_chain.length;
return (
<button
type="button"
className={cn(
'w-full text-left rounded-xl border p-3 h-[88px] transition-all duration-150',
isSelected
? 'border-accent/50 bg-accent/10'
: 'border-border bg-card hover:bg-muted/50 hover:border-border',
)}
onClick={onSelect}
>
<div className="flex items-center gap-2">
<Wand2
className={cn('h-4 w-4 shrink-0', isSelected ? 'text-accent' : 'text-muted-foreground')}
/>
<span className="text-sm font-medium truncate">{preset.name}</span>
{preset.is_builtin && (
<span className="text-[10px] bg-muted text-muted-foreground px-1.5 py-0.5 rounded-full shrink-0">
built-in
</span>
)}
</div>
<p className="text-xs text-muted-foreground mt-1 line-clamp-1 pl-6">
{preset.description || 'No description'}
</p>
<div className="flex items-center gap-2 mt-1.5 pl-6">
<span className="text-[10px] text-muted-foreground">
{effectCount} effect{effectCount !== 1 ? 's' : ''}
</span>
<span className="text-[10px] text-muted-foreground/50">
{preset.effects_chain
.filter((e) => e.enabled)
.map((e) => e.type)
.join(' → ')}
</span>
</div>
</button>
);
}
@@ -1,20 +0,0 @@
import {EffectsDetail} from "./EffectsDetail";
import {EffectsList} from "./EffectsList";
export function EffectsTab() {
return (
<div className="flex flex-col h-full min-h-0 overflow-hidden">
<div className="flex-1 min-h-0 flex gap-6 overflow-hidden">
{/* Left - Presets list */}
<div className="w-full max-w-[360px] shrink-0 flex flex-col min-h-0">
<EffectsList />
</div>
{/* Right - Detail / editor */}
<div className="flex-1 min-h-0 flex flex-col">
<EffectsDetail />
</div>
</div>
</div>
);
}
@@ -1,103 +0,0 @@
import type { UseFormReturn } from 'react-hook-form';
import { FormControl } from '@/components/ui/form';
import {
Select,
SelectContent,
SelectItem,
SelectTrigger,
SelectValue,
} from '@/components/ui/select';
import { getLanguageOptionsForEngine } from '@/lib/constants/languages';
import type { GenerationFormValues } from '@/lib/hooks/useGenerationForm';
/**
* Engine/model options and their display metadata.
* Adding a new engine means adding one entry here.
*/
const ENGINE_OPTIONS = [
{ value: 'qwen:1.7B', label: 'Qwen3-TTS 1.7B' },
{ value: 'qwen:0.6B', label: 'Qwen3-TTS 0.6B' },
{ value: 'luxtts', label: 'LuxTTS' },
{ value: 'chatterbox', label: 'Chatterbox' },
{ value: 'chatterbox_turbo', label: 'Chatterbox Turbo' },
] as const;
const ENGINE_DESCRIPTIONS: Record<string, string> = {
qwen: 'Multi-language, two sizes',
luxtts: 'Fast, English-focused',
chatterbox: '23 languages, incl. Hebrew',
chatterbox_turbo: 'English, [laugh] [cough] tags',
};
/** Engines that only support English and should force language to 'en' on select. */
const ENGLISH_ONLY_ENGINES = new Set(['luxtts', 'chatterbox_turbo']);
function getSelectValue(engine: string, modelSize?: string): string {
if (engine === 'qwen') return `qwen:${modelSize || '1.7B'}`;
return engine;
}
function handleEngineChange(form: UseFormReturn<GenerationFormValues>, value: string) {
if (value.startsWith('qwen:')) {
const [, modelSize] = value.split(':');
form.setValue('engine', 'qwen');
form.setValue('modelSize', modelSize as '1.7B' | '0.6B');
// Validate language is supported by Qwen
const currentLang = form.getValues('language');
const available = getLanguageOptionsForEngine('qwen');
if (!available.some((l) => l.value === currentLang)) {
form.setValue('language', available[0]?.value ?? 'en');
}
} else {
form.setValue('engine', value as GenerationFormValues['engine']);
form.setValue('modelSize', undefined as unknown as '1.7B' | '0.6B');
if (ENGLISH_ONLY_ENGINES.has(value)) {
form.setValue('language', 'en');
} else {
// If current language isn't supported by the new engine, reset to first available
const currentLang = form.getValues('language');
const available = getLanguageOptionsForEngine(value);
if (!available.some((l) => l.value === currentLang)) {
form.setValue('language', available[0]?.value ?? 'en');
}
}
}
}
interface EngineModelSelectorProps {
form: UseFormReturn<GenerationFormValues>;
compact?: boolean;
}
export function EngineModelSelector({ form, compact }: EngineModelSelectorProps) {
const engine = form.watch('engine') || 'qwen';
const modelSize = form.watch('modelSize');
const selectValue = getSelectValue(engine, modelSize);
const itemClass = compact ? 'text-xs text-muted-foreground' : undefined;
const triggerClass = compact
? 'h-8 text-xs bg-card border-border rounded-full hover:bg-background/50 transition-all'
: undefined;
return (
<Select value={selectValue} onValueChange={(v) => handleEngineChange(form, v)}>
<FormControl>
<SelectTrigger className={triggerClass}>
<SelectValue />
</SelectTrigger>
</FormControl>
<SelectContent>
{ENGINE_OPTIONS.map((opt) => (
<SelectItem key={opt.value} value={opt.value} className={itemClass}>
{opt.label}
</SelectItem>
))}
</SelectContent>
</Select>
);
}
/** Returns a human-readable description for the currently selected engine. */
export function getEngineDescription(engine: string): string {
return ENGINE_DESCRIPTIONS[engine] ?? '';
}
@@ -1,7 +1,8 @@
import { useQuery } from '@tanstack/react-query';
import { SparklesIcon, TextSquareIcon } from '@hugeicons/core-free-icons';
import { HugeiconsIcon } from '@hugeicons/react';
import { Icon } from '@iconify/react';
import { useMatchRoute } from '@tanstack/react-router';
import { AnimatePresence, motion } from 'framer-motion';
import { Loader2, Sparkles } from 'lucide-react';
import { useEffect, useRef, useState } from 'react';
import { Button } from '@/components/ui/button';
import { Form, FormControl, FormField, FormItem, FormMessage } from '@/components/ui/form';
@@ -13,17 +14,14 @@ import {
SelectValue,
} from '@/components/ui/select';
import { Textarea } from '@/components/ui/textarea';
import { apiClient } from '@/lib/api/client';
import { getLanguageOptionsForEngine, type LanguageCode } from '@/lib/constants/languages';
import { useToast } from '@/components/ui/use-toast';
import { LANGUAGE_OPTIONS } from '@/lib/constants/languages';
import { useGenerationForm } from '@/lib/hooks/useGenerationForm';
import { useProfile, useProfiles } from '@/lib/hooks/useProfiles';
import { useStory } from '@/lib/hooks/useStories';
import { useAddStoryItem, useStory } from '@/lib/hooks/useStories';
import { cn } from '@/lib/utils/cn';
import { useGenerationStore } from '@/stores/generationStore';
import { useStoryStore } from '@/stores/storyStore';
import { useUIStore } from '@/stores/uiStore';
import { EngineModelSelector } from './EngineModelSelector';
import { ParalinguisticInput } from './ParalinguisticInput';
interface FloatingGenerateBoxProps {
isPlayerOpen?: boolean;
@@ -39,7 +37,7 @@ export function FloatingGenerateBox({
const { data: selectedProfile } = useProfile(selectedProfileId || '');
const { data: profiles } = useProfiles();
const [isExpanded, setIsExpanded] = useState(false);
const [selectedPresetId, setSelectedPresetId] = useState<string | null>(null);
const [isInstructMode, setIsInstructMode] = useState(false);
const containerRef = useRef<HTMLDivElement>(null);
const textareaRef = useRef<HTMLTextAreaElement | null>(null);
const matchRoute = useMatchRoute();
@@ -47,13 +45,8 @@ export function FloatingGenerateBox({
const selectedStoryId = useStoryStore((state) => state.selectedStoryId);
const trackEditorHeight = useStoryStore((state) => state.trackEditorHeight);
const { data: currentStory } = useStory(selectedStoryId);
const addPendingStoryAdd = useGenerationStore((s) => s.addPendingStoryAdd);
// Fetch effect presets for the dropdown
const { data: effectPresets } = useQuery({
queryKey: ['effectPresets'],
queryFn: () => apiClient.listEffectPresets(),
});
const addStoryItem = useAddStoryItem();
const { toast } = useToast();
// Calculate if track editor is visible (on stories route with items)
const hasTrackEditor = isStoriesRoute && currentStory && currentStory.items.length > 0;
@@ -61,16 +54,27 @@ export function FloatingGenerateBox({
const { form, handleSubmit, isPending } = useGenerationForm({
onSuccess: async (generationId) => {
setIsExpanded(false);
// Defer the story add until TTS completes -- useGenerationProgress handles it
// If on stories route and a story is selected, add generation to story
if (isStoriesRoute && selectedStoryId && generationId) {
addPendingStoryAdd(generationId, selectedStoryId);
try {
await addStoryItem.mutateAsync({
storyId: selectedStoryId,
data: { generation_id: generationId },
});
toast({
title: 'Added to story',
description: `Generation added to "${currentStory?.name || 'story'}"`,
});
} catch (error) {
toast({
title: 'Failed to add to story',
description:
error instanceof Error ? error.message : 'Could not add generation to story',
variant: 'destructive',
});
}
}
},
getEffectsChain: () => {
if (!selectedPresetId || !effectPresets) return undefined;
const preset = effectPresets.find((p) => p.id === selectedPresetId);
return preset?.effects_chain;
},
});
// Click away handler to collapse the box
@@ -110,13 +114,6 @@ export function FloatingGenerateBox({
}
}, [selectedProfileId, profiles, setSelectedProfileId]);
// Sync generation form language with selected profile's language
useEffect(() => {
if (selectedProfile?.language) {
form.setValue('language', selectedProfile.language as LanguageCode);
}
}, [selectedProfile, form]);
// Auto-resize textarea based on content (only when expanded)
useEffect(() => {
if (!isExpanded) {
@@ -179,7 +176,7 @@ export function FloatingGenerateBox({
isStoriesRoute
? // Position aligned with story list: after sidebar + padding, width 360px
'left-[calc(5rem+2rem)] w-[360px]'
: 'left-[calc(5rem+2rem)] right-8 lg:right-auto lg:w-[calc((100%-5rem-4rem)/2-1rem)]',
: 'left-[calc(5rem+2rem)] w-[calc((100%-5rem-4rem)/2-1rem)]',
)}
style={{
// On stories route: offset by track editor height when visible
@@ -198,46 +195,33 @@ export function FloatingGenerateBox({
<Form {...form}>
<form onSubmit={form.handleSubmit(onSubmit)}>
<div className="flex gap-2">
<motion.div className="flex-1" transition={{ duration: 0.3, ease: 'easeOut' }}>
<FormField
control={form.control}
name="text"
render={({ field }) => (
<FormItem>
<FormControl>
<motion.div
animate={{
height: isExpanded ? 'auto' : '32px',
}}
transition={{ duration: 0.15, ease: 'easeOut' }}
style={{ overflow: 'hidden' }}
>
{form.watch('engine') === 'chatterbox_turbo' ? (
<ParalinguisticInput
value={field.value}
onChange={field.onChange}
placeholder={
isStoriesRoute && currentStory
? `Generate speech for "${currentStory.name}"... (type / for effects)`
: selectedProfile
? `Type / for effects like [laugh], [sigh]...`
: 'Select a voice profile above...'
}
className="px-3 py-2 resize-none bg-transparent border-none focus-visible:ring-0 focus-visible:ring-offset-0 focus:outline-none focus:ring-0 outline-none ring-0 rounded-2xl text-sm w-full"
style={{
minHeight: isExpanded ? '100px' : '32px',
maxHeight: '300px',
overflowY: 'auto',
}}
disabled={!selectedProfileId}
onClick={() => setIsExpanded(true)}
onFocus={() => setIsExpanded(true)}
/>
) : (
<motion.div
className={cn('flex-1', isExpanded && 'mr-12')}
transition={{ duration: 0.3, ease: 'easeOut' }}
>
{/* Text field - hidden when in instruct mode */}
<div style={{ display: isInstructMode ? 'none' : 'block' }}>
<FormField
control={form.control}
name="text"
render={({ field }) => (
<FormItem>
<FormControl>
<motion.div
animate={{
height: isExpanded ? 'auto' : '32px',
}}
transition={{ duration: 0.15, ease: 'easeOut' }}
style={{ overflow: 'hidden' }}
>
<Textarea
{...field}
ref={(node: HTMLTextAreaElement | null) => {
textareaRef.current = node;
// Store ref for auto-resize (only for active field)
if (!isInstructMode) {
textareaRef.current = node;
}
// Forward ref to react-hook-form
if (typeof field.ref === 'function') {
field.ref(node);
}
@@ -258,13 +242,57 @@ export function FloatingGenerateBox({
onClick={() => setIsExpanded(true)}
onFocus={() => setIsExpanded(true)}
/>
)}
</motion.div>
</FormControl>
<FormMessage className="text-xs" />
</FormItem>
)}
/>
</motion.div>
</FormControl>
<FormMessage className="text-xs" />
</FormItem>
)}
/>
</div>
{/* Instruct field - hidden when in text mode */}
<div style={{ display: isInstructMode ? 'block' : 'none' }}>
<FormField
control={form.control}
name="instruct"
render={({ field }) => (
<FormItem>
<FormControl>
<motion.div
animate={{
height: isExpanded ? 'auto' : '32px',
}}
transition={{ duration: 0.15, ease: 'easeOut' }}
style={{ overflow: 'hidden' }}
>
<Textarea
{...field}
ref={(node: HTMLTextAreaElement | null) => {
// Store ref for auto-resize (only for active field)
if (isInstructMode) {
textareaRef.current = node;
}
// Forward ref to react-hook-form
if (typeof field.ref === 'function') {
field.ref(node);
}
}}
placeholder="e.g. very happy and excited"
className="resize-none bg-transparent border-none focus-visible:ring-0 focus-visible:ring-offset-0 focus:outline-none focus:ring-0 outline-none ring-0 rounded-2xl text-sm placeholder:text-muted-foreground/60 w-full"
style={{
minHeight: isExpanded ? '100px' : '32px',
maxHeight: '300px',
}}
disabled={!selectedProfileId}
onClick={() => setIsExpanded(true)}
onFocus={() => setIsExpanded(true)}
/>
</motion.div>
</FormControl>
<FormMessage className="text-xs" />
</FormItem>
)}
/>
</div>
</motion.div>
<div className="relative shrink-0">
@@ -272,20 +300,13 @@ export function FloatingGenerateBox({
<Button
type="submit"
disabled={isPending || !selectedProfileId}
className="h-10 w-10 rounded-full bg-accent hover:bg-accent/90 hover:scale-105 text-accent-foreground shadow-lg hover:shadow-accent/50 transition-all duration-200"
className="h-10 w-10 rounded-full bg-accent hover:bg-accent/90 hover:scale-105 text-accent-foreground shadow-lg transition-all duration-200"
size="icon"
aria-label={
isPending
? 'Generating...'
: !selectedProfileId
? 'Select a voice profile first'
: 'Generate speech'
}
>
{isPending ? (
<Loader2 className="h-4 w-4 animate-spin" />
<Icon icon="svg-spinners:ring-resize" className="h-4 w-4 animate-spin" />
) : (
<Sparkles className="h-4 w-4" />
<HugeiconsIcon icon={SparklesIcon} size={16} className="h-4 w-4" />
)}
</Button>
<span className="pointer-events-none absolute bottom-full left-1/2 -translate-x-1/2 mb-2 whitespace-nowrap rounded-md bg-popover px-3 py-1.5 text-xs text-popover-foreground border border-border opacity-0 transition-opacity group-hover:opacity-100 z-[9999]">
@@ -296,6 +317,37 @@ export function FloatingGenerateBox({
: 'Generate speech'}
</span>
</div>
<AnimatePresence>
{isExpanded && (
<motion.div
initial={{ opacity: 0, scale: 0.8 }}
animate={{ opacity: 1, scale: 1 }}
exit={{ opacity: 0, scale: 0.8 }}
transition={{ duration: 0.2 }}
className="absolute top-0 right-[calc(100%+0.5rem)]"
>
<div className="group relative">
<Button
type="button"
variant="ghost"
size="icon"
onClick={() => setIsInstructMode(!isInstructMode)}
className={cn(
'h-10 w-10 rounded-full transition-all duration-200',
isInstructMode
? 'bg-accent text-accent-foreground border border-accent hover:bg-accent/90'
: 'bg-card border border-border hover:bg-background/50',
)}
>
<HugeiconsIcon icon={TextSquareIcon} size={16} className="h-4 w-4" />
</Button>
<span className="pointer-events-none absolute bottom-full left-1/2 -translate-x-1/2 mb-2 whitespace-nowrap rounded-md bg-popover px-3 py-1.5 text-xs text-popover-foreground border border-border opacity-0 transition-opacity group-hover:opacity-100 z-[9999]">
Fine tune instructions
</span>
</div>
</motion.div>
)}
</AnimatePresence>
</div>
</div>
@@ -331,58 +383,51 @@ export function FloatingGenerateBox({
<FormField
control={form.control}
name="language"
render={({ field }) => {
const engineLangs = getLanguageOptionsForEngine(
form.watch('engine') || 'qwen',
);
return (
<FormItem className="flex-1 space-y-0">
<Select onValueChange={field.onChange} value={field.value}>
<FormControl>
<SelectTrigger className="h-8 text-xs bg-card border-border rounded-full hover:bg-background/50 transition-all">
<SelectValue />
</SelectTrigger>
</FormControl>
<SelectContent>
{engineLangs.map((lang) => (
<SelectItem key={lang.value} value={lang.value} className="text-xs">
{lang.label}
</SelectItem>
))}
</SelectContent>
</Select>
<FormMessage className="text-xs" />
</FormItem>
);
}}
render={({ field }) => (
<FormItem className="flex-1 space-y-0">
<Select onValueChange={field.onChange} defaultValue={field.value}>
<FormControl>
<SelectTrigger className="h-8 text-xs bg-card border-border rounded-full hover:bg-background/50 transition-all">
<SelectValue />
</SelectTrigger>
</FormControl>
<SelectContent>
{LANGUAGE_OPTIONS.map((lang) => (
<SelectItem key={lang.value} value={lang.value} className="text-xs">
{lang.label}
</SelectItem>
))}
</SelectContent>
</Select>
<FormMessage className="text-xs" />
</FormItem>
)}
/>
<FormItem className="flex-1 space-y-0">
<EngineModelSelector form={form} compact />
</FormItem>
<FormItem className="flex-1 space-y-0">
<Select
value={selectedPresetId || 'none'}
onValueChange={(value) =>
setSelectedPresetId(value === 'none' ? null : value)
}
>
<SelectTrigger className="h-8 text-xs bg-card border-border rounded-full hover:bg-background/50 transition-all">
<SelectValue placeholder="No effects" />
</SelectTrigger>
<SelectContent>
<SelectItem value="none" className="text-xs">
No effects
</SelectItem>
{effectPresets?.map((preset) => (
<SelectItem key={preset.id} value={preset.id} className="text-xs">
{preset.name}
</SelectItem>
))}
</SelectContent>
</Select>
</FormItem>
<FormField
control={form.control}
name="modelSize"
render={({ field }) => (
<FormItem className="flex-1 space-y-0">
<Select onValueChange={field.onChange} defaultValue={field.value}>
<FormControl>
<SelectTrigger className="h-8 text-xs bg-card border-border rounded-full hover:bg-background/50 transition-all">
<SelectValue />
</SelectTrigger>
</FormControl>
<SelectContent>
<SelectItem value="1.7B" className="text-xs text-muted-foreground">
Qwen3-TTS 1.7B
</SelectItem>
<SelectItem value="0.6B" className="text-xs text-muted-foreground">
Qwen3-TTS 0.6B
</SelectItem>
</SelectContent>
</Select>
<FormMessage className="text-xs" />
</FormItem>
)}
/>
</div>
</motion.div>
</AnimatePresence>
@@ -1,4 +1,6 @@
import { Loader2, Mic } from 'lucide-react';
import { Mic01Icon } from '@hugeicons/core-free-icons';
import { HugeiconsIcon } from '@hugeicons/react';
import { Icon } from '@iconify/react';
import { Button } from '@/components/ui/button';
import { Card, CardContent, CardHeader, CardTitle } from '@/components/ui/card';
import {
@@ -19,12 +21,10 @@ import {
SelectValue,
} from '@/components/ui/select';
import { Textarea } from '@/components/ui/textarea';
import { getLanguageOptionsForEngine } from '@/lib/constants/languages';
import { LANGUAGE_OPTIONS } from '@/lib/constants/languages';
import { useGenerationForm } from '@/lib/hooks/useGenerationForm';
import { useProfile } from '@/lib/hooks/useProfiles';
import { useUIStore } from '@/stores/uiStore';
import { EngineModelSelector, getEngineDescription } from './EngineModelSelector';
import { ParalinguisticInput } from './ParalinguisticInput';
export function GenerationForm() {
const selectedProfileId = useUIStore((state) => state.selectedProfileId);
@@ -48,7 +48,11 @@ export function GenerationForm() {
<FormLabel>Voice Profile</FormLabel>
{selectedProfile ? (
<div className="mt-2 p-3 border rounded-md bg-muted/50 flex items-center gap-2">
<Mic className="h-4 w-4 text-muted-foreground" />
<HugeiconsIcon
icon={Mic01Icon}
size={16}
className="h-4 w-4 text-muted-foreground"
/>
<span className="font-medium">{selectedProfile.name}</span>
<span className="text-sm text-muted-foreground">{selectedProfile.language}</span>
</div>
@@ -66,90 +70,87 @@ export function GenerationForm() {
<FormItem>
<FormLabel>Text to Speak</FormLabel>
<FormControl>
{form.watch('engine') === 'chatterbox_turbo' ? (
<ParalinguisticInput
value={field.value}
onChange={field.onChange}
placeholder="Enter text... type / for effects like [laugh], [sigh]"
className="min-h-[150px] rounded-md border border-input bg-background px-3 py-2"
/>
) : (
<Textarea
placeholder="Enter the text you want to generate..."
className="min-h-[150px]"
{...field}
/>
)}
<Textarea
placeholder="Enter the text you want to generate..."
className="min-h-[150px]"
{...field}
/>
</FormControl>
<FormDescription>Max 5000 characters</FormDescription>
<FormMessage />
</FormItem>
)}
/>
<FormField
control={form.control}
name="instruct"
render={({ field }) => (
<FormItem>
<FormLabel>Delivery Instructions (optional)</FormLabel>
<FormControl>
<Textarea
placeholder="e.g. Speak slowly with emphasis, Warm and friendly tone, Professional and authoritative..."
className="min-h-[80px]"
{...field}
/>
</FormControl>
<FormDescription>
{form.watch('engine') === 'chatterbox_turbo'
? 'Max 5000 characters. Type / to insert sound effects.'
: 'Max 5000 characters'}
Natural language instructions to control speech delivery (tone, emotion, pace).
Max 500 characters
</FormDescription>
<FormMessage />
</FormItem>
)}
/>
{form.watch('engine') === 'qwen' && (
<div className="grid gap-4 md:grid-cols-3">
<FormField
control={form.control}
name="instruct"
name="language"
render={({ field }) => (
<FormItem>
<FormLabel>Delivery Instructions (optional)</FormLabel>
<FormControl>
<Textarea
placeholder="e.g. Speak slowly with emphasis, Warm and friendly tone, Professional and authoritative..."
className="min-h-[80px]"
{...field}
/>
</FormControl>
<FormDescription>
Natural language instructions to control speech delivery (tone, emotion,
pace). Max 500 characters
</FormDescription>
<FormLabel>Language</FormLabel>
<Select onValueChange={field.onChange} defaultValue={field.value}>
<FormControl>
<SelectTrigger>
<SelectValue />
</SelectTrigger>
</FormControl>
<SelectContent>
{LANGUAGE_OPTIONS.map((lang) => (
<SelectItem key={lang.value} value={lang.value}>
{lang.label}
</SelectItem>
))}
</SelectContent>
</Select>
<FormMessage />
</FormItem>
)}
/>
)}
<div className="grid gap-4 md:grid-cols-3">
<FormItem>
<FormLabel>Model</FormLabel>
<EngineModelSelector form={form} />
<FormDescription>
{getEngineDescription(form.watch('engine') || 'qwen')}
</FormDescription>
</FormItem>
<FormField
control={form.control}
name="language"
render={({ field }) => {
const engineLangs = getLanguageOptionsForEngine(form.watch('engine') || 'qwen');
return (
<FormItem>
<FormLabel>Language</FormLabel>
<Select onValueChange={field.onChange} value={field.value}>
<FormControl>
<SelectTrigger>
<SelectValue />
</SelectTrigger>
</FormControl>
<SelectContent>
{engineLangs.map((lang) => (
<SelectItem key={lang.value} value={lang.value}>
{lang.label}
</SelectItem>
))}
</SelectContent>
</Select>
<FormMessage />
</FormItem>
);
}}
name="modelSize"
render={({ field }) => (
<FormItem>
<FormLabel>Model Size</FormLabel>
<Select onValueChange={field.onChange} defaultValue={field.value}>
<FormControl>
<SelectTrigger>
<SelectValue />
</SelectTrigger>
</FormControl>
<SelectContent>
<SelectItem value="1.7B">Qwen TTS 1.7B (Higher Quality)</SelectItem>
<SelectItem value="0.6B">Qwen TTS 0.6B (Faster)</SelectItem>
</SelectContent>
</Select>
<FormDescription>Larger models produce better quality</FormDescription>
<FormMessage />
</FormItem>
)}
/>
<FormField
@@ -178,7 +179,7 @@ export function GenerationForm() {
<Button type="submit" className="w-full" disabled={isPending || !selectedProfileId}>
{isPending ? (
<>
<Loader2 className="mr-2 h-4 w-4 animate-spin" />
<Icon icon="svg-spinners:ring-resize" className="mr-2 h-4 w-4 animate-spin" />
Generating...
</>
) : (
@@ -1,422 +0,0 @@
/**
* ParalinguisticInput — a contentEditable rich text input that renders
* Chatterbox Turbo paralinguistic tags (e.g. [laugh]) as inline badges.
*
* Trigger: typing "/" opens an autocomplete dropdown.
* Paste: pasting text with [tag] patterns auto-converts to badges.
* Output: serializes badges back to plain [tag] text for the API.
*/
import { AnimatePresence, motion } from 'framer-motion';
import { forwardRef, useCallback, useEffect, useImperativeHandle, useRef, useState } from 'react';
import { createPortal } from 'react-dom';
import { cn } from '@/lib/utils/cn';
// ── Tag definitions ─────────────────────────────────────────────────
const PARALINGUISTIC_TAGS = [
{ tag: '[laugh]', label: 'laugh', emoji: '\u{1F602}' },
{ tag: '[chuckle]', label: 'chuckle', emoji: '\u{1F60F}' },
{ tag: '[gasp]', label: 'gasp', emoji: '\u{1F62E}' },
{ tag: '[cough]', label: 'cough', emoji: '\u{1F637}' },
{ tag: '[sigh]', label: 'sigh', emoji: '\u{1F614}' },
{ tag: '[groan]', label: 'groan', emoji: '\u{1F629}' },
{ tag: '[sniff]', label: 'sniff', emoji: '\u{1F443}' },
{ tag: '[shush]', label: 'shush', emoji: '\u{1F92B}' },
{ tag: '[clear throat]', label: 'clear throat', emoji: '\u{1F64A}' },
] as const;
const TAG_REGEX = /\[(laugh|chuckle|gasp|cough|sigh|groan|sniff|shush|clear throat)\]/gi;
// Data attribute used to identify badge spans in the DOM
const BADGE_ATTR = 'data-ptag';
// ── Helpers ─────────────────────────────────────────────────────────
/** Build an inline badge <span> for a tag. */
function makeBadgeHTML(tag: string): string {
const entry = PARALINGUISTIC_TAGS.find((t) => t.tag.toLowerCase() === tag.toLowerCase());
const label = entry?.label ?? tag.replace(/[[\]]/g, '');
const emoji = entry?.emoji ?? '';
// Non-editable inline badge. Zero-width spaces around it let the
// caret sit on either side so the user can type before/after.
return `\u200B<span ${BADGE_ATTR}="${tag}" contenteditable="false" class="ptag-badge">${emoji ? `${emoji}\u00A0` : ''}${label}</span>\u200B`;
}
/** Convert plain text with [tag] patterns into HTML with badge spans. */
function textToHTML(text: string): string {
// Escape HTML entities first
const escaped = text.replace(/&/g, '&amp;').replace(/</g, '&lt;').replace(/>/g, '&gt;');
// Replace tag patterns with badge HTML
return escaped.replace(TAG_REGEX, (match) => makeBadgeHTML(match));
}
/** Serialize the contentEditable innerHTML back to plain text with [tag] syntax. */
function htmlToText(container: HTMLElement): string {
let result = '';
for (const node of container.childNodes) {
if (node.nodeType === Node.TEXT_NODE) {
// Strip zero-width spaces we added around badges
result += (node.textContent ?? '').replace(/\u200B/g, '');
} else if (node.nodeType === Node.ELEMENT_NODE) {
const el = node as HTMLElement;
if (el.hasAttribute(BADGE_ATTR)) {
result += el.getAttribute(BADGE_ATTR) ?? '';
} else if (el.tagName === 'BR') {
result += '\n';
} else {
// Recurse for nested elements (e.g. spans from paste)
result += htmlToText(el);
}
}
}
return result;
}
/** Get the text content from the current caret position back to the last
* whitespace or start of container, to detect the "/" trigger. */
function getWordBeforeCaret(_container: HTMLElement): { word: string; range: Range | null } {
const sel = window.getSelection();
if (!sel || sel.rangeCount === 0) return { word: '', range: null };
const range = sel.getRangeAt(0).cloneRange();
range.collapse(true);
// Walk backwards from caret through the text node
const textNode = range.startContainer;
if (textNode.nodeType !== Node.TEXT_NODE) return { word: '', range: null };
const text = textNode.textContent ?? '';
const offset = range.startOffset;
let start = offset;
while (
start > 0 &&
text[start - 1] !== ' ' &&
text[start - 1] !== '\n' &&
text[start - 1] !== '\u00A0'
) {
start--;
}
const word = text.slice(start, offset);
const wordRange = document.createRange();
wordRange.setStart(textNode, start);
wordRange.setEnd(textNode, offset);
return { word, range: wordRange };
}
// ── Component ───────────────────────────────────────────────────────
export interface ParalinguisticInputProps {
value?: string;
onChange?: (value: string) => void;
placeholder?: string;
disabled?: boolean;
className?: string;
style?: React.CSSProperties;
onClick?: () => void;
onFocus?: () => void;
}
export interface ParalinguisticInputRef {
focus: () => void;
element: HTMLDivElement | null;
}
export const ParalinguisticInput = forwardRef<ParalinguisticInputRef, ParalinguisticInputProps>(
function ParalinguisticInput(
{ value, onChange, placeholder, disabled, className, style, onClick, onFocus },
ref,
) {
const editorRef = useRef<HTMLDivElement>(null);
const [showMenu, setShowMenu] = useState(false);
const [menuFilter, setMenuFilter] = useState('');
const [menuIndex, setMenuIndex] = useState(0);
const [menuPosition, setMenuPosition] = useState<{ bottom: number; left: number }>({
bottom: 0,
left: 0,
});
const triggerRangeRef = useRef<Range | null>(null);
const lastSerializedRef = useRef<string>('');
const isComposingRef = useRef(false);
useImperativeHandle(ref, () => ({
focus: () => editorRef.current?.focus(),
element: editorRef.current,
}));
// Filtered tag list for the autocomplete menu
const filteredTags = PARALINGUISTIC_TAGS.filter((t) =>
t.label.toLowerCase().includes(menuFilter.toLowerCase()),
);
// ── Sync external value → editor ──────────────────────────────
useEffect(() => {
const el = editorRef.current;
if (!el) return;
// Only update DOM if the external value differs from what we last emitted
if (value !== undefined && value !== lastSerializedRef.current) {
lastSerializedRef.current = value;
el.innerHTML = value ? textToHTML(value) : '';
}
}, [value]);
// ── Emit plain-text value on input ────────────────────────────
const emitChange = useCallback(() => {
const el = editorRef.current;
if (!el || !onChange) return;
const text = htmlToText(el);
lastSerializedRef.current = text;
onChange(text);
}, [onChange]);
// ── Insert a tag badge at the caret ───────────────────────────
const insertTag = useCallback(
(tag: string) => {
const el = editorRef.current;
if (!el) return;
// Delete the /filter text
const wordRange = triggerRangeRef.current;
if (wordRange) {
wordRange.deleteContents();
}
// Insert badge HTML
const temp = document.createElement('span');
temp.innerHTML = makeBadgeHTML(tag);
const frag = document.createDocumentFragment();
let lastNode: Node | null = null;
while (temp.firstChild) {
lastNode = frag.appendChild(temp.firstChild);
}
const sel = window.getSelection();
if (sel && sel.rangeCount > 0) {
const range = sel.getRangeAt(0);
range.deleteContents();
range.insertNode(frag);
// Move caret after the badge
if (lastNode) {
const newRange = document.createRange();
newRange.setStartAfter(lastNode);
newRange.collapse(true);
sel.removeAllRanges();
sel.addRange(newRange);
}
}
setShowMenu(false);
setMenuFilter('');
emitChange();
el.focus();
},
[emitChange],
);
// ── Handle keydown for autocomplete navigation ────────────────
const handleKeyDown = useCallback(
(e: React.KeyboardEvent) => {
if (showMenu) {
if (filteredTags.length === 0) {
if (e.key === 'Escape') {
e.preventDefault();
setShowMenu(false);
}
return;
}
if (e.key === 'ArrowDown') {
e.preventDefault();
setMenuIndex((i) => (i + 1) % filteredTags.length);
} else if (e.key === 'ArrowUp') {
e.preventDefault();
setMenuIndex((i) => (i - 1 + filteredTags.length) % filteredTags.length);
} else if (e.key === 'Enter' || e.key === 'Tab') {
e.preventDefault();
if (filteredTags[menuIndex]) {
insertTag(filteredTags[menuIndex].tag);
}
} else if (e.key === 'Escape') {
e.preventDefault();
setShowMenu(false);
}
} else {
// Prevent Enter from creating <div> blocks in contentEditable
if (e.key === 'Enter' && !e.shiftKey) {
// Let the form handle submit
}
}
},
[showMenu, filteredTags, menuIndex, insertTag],
);
// ── Handle input (check for / trigger) ────────────────────────
const handleInput = useCallback(() => {
if (isComposingRef.current) return;
const el = editorRef.current;
if (!el) return;
const { word, range } = getWordBeforeCaret(el);
if (word.startsWith('/')) {
const filter = word.slice(1); // strip the /
setMenuFilter(filter);
setMenuIndex(0);
triggerRangeRef.current = range;
// Position the menu above the caret using viewport coords (portalled)
const sel = window.getSelection();
if (sel && sel.rangeCount > 0) {
const rect = sel.getRangeAt(0).getBoundingClientRect();
setMenuPosition({
bottom: window.innerHeight - rect.top + 4,
left: rect.left,
});
}
setShowMenu(true);
} else {
setShowMenu(false);
}
emitChange();
}, [emitChange]);
// ── Handle paste — convert [tag] patterns to badges ───────────
const handlePaste = useCallback(
(e: React.ClipboardEvent) => {
e.preventDefault();
const text = e.clipboardData.getData('text/plain');
if (!text) return;
const el = editorRef.current;
if (!el) return;
const html = textToHTML(text);
// Insert at caret
const sel = window.getSelection();
if (sel && sel.rangeCount > 0) {
const range = sel.getRangeAt(0);
range.deleteContents();
const temp = document.createElement('div');
temp.innerHTML = html;
const frag = document.createDocumentFragment();
let lastNode: Node | null = null;
while (temp.firstChild) {
lastNode = frag.appendChild(temp.firstChild);
}
range.insertNode(frag);
if (lastNode) {
const newRange = document.createRange();
newRange.setStartAfter(lastNode);
newRange.collapse(true);
sel.removeAllRanges();
sel.addRange(newRange);
}
}
emitChange();
},
[emitChange],
);
// ── Show placeholder ──────────────────────────────────────────
const isEmpty = !value || value.trim() === '';
return (
<div className="relative">
{/* Placeholder */}
{isEmpty && placeholder && (
<div
className="pointer-events-none absolute inset-0 text-sm text-muted-foreground/60 px-3 py-2 select-none"
aria-hidden
>
{placeholder}
</div>
)}
{/* Editable area */}
<div
ref={editorRef}
contentEditable={!disabled}
suppressContentEditableWarning
role={disabled ? undefined : 'textbox'}
aria-multiline={disabled ? undefined : true}
aria-placeholder={placeholder}
aria-disabled={disabled}
tabIndex={disabled ? -1 : 0}
className={cn(
'min-h-[32px] text-sm whitespace-pre-wrap break-words outline-none',
'[&_.ptag-badge]:inline-flex [&_.ptag-badge]:items-center [&_.ptag-badge]:rounded-full',
'[&_.ptag-badge]:bg-accent/20 [&_.ptag-badge]:text-accent [&_.ptag-badge]:border [&_.ptag-badge]:border-accent/30',
'[&_.ptag-badge]:px-2 [&_.ptag-badge]:py-0 [&_.ptag-badge]:text-xs [&_.ptag-badge]:font-medium',
'[&_.ptag-badge]:mx-0.5 [&_.ptag-badge]:select-none [&_.ptag-badge]:cursor-default',
'[&_.ptag-badge]:align-baseline',
disabled && 'opacity-50 cursor-not-allowed',
className,
)}
style={style}
onInput={!disabled ? handleInput : undefined}
onKeyDown={!disabled ? handleKeyDown : undefined}
onPaste={!disabled ? handlePaste : undefined}
onClick={!disabled ? onClick : undefined}
onFocus={!disabled ? onFocus : undefined}
onBlur={() => {
setShowMenu(false);
triggerRangeRef.current = null;
}}
onCompositionStart={() => {
isComposingRef.current = true;
}}
onCompositionEnd={() => {
isComposingRef.current = false;
handleInput();
}}
/>
{/* Autocomplete dropdown — portalled to body, positioned above the caret */}
{showMenu &&
filteredTags.length > 0 &&
createPortal(
<AnimatePresence>
<motion.div
initial={{ opacity: 0, y: 4 }}
animate={{ opacity: 1, y: 0 }}
exit={{ opacity: 0, y: 4 }}
transition={{ duration: 0.12 }}
className="fixed z-[9999] min-w-[200px] max-h-[280px] overflow-y-auto rounded-lg border border-border bg-popover shadow-lg"
style={{
bottom: menuPosition.bottom,
left: menuPosition.left,
}}
>
{filteredTags.map((t, i) => (
<button
key={t.tag}
type="button"
className={cn(
'flex items-center gap-2 w-full px-3 py-1.5 text-sm text-left transition-colors',
i === menuIndex
? 'bg-accent/20 text-accent-foreground'
: 'text-popover-foreground hover:bg-muted/50',
)}
onMouseDown={(e) => {
e.preventDefault(); // Keep focus in editor
insertTag(t.tag);
}}
onMouseEnter={() => setMenuIndex(i)}
>
<span className="text-base leading-none">{t.emoji}</span>
<span>{t.label}</span>
<span className="ml-auto text-xs text-muted-foreground font-mono">{t.tag}</span>
</button>
))}
</motion.div>
</AnimatePresence>,
document.body,
)}
</div>
);
},
);
+106 -455
View File
@@ -1,22 +1,14 @@
import { useQueryClient } from '@tanstack/react-query';
import { AnimatePresence, motion } from 'framer-motion';
import {
AlignCenter,
AudioLines,
AudioWaveform,
Download,
FileArchive,
Loader2,
MoreHorizontal,
Play,
RotateCcw,
Star,
Trash2,
Wand2,
} from 'lucide-react';
Archive01Icon,
Delete01Icon,
Download01Icon,
MoreHorizontalIcon,
PlayIcon,
WaveIcon,
} from '@hugeicons/core-free-icons';
import { HugeiconsIcon } from '@hugeicons/react';
import { Icon } from '@iconify/react';
import { useEffect, useRef, useState } from 'react';
import Loader from 'react-loaders';
import { EffectsChainEditor } from '@/components/Effects/EffectsChainEditor';
import { Button } from '@/components/ui/button';
import {
Dialog,
@@ -32,17 +24,10 @@ import {
DropdownMenuItem,
DropdownMenuTrigger,
} from '@/components/ui/dropdown-menu';
import {
Select,
SelectContent,
SelectItem,
SelectTrigger,
SelectValue,
} from '@/components/ui/select';
import { Textarea } from '@/components/ui/textarea';
import { useToast } from '@/components/ui/use-toast';
import { apiClient } from '@/lib/api/client';
import type { EffectConfig, GenerationVersionResponse, HistoryResponse } from '@/lib/api/types';
import type { HistoryResponse } from '@/lib/api/types';
import { BOTTOM_SAFE_AREA_PADDING } from '@/lib/constants/ui';
import {
useDeleteGeneration,
@@ -52,8 +37,7 @@ import {
useImportGeneration,
} from '@/lib/hooks/useHistory';
import { cn } from '@/lib/utils/cn';
import { formatDate, formatDuration, formatEngineName } from '@/lib/utils/format';
import { useGenerationStore } from '@/stores/generationStore';
import { formatDate, formatDuration } from '@/lib/utils/format';
import { usePlayerStore } from '@/stores/playerStore';
// OLD TABLE-BASED COMPONENT - REMOVED (can be found in git history)
@@ -74,18 +58,8 @@ export function HistoryTable() {
const [generationToDelete, setGenerationToDelete] = useState<{ id: string; name: string } | null>(
null,
);
const [effectsDialogOpen, setEffectsDialogOpen] = useState(false);
const [effectsTargetId, setEffectsTargetId] = useState<string | null>(null);
const [effectsTargetVersions, setEffectsTargetVersions] = useState<GenerationVersionResponse[]>(
[],
);
const [effectsSourceVersionId, setEffectsSourceVersionId] = useState<string | null>(null);
const [effectsChain, setEffectsChain] = useState<EffectConfig[]>([]);
const [applyingEffects, setApplyingEffects] = useState(false);
const [expandedVersionsId, setExpandedVersionsId] = useState<string | null>(null);
const limit = 20;
const { toast } = useToast();
const queryClient = useQueryClient();
const {
data: historyData,
@@ -100,7 +74,6 @@ export function HistoryTable() {
const exportGeneration = useExportGeneration();
const exportGenerationAudio = useExportGenerationAudio();
const importGeneration = useImportGeneration();
const addPendingGeneration = useGenerationStore((state) => state.addPendingGeneration);
const setAudioWithAutoPlay = usePlayerStore((state) => state.setAudioWithAutoPlay);
const restartCurrentAudio = usePlayerStore((state) => state.restartCurrentAudio);
const currentAudioId = usePlayerStore((state) => state.audioId);
@@ -224,120 +197,6 @@ export function HistoryTable() {
}
};
const handleRetry = async (generationId: string) => {
try {
const result = await apiClient.retryGeneration(generationId);
addPendingGeneration(result.id);
queryClient.invalidateQueries({ queryKey: ['history'] });
} catch (error) {
toast({
title: 'Retry failed',
description: error instanceof Error ? error.message : 'Could not retry generation',
variant: 'destructive',
});
}
};
const handleRegenerate = async (generationId: string) => {
try {
await apiClient.regenerateGeneration(generationId);
addPendingGeneration(generationId);
queryClient.invalidateQueries({ queryKey: ['history'] });
} catch (error) {
toast({
title: 'Regenerate failed',
description: error instanceof Error ? error.message : 'Could not regenerate',
variant: 'destructive',
});
}
};
const handleToggleFavorite = async (generationId: string) => {
try {
await apiClient.toggleFavorite(generationId);
queryClient.invalidateQueries({ queryKey: ['history'] });
} catch (error) {
toast({
title: 'Failed to update favorite',
description: error instanceof Error ? error.message : 'Unknown error',
variant: 'destructive',
});
}
};
const handleApplyEffects = (generationId: string) => {
const gen = allHistory.find((g) => g.id === generationId);
const versions = gen?.versions ?? [];
setEffectsTargetId(generationId);
setEffectsTargetVersions(versions);
// Default to clean/original version (no effects chain)
const cleanVersion = versions.find((v) => !v.effects_chain || v.effects_chain.length === 0);
setEffectsSourceVersionId(cleanVersion?.id ?? null);
setEffectsChain([]);
setEffectsDialogOpen(true);
};
const handleApplyEffectsConfirm = async () => {
if (!effectsTargetId || effectsChain.length === 0) return;
setApplyingEffects(true);
try {
const newVersion = await apiClient.applyEffectsToGeneration(effectsTargetId, {
effects_chain: effectsChain,
source_version_id: effectsSourceVersionId ?? undefined,
set_as_default: true,
});
queryClient.invalidateQueries({ queryKey: ['history'] });
// If the player is currently on this generation, reload with the new version audio
if (currentAudioId === effectsTargetId) {
const gen = allHistory.find((g) => g.id === effectsTargetId);
if (gen) {
const versionUrl = apiClient.getVersionAudioUrl(newVersion.id);
setAudioWithAutoPlay(
versionUrl,
effectsTargetId,
gen.profile_id,
gen.text.substring(0, 50),
);
}
}
setEffectsDialogOpen(false);
toast({ title: 'Effects applied', description: 'A new version has been created.' });
} catch (error) {
toast({
title: 'Failed to apply effects',
description: error instanceof Error ? error.message : 'Unknown error',
variant: 'destructive',
});
} finally {
setApplyingEffects(false);
}
};
const handleSwitchVersion = async (generationId: string, versionId: string) => {
try {
await apiClient.setDefaultVersion(generationId, versionId);
queryClient.invalidateQueries({ queryKey: ['history'] });
} catch (error) {
toast({
title: 'Failed to switch version',
description: error instanceof Error ? error.message : 'Unknown error',
variant: 'destructive',
});
}
};
const handlePlayVersion = (
generationId: string,
versionId: string,
text: string,
profileId: string,
) => {
const audioUrl = apiClient.getVersionAudioUrl(versionId);
setAudioWithAutoPlay(audioUrl, generationId, profileId, text.substring(0, 50));
};
const handleImportConfirm = () => {
if (selectedFile) {
importGeneration.mutate(selectedFile, {
@@ -366,7 +225,10 @@ export function HistoryTable() {
if (isLoading && page === 0) {
return (
<div className="flex items-center justify-center h-full">
<Loader2 className="h-8 w-8 animate-spin text-muted-foreground" />
<Icon
icon="svg-spinners:ring-resize"
className="h-8 w-8 animate-spin text-muted-foreground"
/>
</div>
);
}
@@ -394,269 +256,104 @@ export function HistoryTable() {
>
{history.map((gen) => {
const isCurrentlyPlaying = currentAudioId === gen.id && isPlaying;
const isInProgress = gen.status === 'loading_model' || gen.status === 'generating';
const isGenerating = isInProgress;
const isFailed = gen.status === 'failed';
const isPlayable = !isGenerating && !isFailed;
const hasVersions = gen.versions && gen.versions.length > 1;
const isVersionsExpanded = expandedVersionsId === gen.id;
return (
<div
key={gen.id}
className={cn(
'border rounded-md bg-card transition-colors text-left w-full',
'flex items-stretch gap-4 h-26 border rounded-md p-3 bg-card hover:bg-muted/70 transition-colors text-left w-full',
isCurrentlyPlaying && 'bg-muted/70',
)}
>
{/* Main row */}
<div
role={isPlayable ? 'button' : undefined}
tabIndex={isPlayable ? 0 : undefined}
className={cn(
'flex items-stretch gap-4 h-26 p-3',
isPlayable && 'hover:bg-muted/70 cursor-pointer rounded-md',
isVersionsExpanded && 'rounded-b-none',
)}
aria-label={
isGenerating
? `Generating speech for ${gen.profile_name}...`
: isFailed
? `Generation failed for ${gen.profile_name}`
: isCurrentlyPlaying
? `Sample from ${gen.profile_name}, ${formatDuration(gen.duration ?? 0)}, ${formatDate(gen.created_at)}. Playing. Press Enter to restart.`
: `Sample from ${gen.profile_name}, ${formatDuration(gen.duration ?? 0)}, ${formatDate(gen.created_at)}. Press Enter to play.`
onMouseDown={(e) => {
// Don't trigger play if clicking on textarea or if text is selected
const target = e.target as HTMLElement;
if (target.closest('textarea') || window.getSelection()?.toString()) {
return;
}
onMouseDown={(e) => {
if (!isPlayable) return;
const target = e.target as HTMLElement;
if (target.closest('textarea') || window.getSelection()?.toString()) {
return;
}
handlePlay(gen.id, gen.text, gen.profile_id);
}}
onKeyDown={(e) => {
if (!isPlayable) return;
const target = e.target as HTMLElement;
if (target.closest('textarea') || target.closest('button')) return;
if (e.key === 'Enter' || e.key === ' ') {
e.preventDefault();
handlePlay(gen.id, gen.text, gen.profile_id);
}
}}
>
{/* Status icon */}
<div className="flex items-center shrink-0 w-10 justify-center overflow-hidden">
<div className="scale-50">
<Loader
type={isGenerating ? 'line-scale' : 'line-scale-pulse-out-rapid'}
active={isGenerating || isCurrentlyPlaying}
/>
</div>
handlePlay(gen.id, gen.text, gen.profile_id);
}}
>
{/* Waveform icon */}
<div className="flex items-center shrink-0">
<HugeiconsIcon
icon={WaveIcon}
size={20}
className="h-5 w-5 text-muted-foreground"
/>
</div>
{/* Left side - Meta information */}
<div className="flex flex-col gap-1.5 w-48 shrink-0 justify-center">
<div className="font-medium text-sm truncate" title={gen.profile_name}>
{gen.profile_name}
</div>
{/* Left side - Meta information */}
<div className="flex flex-col gap-1.5 w-48 shrink-0 justify-center">
<div className="font-medium text-sm truncate" title={gen.profile_name}>
{gen.profile_name}
</div>
<div className="flex items-center gap-2">
<span className="text-xs text-muted-foreground">{gen.language}</span>
<span className="text-xs text-muted-foreground">
{formatEngineName(gen.engine, gen.model_size)}
</span>
{isFailed ? (
<span className="text-xs text-destructive">Failed</span>
) : !isGenerating ? (
<span className="text-xs text-muted-foreground">
{formatDuration(gen.duration ?? 0)}
</span>
) : null}
</div>
<div className="text-xs text-muted-foreground">
{isInProgress ? (
<span className="text-accent">
{gen.status === 'loading_model' ? 'Loading model...' : 'Generating...'}
</span>
) : (
formatDate(gen.created_at)
)}
</div>
<div className="flex items-center gap-2">
<span className="text-xs text-muted-foreground">{gen.language}</span>
<span className="text-xs text-muted-foreground">
{formatDuration(gen.duration)}
</span>
</div>
{/* Right side - Transcript textarea */}
<div className="flex-1 min-w-0 flex">
<Textarea
value={gen.text}
className="flex-1 resize-none text-sm text-muted-foreground select-text"
readOnly
aria-label={`Transcript for sample from ${gen.profile_name}, ${formatDuration(gen.duration ?? 0)}`}
/>
</div>
{/* Far right - Actions */}
<div
className="shrink-0 flex flex-col justify-center items-center gap-0.5"
onMouseDown={(e) => e.stopPropagation()}
onClick={(e) => e.stopPropagation()}
>
<Button
variant="ghost"
size="icon"
className={cn(
'h-6 w-6 text-muted-foreground/50 hover:bg-muted-foreground/20 hover:text-muted-foreground',
gen.is_favorited && 'text-accent hover:text-accent',
)}
aria-label={gen.is_favorited ? 'Unfavorite' : 'Favorite'}
onClick={() => handleToggleFavorite(gen.id)}
>
<Star
className="h-2 w-2"
fill={gen.is_favorited ? 'currentColor' : 'none'}
/>
</Button>
{hasVersions && (
<Button
variant="ghost"
size="icon"
className={cn(
'h-6 w-6 text-muted-foreground/50 hover:bg-muted-foreground/20 hover:text-muted-foreground',
isVersionsExpanded && 'text-accent hover:text-accent',
)}
aria-label="Toggle versions"
onClick={() => setExpandedVersionsId(isVersionsExpanded ? null : gen.id)}
>
<AudioLines className="h-2 w-2" />
</Button>
)}
{isFailed ? (
<Button
variant="ghost"
size="icon"
className="h-6 w-6 text-muted-foreground/50 hover:bg-muted-foreground/20 hover:text-muted-foreground"
aria-label="Retry generation"
onClick={() => handleRetry(gen.id)}
>
<RotateCcw className="h-2 w-2" />
</Button>
) : (
<>
<DropdownMenu>
<DropdownMenuTrigger asChild>
<Button
variant="ghost"
size="icon"
className="h-6 w-6 text-muted-foreground/50 hover:bg-muted-foreground/20 hover:text-muted-foreground"
aria-label="Actions"
disabled={isGenerating}
>
<MoreHorizontal className="h-2 w-2" />
</Button>
</DropdownMenuTrigger>
<DropdownMenuContent align="end">
<DropdownMenuItem
onClick={() => handlePlay(gen.id, gen.text, gen.profile_id)}
>
<Play className="mr-2 h-4 w-4" />
Play
</DropdownMenuItem>
<DropdownMenuItem
onClick={() => handleDownloadAudio(gen.id, gen.text)}
disabled={exportGenerationAudio.isPending}
>
<Download className="mr-2 h-4 w-4" />
Export Audio
</DropdownMenuItem>
<DropdownMenuItem
onClick={() => handleExportPackage(gen.id, gen.text)}
disabled={exportGeneration.isPending}
>
<FileArchive className="mr-2 h-4 w-4" />
Export Package
</DropdownMenuItem>
<DropdownMenuItem onClick={() => handleApplyEffects(gen.id)}>
<Wand2 className="mr-2 h-4 w-4" />
Apply Effects
</DropdownMenuItem>
<DropdownMenuItem onClick={() => handleRegenerate(gen.id)}>
<RotateCcw className="mr-2 h-4 w-4" />
Regenerate
</DropdownMenuItem>
<DropdownMenuItem
onClick={() => handleDeleteClick(gen.id, gen.profile_name)}
disabled={deleteGeneration.isPending}
// className="text-destructive focus:text-destructive"
>
<Trash2 className="mr-2 h-4 w-4" />
Delete
</DropdownMenuItem>
</DropdownMenuContent>
</DropdownMenu>
</>
)}
<div className="text-xs text-muted-foreground">
{formatDate(gen.created_at)}
</div>
</div>
{/* Expandable versions panel */}
<AnimatePresence>
{isVersionsExpanded && gen.versions && (
<motion.div
initial={{ height: 0, opacity: 0 }}
animate={{ height: 'auto', opacity: 1 }}
exit={{ height: 0, opacity: 0 }}
transition={{ duration: 0.2, ease: 'easeOut' }}
className="overflow-hidden"
>
<div className="border-t border-border/50">
<div className="divide-y divide-border/40">
{gen.versions.map((v) => {
// Show source provenance when effects were applied to a non-clean version
const sourceVersion = v.source_version_id
? gen.versions?.find((sv) => sv.id === v.source_version_id)
: null;
const showSource =
sourceVersion &&
sourceVersion.effects_chain &&
sourceVersion.effects_chain.length > 0;
{/* Right side - Transcript textarea */}
<div className="flex-1 min-w-0 flex">
<Textarea
value={gen.text}
className="flex-1 resize-none text-sm text-muted-foreground select-text"
readOnly
/>
</div>
return (
<button
key={v.id}
type="button"
className="flex items-center gap-2 w-full h-9 px-3 text-left hover:bg-muted/50 transition-colors"
onClick={() => {
handlePlayVersion(gen.id, v.id, gen.text, gen.profile_id);
if (!v.is_default) {
handleSwitchVersion(gen.id, v.id);
}
}}
>
<AudioLines className="h-3 w-3 shrink-0 text-muted-foreground" />
<span className="truncate text-xs font-medium">{v.label}</span>
{v.effects_chain && v.effects_chain.length > 0 && (
<span className="text-[10px] text-muted-foreground truncate">
{v.effects_chain.map((e) => e.type).join(' → ')}
</span>
)}
{showSource && (
<span className="text-[10px] text-muted-foreground/60 truncate">
from {sourceVersion.label}
</span>
)}
<span className="flex-1" />
{v.is_default && (
<span className="text-[10px] bg-accent/15 text-accent px-1.5 py-0.5 rounded-full">
active
</span>
)}
</button>
);
})}
</div>
</div>
</motion.div>
)}
</AnimatePresence>
{/* Far right - Ellipsis actions */}
<div
className="w-10 shrink-0 flex justify-end"
onMouseDown={(e) => e.stopPropagation()}
onClick={(e) => e.stopPropagation()}
>
<DropdownMenu>
<DropdownMenuTrigger asChild>
<Button
variant="ghost"
size="icon"
className="h-8 w-8"
aria-label="Actions"
>
<HugeiconsIcon icon={MoreHorizontalIcon} size={16} className="h-4 w-4" />
</Button>
</DropdownMenuTrigger>
<DropdownMenuContent align="end">
<DropdownMenuItem
onClick={() => handlePlay(gen.id, gen.text, gen.profile_id)}
>
<HugeiconsIcon icon={PlayIcon} size={16} className="mr-2 h-4 w-4" />
Play
</DropdownMenuItem>
<DropdownMenuItem
onClick={() => handleDownloadAudio(gen.id, gen.text)}
disabled={exportGenerationAudio.isPending}
>
<HugeiconsIcon icon={Download01Icon} size={16} className="mr-2 h-4 w-4" />
Export Audio
</DropdownMenuItem>
<DropdownMenuItem
onClick={() => handleExportPackage(gen.id, gen.text)}
disabled={exportGeneration.isPending}
>
<HugeiconsIcon icon={Archive01Icon} size={16} className="mr-2 h-4 w-4" />
Export Package
</DropdownMenuItem>
<DropdownMenuItem
onClick={() => handleDeleteClick(gen.id, gen.profile_name)}
disabled={deleteGeneration.isPending}
>
<HugeiconsIcon icon={Delete01Icon} size={16} className="mr-2 h-4 w-4" />
Delete
</DropdownMenuItem>
</DropdownMenuContent>
</DropdownMenu>
</div>
</div>
);
})}
@@ -664,7 +361,12 @@ export function HistoryTable() {
{/* Load more trigger element */}
{hasMore && (
<div ref={loadMoreRef} className="flex items-center justify-center py-4">
{isFetching && <Loader2 className="h-6 w-6 animate-spin text-muted-foreground" />}
{isFetching && (
<Icon
icon="svg-spinners:ring-resize"
className="h-6 w-6 animate-spin text-muted-foreground"
/>
)}
</div>
)}
@@ -738,57 +440,6 @@ export function HistoryTable() {
</DialogFooter>
</DialogContent>
</Dialog>
<Dialog open={effectsDialogOpen} onOpenChange={setEffectsDialogOpen}>
<DialogContent className="max-w-md">
<DialogHeader>
<DialogTitle>Apply Effects</DialogTitle>
<DialogDescription>
Configure post-processing effects to apply to this generation. A new version will be
created.
</DialogDescription>
</DialogHeader>
{effectsTargetVersions.length > 1 && (
<div className="space-y-1.5">
<label className="text-xs font-medium text-muted-foreground">Source</label>
<Select
value={effectsSourceVersionId ?? ''}
onValueChange={(val) => setEffectsSourceVersionId(val || null)}
>
<SelectTrigger className="h-8 text-xs">
<SelectValue placeholder="Select source version" />
</SelectTrigger>
<SelectContent>
{effectsTargetVersions.map((v) => (
<SelectItem key={v.id} value={v.id} className="text-xs">
{v.label}
{v.effects_chain && v.effects_chain.length > 0 && (
<span className="text-muted-foreground ml-1.5">
({v.effects_chain.map((e) => e.type).join(' + ')})
</span>
)}
</SelectItem>
))}
</SelectContent>
</Select>
</div>
)}
<div className="py-2 max-h-80 overflow-y-auto">
<EffectsChainEditor value={effectsChain} onChange={setEffectsChain} />
</div>
<DialogFooter>
<Button variant="outline" onClick={() => setEffectsDialogOpen(false)}>
Cancel
</Button>
<Button
onClick={handleApplyEffectsConfirm}
disabled={applyingEffects || effectsChain.length === 0}
>
{applyingEffects ? 'Applying...' : 'Apply'}
</Button>
</DialogFooter>
</DialogContent>
</Dialog>
</div>
);
}
+11 -10
View File
@@ -1,4 +1,5 @@
import { Sparkles, Upload } from 'lucide-react';
import { HugeiconsIcon } from '@hugeicons/react';
import { SparklesIcon, Upload01Icon } from '@hugeicons/core-free-icons';
import { useRef, useState } from 'react';
import { FloatingGenerateBox } from '@/components/Generation/FloatingGenerateBox';
import { HistoryTable } from '@/components/History/HistoryTable';
@@ -13,7 +14,7 @@ import {
} from '@/components/ui/dialog';
import { useToast } from '@/components/ui/use-toast';
import { ProfileList } from '@/components/VoiceProfiles/ProfileList';
import { BOTTOM_SAFE_AREA_PADDING } from '@/lib/constants/ui';
import { useImportProfile } from '@/lib/hooks/useProfiles';
import { cn } from '@/lib/utils/cn';
import { usePlayerStore } from '@/stores/playerStore';
@@ -77,9 +78,9 @@ export function MainEditor() {
return (
// Main view: Profiles top left, Generator bottom left, History right
<div className="grid grid-cols-1 lg:grid-cols-2 lg:gap-6 h-full min-h-0 overflow-hidden relative">
<div className="grid grid-cols-1 lg:grid-cols-2 gap-6 h-full min-h-0 overflow-hidden relative">
{/* Left Column */}
<div className="flex flex-col min-h-0 overflow-hidden relative lg:overflow-hidden">
<div className="flex flex-col min-h-0 overflow-hidden relative">
{/* Scroll Mask - Always visible, behind content */}
<div className="absolute top-0 left-0 right-0 h-16 bg-gradient-to-b from-background to-transparent z-0 pointer-events-none" />
@@ -89,7 +90,7 @@ export function MainEditor() {
<h2 className="text-2xl font-bold">Voicebox</h2>
<div className="flex gap-2">
<Button variant="outline" onClick={handleImportClick}>
<Upload className="mr-2 h-4 w-4" />
<HugeiconsIcon icon={Upload01Icon} size={16} className="mr-2 h-4 w-4" />
Import Voice
</Button>
<input
@@ -100,7 +101,7 @@ export function MainEditor() {
className="hidden"
/>
<Button onClick={() => setDialogOpen(true)}>
<Sparkles className="mr-2 h-4 w-4" />
<HugeiconsIcon icon={SparklesIcon} size={16} className="mr-2 h-4 w-4" />
Create Voice
</Button>
</div>
@@ -110,7 +111,10 @@ export function MainEditor() {
{/* Scrollable Content */}
<div
ref={scrollRef}
className={cn('flex-1 min-h-0 overflow-y-auto pt-14 pb-4', isPlayerVisible && 'lg:pb-32')}
className={cn(
'flex-1 min-h-0 overflow-y-auto pt-14',
isPlayerVisible ? BOTTOM_SAFE_AREA_PADDING : 'pb-4',
)}
>
<div className="flex flex-col gap-6">
<div className="shrink-0 flex flex-col">
@@ -120,9 +124,6 @@ export function MainEditor() {
</div>
</div>
{/* Divider - single column only */}
{/* <div className="border-t border-border -my-3 lg:hidden" /> */}
{/* Right Column - History */}
<div className="flex flex-col min-h-0 overflow-hidden">
<HistoryTable />
+1 -1
View File
@@ -2,7 +2,7 @@ import { ModelManagement } from '@/components/ServerSettings/ModelManagement';
export function ModelsTab() {
return (
<div className="h-full flex flex-col">
<div className="space-y-4 overflow-y-auto flex flex-col">
<ModelManagement />
</div>
);
@@ -1,12 +1,9 @@
import { zodResolver } from '@hookform/resolvers/zod';
import { Loader2, XCircle } from 'lucide-react';
import { useEffect } from 'react';
import { useForm } from 'react-hook-form';
import * as z from 'zod';
import { Badge } from '@/components/ui/badge';
import { Button } from '@/components/ui/button';
import { Card, CardContent, CardHeader, CardTitle } from '@/components/ui/card';
import { Checkbox } from '@/components/ui/checkbox';
import {
Form,
FormControl,
@@ -17,10 +14,10 @@ import {
FormMessage,
} from '@/components/ui/form';
import { Input } from '@/components/ui/input';
import { Checkbox } from '@/components/ui/checkbox';
import { useToast } from '@/components/ui/use-toast';
import { useServerHealth } from '@/lib/hooks/useServer';
import { usePlatform } from '@/platform/PlatformContext';
import { useServerStore } from '@/stores/serverStore';
import { usePlatform } from '@/platform/PlatformContext';
const connectionSchema = z.object({
serverUrl: z.string().url('Please enter a valid URL'),
@@ -34,10 +31,7 @@ export function ConnectionForm() {
const setServerUrl = useServerStore((state) => state.setServerUrl);
const keepServerRunningOnClose = useServerStore((state) => state.keepServerRunningOnClose);
const setKeepServerRunningOnClose = useServerStore((state) => state.setKeepServerRunningOnClose);
const mode = useServerStore((state) => state.mode);
const setMode = useServerStore((state) => state.setMode);
const { toast } = useToast();
const { data: health, isLoading, error: healthError } = useServerHealth();
const form = useForm<ConnectionFormValues>({
resolver: zodResolver(connectionSchema),
@@ -55,7 +49,7 @@ export function ConnectionForm() {
function onSubmit(data: ConnectionFormValues) {
setServerUrl(data.serverUrl);
form.reset(data);
form.reset(data); // Reset form state after successful submission
toast({
title: 'Server URL updated',
description: `Connected to ${data.serverUrl}`,
@@ -63,7 +57,7 @@ export function ConnectionForm() {
}
return (
<Card role="region" aria-label="Server Connection" tabIndex={0}>
<Card>
<CardHeader>
<CardTitle>Server Connection</CardTitle>
</CardHeader>
@@ -89,42 +83,10 @@ export function ConnectionForm() {
</form>
</Form>
{/* Connection status */}
<div className="mt-4">
{isLoading ? (
<div className="flex items-center gap-2">
<Loader2 className="h-4 w-4 animate-spin" />
<span className="text-sm text-muted-foreground">Checking connection...</span>
</div>
) : healthError ? (
<div className="flex items-center gap-2">
<XCircle className="h-4 w-4 text-destructive" />
<span className="text-sm text-destructive">
Connection failed: {healthError.message}
</span>
</div>
) : health ? (
<div className="flex flex-wrap gap-2">
<Badge
variant={health.model_loaded || health.model_downloaded ? 'default' : 'secondary'}
>
{health.model_loaded || health.model_downloaded ? 'Model Ready' : 'No Model'}
</Badge>
<Badge variant={health.gpu_available ? 'default' : 'secondary'}>
GPU: {health.gpu_available ? 'Available' : 'Not Available'}
</Badge>
{health.vram_used_mb != null && health.vram_used_mb > 0 && (
<Badge variant="outline">VRAM: {health.vram_used_mb.toFixed(0)} MB</Badge>
)}
</div>
) : null}
</div>
<div className="mt-6 pt-6 border-t">
<div className="flex items-start space-x-3">
<Checkbox
id="keepServerRunning"
className="mt-[6px]"
checked={keepServerRunningOnClose}
onCheckedChange={(checked: boolean) => {
setKeepServerRunningOnClose(checked);
@@ -153,39 +115,6 @@ export function ConnectionForm() {
</div>
</div>
</div>
{platform.metadata.isTauri && (
<div className="mt-6 pt-6 border-t">
<div className="flex items-start space-x-3">
<Checkbox
id="allowNetworkAccess"
className="mt-[6px]"
checked={mode === 'remote'}
onCheckedChange={(checked: boolean) => {
setMode(checked ? 'remote' : 'local');
toast({
title: 'Setting updated',
description: checked
? 'Network access enabled. Restart the app to apply.'
: 'Network access disabled. Restart the app to apply.',
});
}}
/>
<div className="space-y-1">
<label
htmlFor="allowNetworkAccess"
className="text-sm font-medium leading-none peer-disabled:cursor-not-allowed peer-disabled:opacity-70 cursor-pointer"
>
Allow network access
</label>
<p className="text-sm text-muted-foreground">
Makes the server accessible from other devices on your network. Restart the app
after changing this setting.
</p>
</div>
</div>
</div>
)}
</CardContent>
</Card>
);
@@ -0,0 +1,112 @@
import { Folder01Icon, FolderOpenIcon } from '@hugeicons/core-free-icons';
import { HugeiconsIcon } from '@hugeicons/react';
import { Icon } from '@iconify/react';
import { Button } from '@/components/ui/button';
import { Card, CardContent, CardDescription, CardHeader, CardTitle } from '@/components/ui/card';
import { Input } from '@/components/ui/input';
import { useSystemFolders } from '@/lib/hooks/useSystemFolders';
import { usePlatform } from '@/platform/PlatformContext';
interface FolderRowProps {
label: string;
description: string;
path: string | undefined;
isLoading: boolean;
canOpen: boolean;
onOpen: () => void;
}
function FolderRow({ label, description, path, isLoading, canOpen, onOpen }: FolderRowProps) {
return (
<div className="space-y-1.5">
<div className="flex items-center justify-between">
<div>
<div className="text-sm font-medium">{label}</div>
<div className="text-xs text-muted-foreground">{description}</div>
</div>
{canOpen && path && (
<Button
variant="outline"
size="sm"
onClick={onOpen}
disabled={isLoading || !path}
className="shrink-0"
>
<HugeiconsIcon icon={FolderOpenIcon} size={16} className="h-4 w-4 mr-2" />
Open
</Button>
)}
</div>
<Input
value={isLoading ? 'Loading...' : path || 'Not available'}
readOnly
className="font-mono text-xs text-muted-foreground select-all cursor-text"
/>
</div>
);
}
export function DataFolders() {
const { data: folders, isLoading, error } = useSystemFolders();
const platform = usePlatform();
const isTauri = platform.metadata.isTauri;
const handleOpenFolder = async (path: string | undefined) => {
if (!path) return;
const success = await platform.filesystem.openFolder(path);
if (!success && isTauri) {
console.error('Failed to open folder:', path);
}
};
return (
<Card>
<CardHeader>
<CardTitle className="flex items-center gap-2">
<HugeiconsIcon icon={Folder01Icon} size={20} className="h-5 w-5" />
Data Folders
</CardTitle>
<CardDescription>
{isTauri
? 'Click "Open" to view folders in your file explorer, or copy the paths below.'
: 'These are the server-side folder paths where your data is stored.'}
</CardDescription>
</CardHeader>
<CardContent className="space-y-4">
{error ? (
<div className="flex items-center gap-2 text-sm text-destructive">
<Icon icon="lucide:alert-circle" className="h-4 w-4" />
<span>Failed to load folder paths: {error.message}</span>
</div>
) : (
<>
<FolderRow
label="App Data"
description="Voices, generations, and app database"
path={folders?.data_dir}
isLoading={isLoading}
canOpen={isTauri}
onOpen={() => handleOpenFolder(folders?.data_dir)}
/>
<FolderRow
label="Models"
description="Downloaded AI models from HuggingFace Hub"
path={folders?.models_dir}
isLoading={isLoading}
canOpen={isTauri}
onOpen={() => handleOpenFolder(folders?.models_dir)}
/>
<FolderRow
label="Providers"
description="External TTS provider binaries (PyTorch CPU/CUDA)"
path={folders?.providers_dir}
isLoading={isLoading}
canOpen={isTauri}
onOpen={() => handleOpenFolder(folders?.providers_dir)}
/>
</>
)}
</CardContent>
</Card>
);
}
@@ -1,116 +0,0 @@
import { Card, CardContent, CardDescription, CardHeader, CardTitle } from '@/components/ui/card';
import { Checkbox } from '@/components/ui/checkbox';
import { Slider } from '@/components/ui/slider';
import { useServerStore } from '@/stores/serverStore';
export function GenerationSettings() {
const maxChunkChars = useServerStore((state) => state.maxChunkChars);
const setMaxChunkChars = useServerStore((state) => state.setMaxChunkChars);
const crossfadeMs = useServerStore((state) => state.crossfadeMs);
const setCrossfadeMs = useServerStore((state) => state.setCrossfadeMs);
const normalizeAudio = useServerStore((state) => state.normalizeAudio);
const setNormalizeAudio = useServerStore((state) => state.setNormalizeAudio);
const autoplayOnGenerate = useServerStore((state) => state.autoplayOnGenerate);
const setAutoplayOnGenerate = useServerStore((state) => state.setAutoplayOnGenerate);
return (
<Card role="region" aria-label="Generation Settings" tabIndex={0}>
<CardHeader>
<CardTitle>Generation Settings</CardTitle>
<CardDescription>
Controls for long text generation. These settings apply to all engines.
</CardDescription>
</CardHeader>
<CardContent>
<div className="space-y-6">
<div className="space-y-3">
<div className="flex items-center justify-between">
<label htmlFor="maxChunkChars" className="text-sm font-medium leading-none">
Auto-chunking limit
</label>
<span className="text-sm tabular-nums text-muted-foreground">
{maxChunkChars} chars
</span>
</div>
<Slider
id="maxChunkChars"
value={[maxChunkChars]}
onValueChange={([value]) => setMaxChunkChars(value)}
min={100}
max={5000}
step={50}
aria-label="Auto-chunking character limit"
/>
<p className="text-sm text-muted-foreground">
Long text is split into chunks at sentence boundaries before generating. Lower values
can improve quality for long outputs.
</p>
</div>
<div className="space-y-3">
<div className="flex items-center justify-between">
<label htmlFor="crossfadeMs" className="text-sm font-medium leading-none">
Chunk crossfade
</label>
<span className="text-sm tabular-nums text-muted-foreground">
{crossfadeMs === 0 ? 'Cut' : `${crossfadeMs}ms`}
</span>
</div>
<Slider
id="crossfadeMs"
value={[crossfadeMs]}
onValueChange={([value]) => setCrossfadeMs(value)}
min={0}
max={200}
step={10}
aria-label="Chunk crossfade duration"
/>
<p className="text-sm text-muted-foreground">
Blends audio between chunks to smooth transitions. Set to 0 for a hard cut.
</p>
</div>
<div className="flex items-start gap-3">
<Checkbox
id="normalizeAudio"
checked={normalizeAudio}
onCheckedChange={setNormalizeAudio}
className="mt-[6px]"
/>
<div className="space-y-1">
<label
htmlFor="normalizeAudio"
className="text-sm font-medium leading-none cursor-pointer"
>
Normalize audio
</label>
<p className="text-sm text-muted-foreground">
Adjusts output volume to a consistent level across generations.
</p>
</div>
</div>
<div className="flex items-start gap-3">
<Checkbox
id="autoplayOnGenerate"
checked={autoplayOnGenerate}
onCheckedChange={setAutoplayOnGenerate}
className="mt-[6px]"
/>
<div className="space-y-1">
<label
htmlFor="autoplayOnGenerate"
className="text-sm font-medium leading-none cursor-pointer"
>
Autoplay on generate
</label>
<p className="text-sm text-muted-foreground">
Automatically play audio when a generation completes.
</p>
</div>
</div>
</div>
</CardContent>
</Card>
);
}
@@ -1,369 +0,0 @@
import { useQuery, useQueryClient } from '@tanstack/react-query';
import { AlertCircle, Download, Loader2, RotateCw, Trash2 } from 'lucide-react';
import { useCallback, useEffect, useRef, useState } from 'react';
import { Button } from '@/components/ui/button';
import { Card, CardContent, CardHeader, CardTitle } from '@/components/ui/card';
import { Progress } from '@/components/ui/progress';
import { apiClient } from '@/lib/api/client';
import type { CudaDownloadProgress } from '@/lib/api/types';
import { useServerHealth } from '@/lib/hooks/useServer';
import { usePlatform } from '@/platform/PlatformContext';
import { useServerStore } from '@/stores/serverStore';
type RestartPhase = 'idle' | 'stopping' | 'waiting' | 'ready';
export function GpuAcceleration() {
const platform = usePlatform();
const queryClient = useQueryClient();
const serverUrl = useServerStore((state) => state.serverUrl);
const { data: health } = useServerHealth();
const [restartPhase, setRestartPhase] = useState<RestartPhase>('idle');
const [error, setError] = useState<string | null>(null);
const [downloadProgress, setDownloadProgress] = useState<CudaDownloadProgress | null>(null);
const healthPollRef = useRef<ReturnType<typeof setInterval> | null>(null);
// Query CUDA backend status
const {
data: cudaStatus,
isLoading: _cudaStatusLoading,
refetch: refetchCudaStatus,
} = useQuery({
queryKey: ['cuda-status', serverUrl],
queryFn: () => apiClient.getCudaStatus(),
refetchInterval: (query) => (query.state.status === 'pending' ? false : 10000),
retry: 1,
enabled: !!health, // Only fetch when backend is reachable
});
// Derived state
const isCurrentlyCuda = health?.backend_variant === 'cuda';
const cudaAvailable = cudaStatus?.available ?? false;
const cudaDownloading = cudaStatus?.downloading ?? false;
// Clean up health poll on unmount
useEffect(() => {
return () => {
if (healthPollRef.current) {
clearInterval(healthPollRef.current);
healthPollRef.current = null;
}
};
}, []);
// SSE progress tracking during download
useEffect(() => {
if (!cudaDownloading || !serverUrl) {
return;
}
const eventSource = new EventSource(`${serverUrl}/backend/cuda-progress`);
eventSource.onmessage = (event) => {
try {
const data = JSON.parse(event.data) as CudaDownloadProgress;
setDownloadProgress(data);
if (data.status === 'complete') {
eventSource.close();
setDownloadProgress(null);
refetchCudaStatus();
} else if (data.status === 'error') {
eventSource.close();
setError(data.error || 'Download failed');
setDownloadProgress(null);
refetchCudaStatus();
}
} catch (e) {
console.error('Error parsing CUDA progress event:', e);
}
};
eventSource.onerror = () => {
eventSource.close();
};
return () => {
eventSource.close();
};
}, [cudaDownloading, serverUrl, refetchCudaStatus]);
// Start aggressive health polling during restart
const startHealthPolling = useCallback(() => {
if (healthPollRef.current) return;
healthPollRef.current = setInterval(async () => {
try {
const result = await apiClient.getHealth();
if (result.status === 'healthy') {
// Server is back up
if (healthPollRef.current) {
clearInterval(healthPollRef.current);
healthPollRef.current = null;
}
setRestartPhase('ready');
// Invalidate all queries to refresh UI
queryClient.invalidateQueries();
// Reset after a moment
setTimeout(() => setRestartPhase('idle'), 2000);
}
} catch {
// Server still down, keep polling
}
}, 1000);
}, [queryClient]);
const handleDownload = async () => {
setError(null);
try {
await apiClient.downloadCudaBackend();
refetchCudaStatus();
} catch (e: unknown) {
const msg = e instanceof Error ? e.message : 'Failed to start download';
if (msg.includes('already downloaded')) {
refetchCudaStatus();
} else {
setError(msg);
}
}
};
const handleRestart = async () => {
setError(null);
setRestartPhase('stopping');
try {
setRestartPhase('waiting');
startHealthPolling();
await platform.lifecycle.restartServer();
// Invoke resolved — server is likely ready. Stop polling and refresh.
if (healthPollRef.current) {
clearInterval(healthPollRef.current);
healthPollRef.current = null;
}
setRestartPhase('ready');
queryClient.invalidateQueries();
setTimeout(() => setRestartPhase('idle'), 2000);
} catch (e: unknown) {
setRestartPhase('idle');
if (healthPollRef.current) {
clearInterval(healthPollRef.current);
healthPollRef.current = null;
}
setError(e instanceof Error ? e.message : 'Restart failed');
}
};
const handleSwitchToCpu = async () => {
// To switch to CPU: delete the CUDA binary, then restart.
// start_server always prefers CUDA if present, so we must remove it first.
setError(null);
setRestartPhase('stopping');
try {
await apiClient.deleteCudaBackend();
setRestartPhase('waiting');
startHealthPolling();
await platform.lifecycle.restartServer();
// Invoke resolved — server is likely ready
if (healthPollRef.current) {
clearInterval(healthPollRef.current);
healthPollRef.current = null;
}
setRestartPhase('ready');
queryClient.invalidateQueries();
setTimeout(() => setRestartPhase('idle'), 2000);
} catch (e: unknown) {
setRestartPhase('idle');
if (healthPollRef.current) {
clearInterval(healthPollRef.current);
healthPollRef.current = null;
}
setError(e instanceof Error ? e.message : 'Failed to switch to CPU');
refetchCudaStatus();
}
};
const handleDelete = async () => {
setError(null);
try {
await apiClient.deleteCudaBackend();
refetchCudaStatus();
} catch (e: unknown) {
setError(e instanceof Error ? e.message : 'Failed to delete CUDA backend');
}
};
const formatBytes = (bytes: number): string => {
if (bytes === 0) return '0 B';
const k = 1024;
const sizes = ['B', 'KB', 'MB', 'GB'];
const i = Math.floor(Math.log(bytes) / Math.log(k));
return `${(bytes / k ** i).toFixed(1)} ${sizes[i]}`;
};
// Don't render until health data is available
if (!health) return null;
// If the system already has native GPU (MPS, etc.), only show info - no CUDA needed
const hasNativeGpu =
health.gpu_available &&
!isCurrentlyCuda &&
health.gpu_type &&
!health.gpu_type.includes('CUDA');
return (
<Card>
<CardHeader>
<CardTitle>GPU Acceleration</CardTitle>
</CardHeader>
<CardContent className="space-y-4">
{/* GPU status */}
<div className="space-y-1">
{health.gpu_available && health.gpu_type ? (
<>
<div className="text-sm font-medium">
{health.gpu_type.replace(/^(CUDA|ROCm|MPS|Metal|XPU|DirectML)\s*\((.+)\)$/, '$2') ||
health.gpu_type}
</div>
<div className="text-sm text-muted-foreground">
{health.gpu_type.replace(/\s*\(.+\)$/, '')}
{health.vram_used_mb != null && health.vram_used_mb > 0
? ` \u00b7 ${health.vram_used_mb.toFixed(0)} MB VRAM used`
: ''}
</div>
</>
) : (
<>
<div className="text-sm font-medium">CPU</div>
<div className="text-sm text-muted-foreground">No GPU acceleration available</div>
</>
)}
</div>
{/* Native GPU detected - no CUDA download needed */}
{/* CUDA download section - only show when no GPU is active (native or CUDA) */}
{!hasNativeGpu && !isCurrentlyCuda && (
<>
{/* Download progress (manual download or auto-update) */}
{cudaDownloading && downloadProgress && (
<div className="space-y-2">
<div className="flex items-center justify-between text-sm">
<div className="flex items-center gap-2">
<Loader2 className="h-4 w-4 animate-spin" />
<span>
{downloadProgress.filename ||
(cudaAvailable
? 'Updating CUDA backend...'
: 'Downloading CUDA backend...')}
</span>
</div>
{downloadProgress.total > 0 && (
<span className="text-muted-foreground">
{downloadProgress.progress.toFixed(1)}%
</span>
)}
</div>
{downloadProgress.total > 0 && (
<>
<Progress value={downloadProgress.progress} className="h-2" />
<div className="text-xs text-muted-foreground">
{formatBytes(downloadProgress.current)} /{' '}
{formatBytes(downloadProgress.total)}
</div>
</>
)}
</div>
)}
{/* Restart in progress */}
{restartPhase !== 'idle' && (
<div className="flex items-center gap-2 p-3 rounded-lg bg-primary/5 border">
<Loader2 className="h-4 w-4 animate-spin" />
<span className="text-sm">
{restartPhase === 'stopping' && 'Stopping server...'}
{restartPhase === 'waiting' && 'Restarting server...'}
{restartPhase === 'ready' && 'Server restarted successfully!'}
</span>
</div>
)}
{/* Error display */}
{error && (
<div className="flex items-center gap-2 text-sm text-destructive">
<AlertCircle className="h-4 w-4 shrink-0" />
<span>{error}</span>
</div>
)}
{/* Actions */}
{restartPhase === 'idle' && !cudaDownloading && (
<div className="space-y-2">
{/* Not downloaded yet - show download button */}
{!cudaAvailable && (
<div className="space-y-3">
<p className="text-sm text-muted-foreground">
Download the CUDA backend (~2.4 GB) for NVIDIA GPU acceleration. Requires an
NVIDIA GPU with CUDA support.
</p>
<Button onClick={handleDownload} className="w-full" size="sm">
<Download className="h-4 w-4 mr-2" />
Download CUDA Backend
</Button>
</div>
)}
{/* Downloaded but not active - show switch button */}
{cudaAvailable && !isCurrentlyCuda && platform.metadata.isTauri && (
<div className="space-y-3">
<p className="text-sm text-muted-foreground">
CUDA backend is downloaded and ready. Restart the server to enable GPU
acceleration.
</p>
<Button onClick={handleRestart} className="w-full" size="sm">
<RotateCw className="h-4 w-4 mr-2" />
Switch to CUDA Backend
</Button>
</div>
)}
{/* Currently active - show switch back to CPU */}
{isCurrentlyCuda && platform.metadata.isTauri && (
<div className="space-y-3">
<p className="text-sm text-muted-foreground">
Running with CUDA GPU acceleration. Switch back to CPU if needed (you can
re-download later).
</p>
<Button
onClick={handleSwitchToCpu}
variant="outline"
className="w-full"
size="sm"
>
<RotateCw className="h-4 w-4 mr-2" />
Switch to CPU Backend
</Button>
</div>
)}
{/* Delete option when downloaded (and not active) */}
{cudaAvailable && !isCurrentlyCuda && (
<Button
onClick={handleDelete}
variant="ghost"
className="w-full text-muted-foreground hover:text-destructive"
size="sm"
>
<Trash2 className="h-4 w-4 mr-2" />
Remove CUDA Backend
</Button>
)}
</div>
)}
</>
)}
</CardContent>
</Card>
);
}
File diff suppressed because it is too large Load Diff
@@ -1,4 +1,6 @@
import { Loader2, XCircle } from 'lucide-react';
import { CancelCircleIcon } from '@hugeicons/core-free-icons';
import { HugeiconsIcon } from '@hugeicons/react';
import { Icon } from '@iconify/react';
import { useEffect, useState } from 'react';
import { Card, CardContent, CardHeader, CardTitle } from '@/components/ui/card';
import { Progress } from '@/components/ui/progress';
@@ -12,7 +14,11 @@ interface ModelProgressProps {
isDownloading?: boolean;
}
export function ModelProgress({ modelName, displayName, isDownloading = false }: ModelProgressProps) {
export function ModelProgress({
modelName,
displayName,
isDownloading = false,
}: ModelProgressProps) {
const [progress, setProgress] = useState<ModelProgressType | null>(null);
const serverUrl = useServerStore((state) => state.serverUrl);
@@ -74,10 +80,12 @@ export function ModelProgress({ modelName, displayName, isDownloading = false }:
const getStatusIcon = () => {
switch (progress.status) {
case 'error':
return <XCircle className="h-4 w-4 text-destructive" />;
return (
<HugeiconsIcon icon={CancelCircleIcon} size={16} className="h-4 w-4 text-destructive" />
);
case 'downloading':
case 'extracting':
return <Loader2 className="h-4 w-4 animate-spin" />;
return <Icon icon="svg-spinners:ring-resize" className="h-4 w-4 animate-spin" />;
default:
return null;
}
@@ -0,0 +1,400 @@
import { Delete01Icon, Download01Icon } from '@hugeicons/core-free-icons';
import { HugeiconsIcon } from '@hugeicons/react';
import { Icon } from '@iconify/react';
import { useMutation, useQuery, useQueryClient } from '@tanstack/react-query';
import { useCallback, useState } from 'react';
import {
AlertDialog,
AlertDialogAction,
AlertDialogCancel,
AlertDialogContent,
AlertDialogDescription,
AlertDialogFooter,
AlertDialogHeader,
AlertDialogTitle,
} from '@/components/ui/alert-dialog';
import { Button } from '@/components/ui/button';
import { Card, CardContent, CardDescription, CardHeader, CardTitle } from '@/components/ui/card';
import { Label } from '@/components/ui/label';
import { RadioGroup, RadioGroupItem } from '@/components/ui/radio-group';
import { useToast } from '@/components/ui/use-toast';
import { apiClient } from '@/lib/api/client';
import { useModelDownloadToast } from '@/lib/hooks/useModelDownloadToast';
const isMacOS = () => navigator.platform.toLowerCase().includes('mac');
const isWindows = () => navigator.platform.toLowerCase().includes('win');
const getPlatformName = () => {
if (isMacOS()) return 'macOS';
if (isWindows()) return 'Windows';
return 'Linux';
};
type ProviderType =
| 'auto'
| 'apple-mlx'
| 'bundled-pytorch'
| 'pytorch-cpu'
| 'pytorch-cuda'
| 'remote'
| 'openai';
export function ProviderSettings() {
const { toast } = useToast();
const queryClient = useQueryClient();
const [downloadingProvider, setDownloadingProvider] = useState<string | null>(null);
const { data: providersData, isLoading } = useQuery({
queryKey: ['providers'],
queryFn: async () => {
return await apiClient.listProviders();
},
refetchInterval: 5000,
});
const { data: activeProvider } = useQuery({
queryKey: ['activeProvider'],
queryFn: async () => {
return await apiClient.getActiveProvider();
},
refetchInterval: 5000,
});
// Callbacks for download completion
const handleDownloadComplete = useCallback(() => {
setDownloadingProvider(null);
queryClient.invalidateQueries({ queryKey: ['providers'] });
}, [queryClient]);
const handleDownloadError = useCallback(() => {
setDownloadingProvider(null);
}, []);
// Use progress toast hook for the downloading provider
useModelDownloadToast({
modelName: downloadingProvider || '',
displayName: downloadingProvider || '',
enabled: !!downloadingProvider,
onComplete: handleDownloadComplete,
onError: handleDownloadError,
});
const [deleteDialogOpen, setDeleteDialogOpen] = useState(false);
const [providerToDelete, setProviderToDelete] = useState<string | null>(null);
const downloadMutation = useMutation({
mutationFn: async (providerType: string) => {
return await apiClient.downloadProvider(providerType);
},
onSuccess: (_, providerType) => {
setDownloadingProvider(providerType);
queryClient.invalidateQueries({ queryKey: ['providers'] });
},
onError: (error: Error) => {
toast({
title: 'Download failed',
description: error.message,
variant: 'destructive',
});
},
});
const startMutation = useMutation({
mutationFn: async (providerType: string) => {
return await apiClient.startProvider(providerType);
},
onSuccess: () => {
queryClient.invalidateQueries({ queryKey: ['activeProvider'] });
toast({
title: 'Provider started',
description: 'The provider has been started successfully',
});
},
onError: (error: Error) => {
toast({
title: 'Failed to start provider',
description: error.message,
variant: 'destructive',
});
},
});
const deleteMutation = useMutation({
mutationFn: async (providerType: string) => {
return await apiClient.deleteProvider(providerType);
},
onSuccess: () => {
queryClient.invalidateQueries({ queryKey: ['providers'] });
toast({
title: 'Provider deleted',
description: 'The provider has been deleted successfully',
});
},
onError: (error: Error) => {
toast({
title: 'Failed to delete provider',
description: error.message,
variant: 'destructive',
});
},
});
const handleDownload = async (providerType: string) => {
downloadMutation.mutate(providerType);
};
const handleStart = async (providerType: string) => {
startMutation.mutate(providerType);
};
const handleDelete = (providerType: string) => {
setProviderToDelete(providerType);
setDeleteDialogOpen(true);
};
const confirmDelete = () => {
if (providerToDelete) {
deleteMutation.mutate(providerToDelete);
setDeleteDialogOpen(false);
setProviderToDelete(null);
}
};
if (isLoading) {
return (
<Card>
<CardHeader>
<CardTitle>TTS Provider</CardTitle>
<CardDescription>Choose how Voicebox generates speech</CardDescription>
</CardHeader>
<CardContent>
<div className="flex items-center justify-center py-8">
<Icon icon="svg-spinners:ring-resize" className="h-6 w-6 animate-spin" />
</div>
</CardContent>
</Card>
);
}
const installedProviders = providersData?.installed || [];
// Determine current active provider
const currentProvider = activeProvider?.provider;
console.log('currentProvider', currentProvider);
const selectedProvider = currentProvider as ProviderType;
const isStarting = startMutation.isPending;
return (
<>
<Card>
<CardHeader>
<CardTitle>TTS Provider</CardTitle>
<CardDescription>Choose how Voicebox generates speech.</CardDescription>
</CardHeader>
<CardContent className="relative">
{isStarting && (
<div className="absolute inset-0 bg-background/80 backdrop-blur-sm flex items-center justify-center z-10 rounded-lg">
<div className="flex items-center gap-2 text-muted-foreground">
<Icon icon="svg-spinners:ring-resize" className="h-5 w-5" />
<span>Starting provider...</span>
</div>
</div>
)}
<RadioGroup
value={selectedProvider}
onValueChange={(value) => handleStart(value)}
disabled={isStarting}
>
{/* PyTorch CUDA */}
<div className="flex items-center justify-between py-2">
<div className={`flex items-center space-x-3 flex-1 ${isMacOS() || !installedProviders.includes('pytorch-cuda') ? 'opacity-50' : ''}`}>
<RadioGroupItem value="pytorch-cuda" id="cuda" disabled={isMacOS() || isStarting || !installedProviders.includes('pytorch-cuda')} />
<Label
htmlFor="cuda"
className={`flex-1 ${isMacOS() || isStarting || !installedProviders.includes('pytorch-cuda') ? 'cursor-not-allowed' : 'cursor-pointer'}`}
>
<div className="font-medium">PyTorch CUDA</div>
<div className="text-sm text-muted-foreground">
NVIDIA GPU-accelerated provider
</div>
</Label>
</div>
<div className="flex items-center gap-2">
{isMacOS() && (
<>
<span className="text-xs text-muted-foreground">2.4GB</span>
<Button size="sm" variant="secondary" disabled>
Not Available on macOS
</Button>
</>
)}
{!isMacOS() && !installedProviders.includes('pytorch-cuda') && (
<>
<span className="text-xs text-muted-foreground">2.4GB</span>
<Button
onClick={() => handleDownload('pytorch-cuda')}
size="sm"
variant="outline"
disabled={downloadingProvider === 'pytorch-cuda' || isStarting}
className="shrink-0"
>
{downloadingProvider === 'pytorch-cuda' ? (
<Icon icon="svg-spinners:ring-resize" className="h-4 w-4 animate-spin" />
) : (
<>
<HugeiconsIcon icon={Download01Icon} size={16} className="h-4 w-4 mr-2" />
Download
</>
)}
</Button>
</>
)}
{installedProviders.includes('pytorch-cuda') && (
<Button
onClick={() => handleDelete('pytorch-cuda')}
size="sm"
variant="outline"
disabled={isStarting}
className="shrink-0"
>
<HugeiconsIcon icon={Delete01Icon} size={16} className="h-4 w-4 mr-2" />
Uninstall
</Button>
)}
</div>
</div>
{/* PyTorch CPU */}
<div className="flex items-center justify-between py-2">
<div className={`flex items-center space-x-3 flex-1 ${!installedProviders.includes('pytorch-cpu') ? 'opacity-50' : ''}`}>
<RadioGroupItem value="pytorch-cpu" id="cpu" disabled={isStarting || !installedProviders.includes('pytorch-cpu')} />
<Label
htmlFor="cpu"
className={`flex-1 ${isStarting || !installedProviders.includes('pytorch-cpu') ? 'cursor-not-allowed' : 'cursor-pointer'}`}
>
<div className="font-medium">PyTorch CPU</div>
<div className="text-sm text-muted-foreground">
Works on any system, slower inference
</div>
</Label>
</div>
<div className="flex items-center gap-2">
{!installedProviders.includes('pytorch-cpu') && (
<>
<span className="text-xs text-muted-foreground">242MB</span>
<Button
onClick={() => handleDownload('pytorch-cpu')}
size="sm"
variant="outline"
disabled={downloadingProvider === 'pytorch-cpu' || isStarting}
className="shrink-0"
>
{downloadingProvider === 'pytorch-cpu' ? (
<Icon icon="svg-spinners:ring-resize" className="h-4 w-4 animate-spin" />
) : (
<>
<HugeiconsIcon icon={Download01Icon} size={16} className="h-4 w-4 mr-2" />
Download
</>
)}
</Button>
</>
)}
{installedProviders.includes('pytorch-cpu') && (
<Button
onClick={() => handleDelete('pytorch-cpu')}
size="sm"
variant="outline"
disabled={isStarting}
className="shrink-0"
>
<HugeiconsIcon icon={Delete01Icon} size={16} className="h-4 w-4 mr-2" />
Uninstall
</Button>
)}
</div>
</div>
{/* MLX bundled (macOS Apple Silicon only) */}
<div className="flex items-center justify-between py-2">
<div className={`flex items-center space-x-3 flex-1 ${!isMacOS() ? 'opacity-50' : ''}`}>
<RadioGroupItem value="apple-mlx" id="mlx" disabled={isStarting || !isMacOS()} />
<Label
htmlFor="mlx"
className={`flex-1 ${isStarting || !isMacOS() ? 'cursor-not-allowed' : 'cursor-pointer'}`}
>
<div className="font-medium">Apple MLX</div>
<div className="text-sm text-muted-foreground">
{isMacOS()
? 'Bundled with this version, optimized for Apple Silicon'
: 'Only available on Apple Silicon'}
</div>
</Label>
</div>
{!isMacOS() && (
<Button size="sm" variant="secondary" disabled>
Not Available on {getPlatformName()}
</Button>
)}
</div>
{/* Remote */}
<div className="flex items-center justify-between py-2">
<div className="flex items-center space-x-3 flex-1 opacity-50">
<RadioGroupItem value="remote" id="remote" disabled />
<Label htmlFor="remote" className="flex-1 cursor-not-allowed">
<div className="font-medium">Remote Server</div>
<div className="text-sm text-muted-foreground">
Connect to your own TTS server
</div>
</Label>
</div>
<Button size="sm" variant="secondary" disabled>
Coming Soon
</Button>
</div>
{/* OpenAI */}
<div className="flex items-center justify-between py-2">
<div className="flex items-center space-x-3 flex-1 opacity-50">
<RadioGroupItem value="openai" id="openai" disabled />
<Label htmlFor="openai" className="flex-1 cursor-not-allowed">
<div className="font-medium">OpenAI API</div>
<div className="text-sm text-muted-foreground">Use OpenAI's TTS API</div>
</Label>
</div>
<Button size="sm" variant="secondary" disabled>
Coming Soon
</Button>
</div>
</RadioGroup>
<p className="text-xs text-muted-foreground mt-5">
Note: PyTorch and MLX use different versions of the same model. When switching between
them, you will need to redownload the model.
</p>
</CardContent>
</Card>
<AlertDialog open={deleteDialogOpen} onOpenChange={setDeleteDialogOpen}>
<AlertDialogContent>
<AlertDialogHeader>
<AlertDialogTitle>Delete Provider</AlertDialogTitle>
<AlertDialogDescription>
Are you sure you want to delete {providerToDelete}? This will remove the provider
binary from your system. You can download it again later if needed.
</AlertDialogDescription>
</AlertDialogHeader>
<AlertDialogFooter>
<AlertDialogCancel>Cancel</AlertDialogCancel>
<AlertDialogAction
onClick={confirmDelete}
className="bg-destructive text-destructive-foreground"
>
Delete
</AlertDialogAction>
</AlertDialogFooter>
</AlertDialogContent>
</AlertDialog>
</>
);
}
@@ -1,15 +1,18 @@
import { Loader2, XCircle } from 'lucide-react';
import { CancelCircleIcon } from '@hugeicons/core-free-icons';
import { HugeiconsIcon } from '@hugeicons/react';
import { Icon } from '@iconify/react';
import { Badge } from '@/components/ui/badge';
import { Card, CardContent, CardHeader, CardTitle } from '@/components/ui/card';
import { useServerHealth } from '@/lib/hooks/useServer';
import { useServerStore } from '@/stores/serverStore';
import { ModelProgress } from './ModelProgress';
export function ServerStatus() {
const { data: health, isLoading, error } = useServerHealth();
const serverUrl = useServerStore((state) => state.serverUrl);
return (
<Card role="region" aria-label="Server Status" tabIndex={0}>
<Card>
<CardHeader>
<CardTitle>Server Status</CardTitle>
</CardHeader>
@@ -19,14 +22,24 @@ export function ServerStatus() {
<div className="font-mono text-sm">{serverUrl}</div>
</div>
{/* Model download progress */}
<div className="space-y-2">
<ModelProgress modelName="qwen-tts-1.7B" displayName="Qwen TTS 1.7B" />
<ModelProgress modelName="qwen-tts-0.6B" displayName="Qwen TTS 0.6B" />
<ModelProgress modelName="whisper-base" displayName="Whisper Base" />
<ModelProgress modelName="whisper-small" displayName="Whisper Small" />
<ModelProgress modelName="whisper-medium" displayName="Whisper Medium" />
<ModelProgress modelName="whisper-large" displayName="Whisper Large" />
</div>
{isLoading ? (
<div className="flex items-center gap-2">
<Loader2 className="h-4 w-4 animate-spin" />
<Icon icon="svg-spinners:ring-resize" className="h-4 w-4 animate-spin" />
<span className="text-sm">Checking connection...</span>
</div>
) : error ? (
<div className="flex items-center gap-2">
<XCircle className="h-4 w-4 text-destructive" />
<HugeiconsIcon icon={CancelCircleIcon} size={16} className="h-4 w-4 text-destructive" />
<span className="text-sm text-destructive">Connection failed: {error.message}</span>
</div>
) : health ? (
@@ -1,4 +1,5 @@
import { AlertCircle, Download, RefreshCw } from 'lucide-react';
import { HugeiconsIcon } from '@hugeicons/react';
import { AlertCircleIcon, Download01Icon, Refresh01Icon } from '@hugeicons/core-free-icons';
import { useEffect, useState } from 'react';
import { Badge } from '@/components/ui/badge';
import { Button } from '@/components/ui/button';
@@ -11,7 +12,6 @@ export function UpdateStatus() {
const platform = usePlatform();
const { status, checkForUpdates, downloadAndInstall, restartAndInstall } = useAutoUpdater(false);
const [currentVersion, setCurrentVersion] = useState<string>('');
const isDev = !import.meta.env?.PROD;
useEffect(() => {
platform.metadata
@@ -21,7 +21,7 @@ export function UpdateStatus() {
}, [platform]);
return (
<Card role="region" aria-label="App Updates" tabIndex={0}>
<Card>
<CardHeader>
<CardTitle>App Updates</CardTitle>
</CardHeader>
@@ -29,110 +29,97 @@ export function UpdateStatus() {
<div className="flex items-center justify-between">
<div className="space-y-1">
<div className="text-sm font-medium">Current Version</div>
<div className="text-sm text-muted-foreground">
v{currentVersion}
{isDev ? ' (dev)' : ''}
</div>
<div className="text-sm text-muted-foreground">v{currentVersion}</div>
</div>
{!isDev && (
<Button
onClick={checkForUpdates}
disabled={status.checking || status.downloading || status.readyToInstall}
variant="outline"
size="sm"
>
<RefreshCw className={`h-4 w-4 mr-2 ${status.checking ? 'animate-spin' : ''}`} />
Check for Updates
</Button>
)}
<Button
onClick={checkForUpdates}
disabled={status.checking || status.downloading || status.readyToInstall}
variant="outline"
size="sm"
>
<HugeiconsIcon icon={Refresh01Icon} size={16} className={`h-4 w-4 mr-2 ${status.checking ? 'animate-spin' : ''}`} />
Check for Updates
</Button>
</div>
{isDev ? (
<div className="text-sm text-muted-foreground">
Auto-updates are disabled in development mode.
{status.checking && (
<div className="flex items-center gap-2 text-sm text-muted-foreground">
<HugeiconsIcon icon={Refresh01Icon} size={16} className="h-4 w-4 animate-spin" />
Checking for updates...
</div>
) : (
<>
{status.checking && (
<div className="flex items-center gap-2 text-sm text-muted-foreground">
<RefreshCw className="h-4 w-4 animate-spin" />
Checking for updates...
</div>
)}
)}
{status.error && (
<div className="flex items-center gap-2 text-sm text-destructive">
<AlertCircle className="h-4 w-4" />
{status.error}
</div>
)}
{status.error && (
<div className="flex items-center gap-2 text-sm text-destructive">
<HugeiconsIcon icon={AlertCircleIcon} size={16} className="h-4 w-4" />
{status.error}
</div>
)}
{status.available && !status.downloading && !status.readyToInstall && (
<div className="space-y-3 p-4 border rounded-lg bg-primary/5">
<div className="flex items-center justify-between">
<div>
<div className="font-semibold">Update Available</div>
<div className="text-sm text-muted-foreground">Version {status.version}</div>
</div>
<Badge>New</Badge>
{status.available && !status.downloading && !status.readyToInstall && (
<div className="space-y-3 p-4 border rounded-lg bg-primary/5">
<div className="flex items-center justify-between">
<div>
<div className="font-semibold">Update Available</div>
<div className="text-sm text-muted-foreground">Version {status.version}</div>
</div>
<Badge>New</Badge>
</div>
<Button onClick={downloadAndInstall} className="w-full" size="sm">
<HugeiconsIcon icon={Download01Icon} size={16} className="h-4 w-4 mr-2" />
Download Update
</Button>
</div>
)}
{status.downloading && (
<div className="space-y-2">
<div className="flex items-center justify-between text-sm">
<div className="flex items-center gap-2">
<HugeiconsIcon icon={Download01Icon} size={16} className="h-4 w-4" />
Downloading update...
</div>
{status.downloadProgress !== undefined && (
<span className="text-muted-foreground">{status.downloadProgress}%</span>
)}
</div>
<Progress value={status.downloadProgress} />
{status.downloadedBytes !== undefined &&
status.totalBytes !== undefined &&
status.totalBytes > 0 && (
<div className="text-xs text-muted-foreground">
{(status.downloadedBytes / 1024 / 1024).toFixed(1)} MB /{' '}
{(status.totalBytes / 1024 / 1024).toFixed(1)} MB
</div>
<Button onClick={downloadAndInstall} className="w-full" size="sm">
<Download className="h-4 w-4 mr-2" />
Download Update
</Button>
</div>
)}
)}
</div>
)}
{status.downloading && (
<div className="space-y-2">
<div className="flex items-center justify-between text-sm">
<div className="flex items-center gap-2">
<Download className="h-4 w-4" />
Downloading update...
</div>
{status.downloadProgress !== undefined && (
<span className="text-muted-foreground">{status.downloadProgress}%</span>
)}
</div>
<Progress value={status.downloadProgress} />
{status.downloadedBytes !== undefined &&
status.totalBytes !== undefined &&
status.totalBytes > 0 && (
<div className="text-xs text-muted-foreground">
{(status.downloadedBytes / 1024 / 1024).toFixed(1)} MB /{' '}
{(status.totalBytes / 1024 / 1024).toFixed(1)} MB
</div>
)}
</div>
)}
{status.readyToInstall && (
<div className="space-y-3 p-4 border rounded-lg bg-accent/30 border-accent/50">
<div className="flex items-center gap-2">
<div>
<div className="font-semibold">Update Ready to Install</div>
<div className="text-sm text-muted-foreground">
Version {status.version} has been downloaded
</div>
</div>
</div>
{status.readyToInstall && (
<div className="space-y-3 p-4 border rounded-lg bg-accent/30 border-accent/50">
<div className="flex items-center gap-2">
<div>
<div className="font-semibold">Update Ready to Install</div>
<div className="text-sm text-muted-foreground">
The app needs to restart to complete the installation. You can do this now or
later at your convenience.
Version {status.version} has been downloaded
</div>
<Button onClick={restartAndInstall} className="w-full" size="sm">
<RefreshCw className="h-4 w-4 mr-2" />
Restart Now
</Button>
</div>
)}
</div>
<div className="text-sm text-muted-foreground">
The app needs to restart to complete the installation. You can do this now or later at
your convenience.
</div>
<Button onClick={restartAndInstall} className="w-full" size="sm">
<HugeiconsIcon icon={Refresh01Icon} size={16} className="h-4 w-4 mr-2" />
Restart Now
</Button>
</div>
)}
{!status.available && !status.checking && !status.error && (
<div className="flex items-center gap-2 text-sm text-muted-foreground">
You're up to date
</div>
)}
</>
{!status.available && !status.checking && !status.error && status.checking === false && (
<div className="flex items-center gap-2 text-sm text-muted-foreground">
You're up to date
</div>
)}
</CardContent>
</Card>
+8 -12
View File
@@ -1,25 +1,21 @@
import { ConnectionForm } from '@/components/ServerSettings/ConnectionForm';
import { GenerationSettings } from '@/components/ServerSettings/GenerationSettings';
import { GpuAcceleration } from '@/components/ServerSettings/GpuAcceleration';
import { DataFolders } from '@/components/ServerSettings/DataFolders';
import { ProviderSettings } from '@/components/ServerSettings/ProviderSettings';
import { ServerStatus } from '@/components/ServerSettings/ServerStatus';
import { UpdateStatus } from '@/components/ServerSettings/UpdateStatus';
import { BOTTOM_SAFE_AREA_PADDING } from '@/lib/constants/ui';
import { cn } from '@/lib/utils/cn';
import { usePlatform } from '@/platform/PlatformContext';
import { usePlayerStore } from '@/stores/playerStore';
export function ServerTab() {
const platform = usePlatform();
const isPlayerVisible = !!usePlayerStore((state) => state.audioUrl);
return (
<div
className={cn('overflow-y-auto flex flex-col', isPlayerVisible && BOTTOM_SAFE_AREA_PADDING)}
>
<div className="space-y-4 overflow-y-auto flex flex-col">
<div className="grid gap-4 md:grid-cols-2">
<ConnectionForm />
<GenerationSettings />
{platform.metadata.isTauri && <GpuAcceleration />}
{platform.metadata.isTauri && <UpdateStatus />}
<ServerStatus />
</div>
<ProviderSettings />
<DataFolders />
{platform.metadata.isTauri && <UpdateStatus />}
<div className="py-8 text-center text-sm text-muted-foreground">
Created by{' '}
<a
+41 -61
View File
@@ -1,34 +1,37 @@
import {
Book01Icon,
Mic01Icon,
PackageIcon,
ServerStack01Icon,
SpeakerIcon,
VolumeHighIcon,
} from '@hugeicons/core-free-icons';
import { HugeiconsIcon } from '@hugeicons/react';
import { Icon } from '@iconify/react';
import { Link, useMatchRoute } from '@tanstack/react-router';
import { AudioLines, Box, Mic, Server, Speaker, Volume2, Wand2 } from 'lucide-react';
import { useEffect, useState } from 'react';
import voiceboxLogo from '@/assets/voicebox-logo.png';
import { cn } from '@/lib/utils/cn';
import { usePlatform } from '@/platform/PlatformContext';
import type { UpdateStatus } from '@/platform/types';
import { useGenerationStore } from '@/stores/generationStore';
import { usePlayerStore } from '@/stores/playerStore';
import { version } from '../../package.json';
interface SidebarProps {
isMacOS?: boolean;
}
const tabs = [
{ id: 'main', path: '/', icon: Volume2, label: 'Generate' },
{ id: 'stories', path: '/stories', icon: AudioLines, label: 'Stories' },
{ id: 'voices', path: '/voices', icon: Mic, label: 'Voices' },
{ id: 'effects', path: '/effects', icon: Wand2, label: 'Effects' },
{ id: 'audio', path: '/audio', icon: Speaker, label: 'Audio' },
{ id: 'models', path: '/models', icon: Box, label: 'Models' },
{ id: 'server', path: '/server', icon: Server, label: 'Server' },
{ id: 'main', path: '/', icon: VolumeHighIcon, label: 'Generate' },
{ id: 'stories', path: '/stories', icon: Book01Icon, label: 'Stories' },
{ id: 'voices', path: '/voices', icon: Mic01Icon, label: 'Voices' },
{ id: 'audio', path: '/audio', icon: SpeakerIcon, label: 'Audio' },
{ id: 'models', path: '/models', icon: PackageIcon, label: 'Models' },
{ id: 'server', path: '/server', icon: ServerStack01Icon, label: 'Server' },
];
export function Sidebar({ isMacOS }: SidebarProps) {
const isGenerating = useGenerationStore((state) => state.isGenerating);
const audioUrl = usePlayerStore((state) => state.audioUrl);
const isPlayerVisible = !!audioUrl;
const matchRoute = useMatchRoute();
const isPlayerOpen = !!usePlayerStore((s) => s.audioUrl);
const platform = usePlatform();
const [updateStatus, setUpdateStatus] = useState<UpdateStatus>(platform.updater.getStatus());
useEffect(() => platform.updater.subscribe(setUpdateStatus), [platform.updater]);
return (
<div
@@ -39,72 +42,49 @@ export function Sidebar({ isMacOS }: SidebarProps) {
>
{/* Logo */}
<div className="mb-2">
<img
src={voiceboxLogo}
alt="Voicebox"
className="w-12 h-12 object-contain"
style={{
filter:
'drop-shadow(0 0 6px hsl(var(--accent) / 0.5)) drop-shadow(0 0 14px hsl(var(--accent) / 0.35)) drop-shadow(0 0 28px hsl(var(--accent) / 0.2))',
}}
/>
<img src={voiceboxLogo} alt="Voicebox" className="w-12 h-12 object-contain" />
</div>
{/* Navigation Buttons */}
<div className="flex flex-col gap-3">
{tabs.map((tab, index) => {
{tabs.map((tab) => {
const Icon = tab.icon;
// For index route, use exact match; for others, use default matching
const isActive =
tab.path === '/' ? matchRoute({ to: '/', exact: true }) : matchRoute({ to: tab.path });
// Accent fades as buttons get further from the logo
const accentOpacity = Math.max(0.08, 0.5 - index * 0.07);
tab.path === '/' ? matchRoute({ to: '/' }) : matchRoute({ to: tab.path });
return (
<Link
key={tab.id}
to={tab.path}
className={cn(
'relative w-12 h-12 rounded-full flex items-center justify-center transition-all duration-200 overflow-hidden',
isActive
? 'bg-white/[0.07] text-foreground shadow-lg backdrop-blur-sm border border-white/[0.08]'
: 'text-muted-foreground hover:bg-muted/50',
'w-12 h-12 rounded-full flex items-center justify-center transition-all duration-200',
'hover:bg-muted/50',
isActive ? 'bg-muted/50 text-foreground shadow-lg' : 'text-muted-foreground',
)}
title={tab.label}
aria-label={tab.label}
>
{isActive && (
<div
className="absolute inset-0 rounded-full pointer-events-none"
style={{
maskImage: 'linear-gradient(to bottom, black, transparent 60%)',
WebkitMaskImage: 'linear-gradient(to bottom, black, transparent 60%)',
border: `1px solid hsl(var(--accent) / ${accentOpacity})`,
}}
/>
)}
<Icon className="h-5 w-5 relative z-10" />
<HugeiconsIcon icon={Icon} size={20} className="h-5 w-5" />
</Link>
);
})}
</div>
{/* Version */}
<div
className="mt-auto flex flex-col items-center gap-1.5 transition-all duration-300"
style={{ paddingBottom: isPlayerOpen ? '7rem' : undefined }}
>
<span className="text-[10px] text-muted-foreground/50">v{version}</span>
{updateStatus.available && (
<Link
to="/server"
className="text-[9px] font-semibold tracking-wide uppercase px-2 py-0.5 rounded-full bg-accent/15 text-accent hover:bg-accent/25 transition-colors"
>
Update
</Link>
)}
</div>
{/* Spacer to push loader to bottom */}
<div className="flex-1" />
{/* Generation Loader */}
{isGenerating && (
<div
className={cn(
'w-full flex items-center justify-center transition-all duration-200',
isPlayerVisible ? 'mb-[120px]' : 'mb-0',
)}
>
<Icon icon="svg-spinners:ring-resize" className="h-6 w-6 text-accent animate-spin" />
</div>
)}
</div>
);
}
+1 -4
View File
@@ -1,11 +1,8 @@
import { FloatingGenerateBox } from '@/components/Generation/FloatingGenerateBox';
import { usePlayerStore } from '@/stores/playerStore';
import { StoryContent } from './StoryContent';
import { StoryList } from './StoryList';
export function StoriesTab() {
const audioUrl = usePlayerStore((state) => state.audioUrl);
return (
<div className="flex flex-col h-full min-h-0 overflow-hidden">
{/* Main content area */}
@@ -21,7 +18,7 @@ export function StoriesTab() {
</div>
{/* Floating Generate Box - position is managed via storyStore.trackEditorHeight */}
<FloatingGenerateBox showVoiceSelector isPlayerOpen={!!audioUrl} />
<FloatingGenerateBox showVoiceSelector />
</div>
</div>
);
+13 -26
View File
@@ -1,7 +1,7 @@
import { useSortable } from '@dnd-kit/sortable';
import { CSS } from '@dnd-kit/utilities';
import { GripVertical, Mic, MoreHorizontal, Play, Trash2 } from 'lucide-react';
import { useState } from 'react';
import { HugeiconsIcon } from '@hugeicons/react';
import { DragDropVerticalIcon, MoreHorizontalIcon, PlayIcon, Delete01Icon } from '@hugeicons/core-free-icons';
import { Button } from '@/components/ui/button';
import {
DropdownMenu,
@@ -10,10 +10,10 @@ import {
DropdownMenuTrigger,
} from '@/components/ui/dropdown-menu';
import { Textarea } from '@/components/ui/textarea';
import { ProfileAvatar } from '@/components/VoiceProfiles/ProfileAvatar';
import type { StoryItemDetail } from '@/lib/api/types';
import { cn } from '@/lib/utils/cn';
import { useStoryStore } from '@/stores/storyStore';
import { useServerStore } from '@/stores/serverStore';
interface StoryChatItemProps {
item: StoryItemDetail;
@@ -35,10 +35,6 @@ export function StoryChatItem({
isDragging,
}: StoryChatItemProps) {
const seek = useStoryStore((state) => state.seek);
const serverUrl = useServerStore((state) => state.serverUrl);
const [avatarError, setAvatarError] = useState(false);
const avatarUrl = `${serverUrl}/profiles/${item.profile_id}/avatar`;
// Check if this item is currently playing based on timecode
const itemStartMs = item.start_time_ms;
@@ -74,27 +70,18 @@ export function StoryChatItem({
className="shrink-0 cursor-grab active:cursor-grabbing touch-none text-muted-foreground hover:text-foreground transition-colors"
{...dragHandleProps}
>
<GripVertical className="h-5 w-5" />
<HugeiconsIcon icon={DragDropVerticalIcon} size={20} className="h-5 w-5" />
</button>
)}
{/* Voice Avatar */}
<div className="shrink-0">
<div className="h-10 w-10 rounded-full bg-muted flex items-center justify-center overflow-hidden">
{!avatarError ? (
<img
src={avatarUrl}
alt={`${item.profile_name} avatar`}
className={cn(
'h-full w-full object-cover transition-all duration-200',
!isCurrentlyPlaying && 'grayscale'
)}
onError={() => setAvatarError(true)}
/>
) : (
<Mic className="h-5 w-5 text-muted-foreground" />
)}
</div>
<ProfileAvatar
profileId={item.profile_id}
size="lg"
grayscale={!isCurrentlyPlaying}
alt={`${item.profile_name} avatar`}
/>
</div>
{/* Content */}
@@ -119,16 +106,16 @@ export function StoryChatItem({
<DropdownMenu>
<DropdownMenuTrigger asChild>
<Button variant="ghost" size="icon" className="h-8 w-8" aria-label="Actions">
<MoreHorizontal className="h-4 w-4" />
<HugeiconsIcon icon={MoreHorizontalIcon} size={16} className="h-4 w-4" />
</Button>
</DropdownMenuTrigger>
<DropdownMenuContent align="end">
<DropdownMenuItem onClick={handlePlay}>
<Play className="mr-2 h-4 w-4" />
<HugeiconsIcon icon={PlayIcon} size={16} className="mr-2 h-4 w-4" />
Play from here
</DropdownMenuItem>
<DropdownMenuItem onClick={onRemove} className="text-destructive focus:text-destructive">
<Trash2 className="mr-2 h-4 w-4" />
<HugeiconsIcon icon={Delete01Icon} size={16} className="mr-2 h-4 w-4" />
Remove from Story
</DropdownMenuItem>
</DropdownMenuContent>
+10 -36
View File
@@ -13,11 +13,9 @@ import {
sortableKeyboardCoordinates,
verticalListSortingStrategy,
} from '@dnd-kit/sortable';
import { Link } from '@tanstack/react-router';
import { AnimatePresence, motion } from 'framer-motion';
import { Download, Plus } from 'lucide-react';
import { HugeiconsIcon } from '@hugeicons/react';
import { Download01Icon, Add01Icon } from '@hugeicons/core-free-icons';
import { useEffect, useMemo, useRef, useState } from 'react';
import Loader from 'react-loaders';
import { Button } from '@/components/ui/button';
import { Input } from '@/components/ui/input';
import { Popover, PopoverContent, PopoverTrigger } from '@/components/ui/popover';
@@ -31,7 +29,6 @@ import {
useStory,
} from '@/lib/hooks/useStories';
import { useStoryPlayback } from '@/lib/hooks/useStoryPlayback';
import { useGenerationStore } from '@/stores/generationStore';
import { useStoryStore } from '@/stores/storyStore';
import { SortableStoryChatItem } from './StoryChatItem';
@@ -44,7 +41,6 @@ export function StoryContent() {
const addStoryItem = useAddStoryItem();
const { toast } = useToast();
const scrollRef = useRef<HTMLDivElement>(null);
const pendingCount = useGenerationStore((s) => s.pendingGenerationIds.size);
// Add generation popover state
const [searchQuery, setSearchQuery] = useState('');
@@ -58,9 +54,9 @@ export function StoryContent() {
const query = searchQuery.toLowerCase();
return historyData.items.filter(
(gen) =>
gen.status === 'completed' &&
!storyGenerationIds.has(gen.id) &&
(gen.text.toLowerCase().includes(query) || gen.profile_name.toLowerCase().includes(query)),
(gen.text.toLowerCase().includes(query) ||
gen.profile_name.toLowerCase().includes(query)),
);
}, [historyData, story, searchQuery]);
@@ -272,35 +268,11 @@ export function StoryContent() {
<p className="text-sm text-muted-foreground mt-1">{story.description}</p>
)}
</div>
<div className="flex gap-2 items-center">
<AnimatePresence>
{pendingCount > 0 && (
<motion.div
initial={{ opacity: 0, scale: 0.9, width: 0 }}
animate={{ opacity: 1, scale: 1, width: 'auto' }}
exit={{ opacity: 0, scale: 0.9, width: 0 }}
transition={{ duration: 0.2 }}
>
<Link
to="/"
className="flex items-center gap-2 h-8 pl-1.5 pr-3 rounded-full bg-card border border-border hover:bg-muted/50 transition-all duration-200 cursor-pointer"
>
<div className="shrink-0 w-10 h-5 overflow-hidden flex items-center justify-center">
<div className="scale-[0.45]">
<Loader type="line-scale" active />
</div>
</div>
<span className="text-xs text-muted-foreground whitespace-nowrap">
Generating {pendingCount} {pendingCount === 1 ? 'audio' : 'audios'}
</span>
</Link>
</motion.div>
)}
</AnimatePresence>
<div className="flex gap-2">
<Popover open={isAddOpen} onOpenChange={setIsAddOpen}>
<PopoverTrigger asChild>
<Button variant="outline" size="sm">
<Plus className="mr-2 h-4 w-4" />
<HugeiconsIcon icon={Add01Icon} size={16} className="mr-2 h-4 w-4" />
Add
</Button>
</PopoverTrigger>
@@ -316,7 +288,9 @@ export function StoryContent() {
<div className="max-h-60 overflow-y-auto">
{availableGenerations.length === 0 ? (
<div className="p-4 text-center text-sm text-muted-foreground">
{searchQuery ? 'No matching generations found' : 'No available generations'}
{searchQuery
? 'No matching generations found'
: 'No available generations'}
</div>
) : (
availableGenerations.map((gen) => (
@@ -343,7 +317,7 @@ export function StoryContent() {
onClick={handleExportAudio}
disabled={exportAudio.isPending}
>
<Download className="mr-2 h-4 w-4" />
<HugeiconsIcon icon={Download01Icon} size={16} className="mr-2 h-4 w-4" />
Export Audio
</Button>
)}
+81 -95
View File
@@ -1,5 +1,6 @@
import { BookOpen, MoreHorizontal, Pencil, Plus, Trash2 } from 'lucide-react';
import { useEffect, useState } from 'react';
import { HugeiconsIcon } from '@hugeicons/react';
import { Add01Icon, Book01Icon, MoreHorizontalIcon, PencilIcon, Delete01Icon } from '@hugeicons/core-free-icons';
import { useState, useMemo } from 'react';
import {
AlertDialog,
AlertDialogAction,
@@ -29,13 +30,7 @@ import { Input } from '@/components/ui/input';
import { Label } from '@/components/ui/label';
import { Textarea } from '@/components/ui/textarea';
import { useToast } from '@/components/ui/use-toast';
import {
useCreateStory,
useDeleteStory,
useStories,
useStory,
useUpdateStory,
} from '@/lib/hooks/useStories';
import { useStories, useCreateStory, useUpdateStory, useDeleteStory, useStory } from '@/lib/hooks/useStories';
import { cn } from '@/lib/utils/cn';
import { formatDate } from '@/lib/utils/format';
import { useStoryStore } from '@/stores/storyStore';
@@ -45,7 +40,7 @@ export function StoryList() {
const selectedStoryId = useStoryStore((state) => state.selectedStoryId);
const setSelectedStoryId = useStoryStore((state) => state.setSelectedStoryId);
const trackEditorHeight = useStoryStore((state) => state.trackEditorHeight);
const { data: selectedStory } = useStory(selectedStoryId);
const { data: currentStory } = useStory(selectedStoryId);
const createStory = useCreateStory();
const updateStory = useUpdateStory();
const deleteStory = useDeleteStory();
@@ -62,12 +57,15 @@ export function StoryList() {
const [newStoryDescription, setNewStoryDescription] = useState('');
const { toast } = useToast();
// Auto-select the first story when the list loads with no selection
useEffect(() => {
if (!selectedStoryId && stories && stories.length > 0) {
setSelectedStoryId(stories[0].id);
}
}, [selectedStoryId, stories, setSelectedStoryId]);
// Calculate bottom padding to account for FloatingGenerateBox and StoryTrackEditor
const hasTrackEditor = currentStory && currentStory.items.length > 0;
const bottomPadding = useMemo(() => {
// FloatingGenerateBox height (~100px) + gap (24px)
const generateBoxHeight = 124;
// Track editor height when visible
const editorHeight = hasTrackEditor ? trackEditorHeight + 24 : 0;
return generateBoxHeight + editorHeight;
}, [hasTrackEditor, trackEditorHeight]);
const handleCreateStory = () => {
if (!newStoryName.trim()) {
@@ -185,98 +183,86 @@ export function StoryList() {
}
const storyList = stories || [];
const hasTrackEditor = selectedStoryId && selectedStory && selectedStory.items.length > 0;
return (
<div className="h-full flex flex-col relative overflow-hidden">
{/* Scroll Mask */}
<div className="absolute top-0 left-0 right-0 h-16 bg-gradient-to-b from-background to-transparent z-10 pointer-events-none" />
{/* Fixed Header */}
<div className="absolute top-0 left-0 right-0 z-20">
<div className="flex items-center justify-between mb-4 px-1">
<h2 className="text-2xl font-bold">Stories</h2>
<Button onClick={() => setCreateDialogOpen(true)} size="sm">
<Plus className="mr-2 h-4 w-4" />
New Story
</Button>
</div>
<div className="flex flex-col h-full min-h-0">
{/* Header */}
<div className="flex items-center justify-between mb-4 px-1">
<h2 className="text-2xl font-bold">Stories</h2>
<Button onClick={() => setCreateDialogOpen(true)} size="sm">
<HugeiconsIcon icon={Add01Icon} size={16} className="mr-2 h-4 w-4" />
New Story
</Button>
</div>
{/* Scrollable Story List */}
<div
className="flex-1 overflow-y-auto pt-14 relative z-0"
style={{ paddingBottom: hasTrackEditor ? `${trackEditorHeight + 140}px` : '170px' }}
{/* Story List */}
<div
className="flex-1 min-h-0 overflow-y-auto space-y-2"
style={{ paddingBottom: `${bottomPadding}px` }}
>
{storyList.length === 0 ? (
<div className="text-center py-12 px-5 border-2 border-dashed border-muted rounded-2xl text-muted-foreground">
<BookOpen className="h-12 w-12 mx-auto mb-4 opacity-50" />
<HugeiconsIcon icon={Book01Icon} size={48} className="h-12 w-12 mx-auto mb-4 opacity-50" />
<p className="text-sm">No stories yet</p>
<p className="text-xs mt-2">Create your first story to get started</p>
</div>
) : (
<div className="space-y-0.5">
{storyList.map((story) => (
<div
key={story.id}
role="button"
tabIndex={0}
className={cn(
'px-5 py-3 rounded-lg transition-colors group flex items-center cursor-pointer',
selectedStoryId === story.id ? 'bg-muted' : 'hover:bg-muted/50',
)}
aria-label={`Story ${story.name}, ${story.item_count} ${story.item_count === 1 ? 'item' : 'items'}, ${formatDate(story.updated_at)}`}
aria-pressed={selectedStoryId === story.id}
onClick={() => setSelectedStoryId(story.id)}
onKeyDown={(e) => {
if (e.target !== e.currentTarget) return;
if (e.key === 'Enter' || e.key === ' ') {
e.preventDefault();
setSelectedStoryId(story.id);
}
}}
>
<div className="flex items-start justify-between gap-2 w-full min-w-0">
<div className="flex-1 min-w-0 text-left overflow-hidden">
<h3 className="text-sm font-medium truncate">{story.name}</h3>
<div className="flex items-center gap-2 mt-1 text-xs text-muted-foreground">
<span>
{story.item_count} {story.item_count === 1 ? 'item' : 'items'}
</span>
<span>·</span>
<span>{formatDate(story.updated_at)}</span>
</div>
storyList.map((story) => (
<div
key={story.id}
className={cn(
'h-24 p-4 border rounded-2xl transition-colors group flex items-center',
selectedStoryId === story.id && 'bg-muted border-primary',
)}
>
<div className="flex items-start justify-between gap-2 w-full min-w-0">
<button
type="button"
className="flex-1 min-w-0 text-left cursor-pointer overflow-hidden"
onClick={() => setSelectedStoryId(story.id)}
>
<h3 className="font-medium truncate">{story.name}</h3>
{story.description && (
<p className="text-sm text-muted-foreground mt-1 truncate">
{story.description}
</p>
)}
<div className="flex items-center gap-3 mt-2 text-xs text-muted-foreground">
<span>
{story.item_count} {story.item_count === 1 ? 'item' : 'items'}
</span>
<span>•</span>
<span>{formatDate(story.updated_at)}</span>
</div>
<DropdownMenu>
<DropdownMenuTrigger asChild>
<Button
variant="ghost"
size="icon"
className="h-7 w-7 opacity-0 group-hover:opacity-100 transition-opacity"
onClick={(e) => e.stopPropagation()}
aria-label={`Actions for ${story.name}`}
>
<MoreHorizontal className="h-3.5 w-3.5" />
</Button>
</DropdownMenuTrigger>
<DropdownMenuContent align="end">
<DropdownMenuItem onClick={() => handleEditClick(story)}>
<Pencil className="mr-2 h-4 w-4" />
Edit
</DropdownMenuItem>
<DropdownMenuItem
onClick={() => handleDeleteClick(story.id)}
className="text-destructive focus:text-destructive"
>
<Trash2 className="mr-2 h-4 w-4" />
Delete
</DropdownMenuItem>
</DropdownMenuContent>
</DropdownMenu>
</div>
</button>
<DropdownMenu>
<DropdownMenuTrigger asChild>
<Button
variant="ghost"
size="icon"
className="h-8 w-8 opacity-0 group-hover:opacity-100 transition-opacity"
onClick={(e) => e.stopPropagation()}
>
<HugeiconsIcon icon={MoreHorizontalIcon} size={16} className="h-4 w-4" />
</Button>
</DropdownMenuTrigger>
<DropdownMenuContent align="end">
<DropdownMenuItem onClick={() => handleEditClick(story)}>
<HugeiconsIcon icon={PencilIcon} size={16} className="mr-2 h-4 w-4" />
Edit
</DropdownMenuItem>
<DropdownMenuItem
onClick={() => handleDeleteClick(story.id)}
className="text-destructive focus:text-destructive"
>
<HugeiconsIcon icon={Delete01Icon} size={16} className="mr-2 h-4 w-4" />
Delete
</DropdownMenuItem>
</DropdownMenuContent>
</DropdownMenu>
</div>
))}
</div>
</div>
))
)}
</div>
@@ -1,25 +1,18 @@
import { HugeiconsIcon } from '@hugeicons/react';
import {
Check,
Copy,
GalleryVerticalEnd,
GripHorizontal,
Minus,
Pause,
Play,
Plus,
Scissors,
Square,
Trash2,
} from 'lucide-react';
Copy01Icon,
DragDropHorizontalIcon,
RemoveIcon,
PauseIcon,
PlayIcon,
Add01Icon,
Scissor01Icon,
SquareIcon,
Delete01Icon,
} from '@hugeicons/core-free-icons';
import { useCallback, useEffect, useMemo, useRef, useState } from 'react';
import WaveSurfer from 'wavesurfer.js';
import { Button } from '@/components/ui/button';
import {
DropdownMenu,
DropdownMenuContent,
DropdownMenuItem,
DropdownMenuTrigger,
} from '@/components/ui/dropdown-menu';
import { useToast } from '@/components/ui/use-toast';
import { apiClient } from '@/lib/api/client';
import type { StoryItemDetail } from '@/lib/api/types';
@@ -27,7 +20,6 @@ import {
useDuplicateStoryItem,
useMoveStoryItem,
useRemoveStoryItem,
useSetStoryItemVersion,
useSplitStoryItem,
useTrimStoryItem,
} from '@/lib/hooks/useStories';
@@ -37,14 +29,12 @@ import { useStoryStore } from '@/stores/storyStore';
// Clip waveform component with trim support
function ClipWaveform({
generationId,
versionId,
width,
trimStartMs,
trimEndMs,
duration,
}: {
generationId: string;
versionId?: string;
width: number;
trimStartMs: number;
trimEndMs: number;
@@ -90,9 +80,7 @@ function ClipWaveform({
wavesurferRef.current = wavesurfer;
const audioUrl = versionId
? apiClient.getVersionAudioUrl(versionId)
: apiClient.getAudioUrl(generationId);
const audioUrl = apiClient.getAudioUrl(generationId);
wavesurfer.load(audioUrl).catch(() => {
// Ignore load errors
});
@@ -101,7 +89,7 @@ function ClipWaveform({
wavesurfer.destroy();
wavesurferRef.current = null;
};
}, [generationId, versionId, fullWaveformWidth]);
}, [generationId, fullWaveformWidth]);
return (
<div className="w-full h-full opacity-60 overflow-hidden">
@@ -148,57 +136,12 @@ export function StoryTrackEditor({ storyId, items }: StoryTrackEditorProps) {
const splitItem = useSplitStoryItem();
const duplicateItem = useDuplicateStoryItem();
const removeItem = useRemoveStoryItem();
const setItemVersion = useSetStoryItemVersion();
const { toast } = useToast();
// Selection state
const selectedClipId = useStoryStore((state) => state.selectedClipId);
const setSelectedClipId = useStoryStore((state) => state.setSelectedClipId);
// Selected clip item (for version picker)
const selectedItem = useMemo(
() => (selectedClipId ? items.find((i) => i.id === selectedClipId) : undefined),
[selectedClipId, items],
);
const selectedItemVersions = selectedItem?.versions;
const hasMultipleVersions = selectedItemVersions && selectedItemVersions.length > 1;
// Determine which version label is active for the selected clip
const activeVersionLabel = useMemo(() => {
if (!selectedItem || !selectedItemVersions) return null;
// If the item has a pinned version_id, find its label
if (selectedItem.version_id) {
const pinned = selectedItemVersions.find((v) => v.id === selectedItem.version_id);
return pinned?.label ?? null;
}
// Otherwise use the generation's default version
const defaultVersion = selectedItemVersions.find((v) => v.is_default);
return defaultVersion?.label ?? null;
}, [selectedItem, selectedItemVersions]);
const handleSetVersion = useCallback(
(versionId: string | null) => {
if (!selectedClipId) return;
setItemVersion.mutate(
{
storyId,
itemId: selectedClipId,
data: { version_id: versionId },
},
{
onError: (error) => {
toast({
title: 'Failed to set version',
description: error instanceof Error ? error.message : String(error),
variant: 'destructive',
});
},
},
);
},
[selectedClipId, storyId, setItemVersion, toast],
);
// Trim state
const [trimmingItem, setTrimmingItem] = useState<string | null>(null);
const [trimSide, setTrimSide] = useState<'start' | 'end' | null>(null);
@@ -781,7 +724,7 @@ export function StoryTrackEditor({ storyId, items }: StoryTrackEditorProps) {
onMouseDown={handleResizeStart}
aria-label="Resize track editor"
>
<GripHorizontal className="h-3 w-3 text-muted-foreground/50 group-hover:text-muted-foreground" />
<HugeiconsIcon icon={DragDropHorizontalIcon} size={12} className="h-3 w-3 text-muted-foreground/50 group-hover:text-muted-foreground" />
</button>
{/* Toolbar */}
@@ -794,9 +737,8 @@ export function StoryTrackEditor({ storyId, items }: StoryTrackEditorProps) {
className="h-7 w-7"
onClick={handlePlayPause}
title="Play/Pause (Space)"
aria-label={isCurrentlyPlaying ? 'Pause' : 'Play'}
>
{isCurrentlyPlaying ? <Pause className="h-4 w-4" /> : <Play className="h-4 w-4" />}
{isCurrentlyPlaying ? <HugeiconsIcon icon={PauseIcon} size={16} className="h-4 w-4" /> : <HugeiconsIcon icon={PlayIcon} size={16} className="h-4 w-4" />}
</Button>
<Button
variant="ghost"
@@ -804,9 +746,8 @@ export function StoryTrackEditor({ storyId, items }: StoryTrackEditorProps) {
className="h-7 w-7"
onClick={handleStop}
disabled={!isCurrentlyPlaying}
aria-label="Stop"
>
<Square className="h-3 w-3" />
<HugeiconsIcon icon={SquareIcon} size={12} className="h-3 w-3" />
</Button>
<span className="text-xs text-muted-foreground tabular-nums ml-2">
{formatTime(currentTimeMs)} / {formatTime(totalDurationMs)}
@@ -822,9 +763,8 @@ export function StoryTrackEditor({ storyId, items }: StoryTrackEditorProps) {
className="h-7 w-7"
onClick={handleSplit}
title="Split at playhead (S)"
aria-label="Split at playhead"
>
<Scissors className="h-4 w-4" />
<HugeiconsIcon icon={Scissor01Icon} size={16} className="h-4 w-4" />
</Button>
<Button
variant="ghost"
@@ -832,9 +772,8 @@ export function StoryTrackEditor({ storyId, items }: StoryTrackEditorProps) {
className="h-7 w-7"
onClick={handleDuplicate}
title="Duplicate (Cmd/Ctrl+D)"
aria-label="Duplicate clip"
>
<Copy className="h-4 w-4" />
<HugeiconsIcon icon={Copy01Icon} size={16} className="h-4 w-4" />
</Button>
<Button
variant="ghost"
@@ -842,76 +781,20 @@ export function StoryTrackEditor({ storyId, items }: StoryTrackEditorProps) {
className="h-7 w-7"
onClick={handleDelete}
title="Delete (Delete/Backspace)"
aria-label="Delete clip"
>
<Trash2 className="h-4 w-4" />
<HugeiconsIcon icon={Delete01Icon} size={16} className="h-4 w-4" />
</Button>
{hasMultipleVersions && (
<>
<div className="w-px h-4 bg-border mx-1" />
<DropdownMenu>
<DropdownMenuTrigger asChild>
<Button
variant="ghost"
className="h-7 gap-1.5 px-2 text-xs"
title="Change version/take"
>
<GalleryVerticalEnd className="h-3.5 w-3.5" />
<span className="max-w-[80px] truncate">
{activeVersionLabel ?? 'default'}
</span>
</Button>
</DropdownMenuTrigger>
<DropdownMenuContent align="center" className="min-w-[160px]">
{selectedItemVersions.map((version) => {
const isActive = selectedItem?.version_id
? version.id === selectedItem.version_id
: version.is_default;
return (
<DropdownMenuItem
key={version.id}
onClick={() => handleSetVersion(version.id)}
className="gap-2 text-xs"
>
<Check
className={cn('h-3 w-3', isActive ? 'opacity-100' : 'opacity-0')}
/>
<span className="truncate">{version.label}</span>
{version.effects_chain && version.effects_chain.length > 0 && (
<span className="text-muted-foreground ml-auto text-[10px]">
{version.effects_chain.length} fx
</span>
)}
</DropdownMenuItem>
);
})}
</DropdownMenuContent>
</DropdownMenu>
</>
)}
</div>
)}
{/* Zoom controls - right side */}
<div className="flex items-center gap-2">
<span className="text-xs text-muted-foreground">Zoom:</span>
<Button
variant="ghost"
size="icon"
className="h-6 w-6"
onClick={handleZoomOut}
aria-label="Zoom out"
>
<Minus className="h-3 w-3" />
<Button variant="ghost" size="icon" className="h-6 w-6" onClick={handleZoomOut}>
<HugeiconsIcon icon={RemoveIcon} size={12} className="h-3 w-3" />
</Button>
<Button
variant="ghost"
size="icon"
className="h-6 w-6"
onClick={handleZoomIn}
aria-label="Zoom in"
>
<Plus className="h-3 w-3" />
<Button variant="ghost" size="icon" className="h-6 w-6" onClick={handleZoomIn}>
<HugeiconsIcon icon={Add01Icon} size={12} className="h-3 w-3" />
</Button>
</div>
</div>
@@ -955,7 +838,7 @@ export function StoryTrackEditor({ storyId, items }: StoryTrackEditorProps) {
type="button"
className="h-6 border-b bg-muted/20 sticky top-0 z-10 cursor-pointer text-left"
style={{ width: `${timelineWidth}px` }}
onClick={handleTimelineClick}
onClick={(e) => handleTimelineClick(e as unknown as React.MouseEvent<HTMLDivElement>)}
aria-label="Seek timeline"
>
{timeMarkers.map((ms) => (
@@ -996,7 +879,7 @@ export function StoryTrackEditor({ storyId, items }: StoryTrackEditorProps) {
<button
type="button"
className="absolute inset-0 z-0 cursor-pointer"
onClick={handleTimelineClick}
onClick={(e) => handleTimelineClick(e as unknown as React.MouseEvent<HTMLDivElement>)}
aria-label="Seek timeline"
/>
@@ -1059,7 +942,6 @@ export function StoryTrackEditor({ storyId, items }: StoryTrackEditorProps) {
<div className="absolute inset-0 top-3">
<ClipWaveform
generationId={item.generation_id}
versionId={item.version_id}
width={clipWidth}
trimStartMs={displayTrimStart}
trimEndMs={displayTrimEnd}
+6 -5
View File
@@ -1,7 +1,8 @@
const isWindows = navigator.userAgent.includes('Windows');
export function TitleBarDragRegion() {
if (isWindows) return null;
return <div data-tauri-drag-region className="fixed top-0 left-0 right-0 h-12 z-[9999]" />;
return (
<div
data-tauri-drag-region
className="fixed top-0 left-0 right-0 h-12 z-[9999]"
/>
);
}
@@ -1,4 +1,5 @@
import { Mic, Pause, Play, Square } from 'lucide-react';
import { HugeiconsIcon } from '@hugeicons/react';
import { Mic01Icon, PauseIcon, PlayIcon, SquareIcon } from '@hugeicons/core-free-icons';
import { memo, useEffect, useState } from 'react';
import { Visualizer } from 'react-sound-visualizer';
import { Button } from '@/components/ui/button';
@@ -58,7 +59,6 @@ export function AudioSampleRecording({
// Request microphone access when component mounts
useEffect(() => {
if (!showWaveform) return;
if (!navigator.mediaDevices || !navigator.mediaDevices.getUserMedia) return;
let stream: MediaStream | null = null;
@@ -96,7 +96,7 @@ export function AudioSampleRecording({
size="lg"
className="relative z-10 flex items-center gap-2"
>
<Mic className="h-5 w-5" />
<HugeiconsIcon icon={Mic01Icon} size={20} className="h-5 w-5" />
Start Recording
</Button>
<p className="relative z-10 text-sm text-muted-foreground text-center">
@@ -123,7 +123,7 @@ export function AudioSampleRecording({
onClick={onStop}
className="relative z-10 flex items-center gap-2 bg-accent text-accent-foreground hover:bg-accent/90"
>
<Square className="h-4 w-4" />
<HugeiconsIcon icon={SquareIcon} size={16} className="h-4 w-4" />
Stop Recording
</Button>
<p className="relative z-10 text-sm text-muted-foreground text-center">
@@ -135,19 +135,13 @@ export function AudioSampleRecording({
{file && !isRecording && (
<div className="flex flex-col items-center justify-center gap-4 p-4 border-2 border-primary rounded-lg bg-primary/5 min-h-[180px]">
<div className="flex items-center gap-2">
<Mic className="h-5 w-5 text-primary" />
<HugeiconsIcon icon={Mic01Icon} size={20} className="h-5 w-5 text-primary" />
<span className="font-medium">Recording complete</span>
</div>
<p className="text-sm text-muted-foreground text-center">File: {file.name}</p>
<div className="flex gap-2">
<Button
type="button"
size="icon"
variant="outline"
onClick={onPlayPause}
aria-label={isPlaying ? 'Pause' : 'Play'}
>
{isPlaying ? <Pause className="h-4 w-4" /> : <Play className="h-4 w-4" />}
<Button type="button" size="icon" variant="outline" onClick={onPlayPause}>
{isPlaying ? <HugeiconsIcon icon={PauseIcon} size={16} className="h-4 w-4" /> : <HugeiconsIcon icon={PlayIcon} size={16} className="h-4 w-4" />}
</Button>
<Button
type="button"
@@ -156,7 +150,7 @@ export function AudioSampleRecording({
disabled={isTranscribing}
className="flex items-center gap-2"
>
<Mic className="h-4 w-4" />
<HugeiconsIcon icon={Mic01Icon} size={16} className="h-4 w-4" />
{isTranscribing ? 'Transcribing...' : 'Transcribe'}
</Button>
<Button
@@ -1,4 +1,5 @@
import { Mic, Monitor, Pause, Play, Square } from 'lucide-react';
import { HugeiconsIcon } from '@hugeicons/react';
import { Mic01Icon, DeskIcon, PauseIcon, PlayIcon, SquareIcon } from '@hugeicons/core-free-icons';
import { Button } from '@/components/ui/button';
import { FormControl, FormItem, FormMessage } from '@/components/ui/form';
import { formatAudioDuration } from '@/lib/utils/audio';
@@ -35,7 +36,7 @@ export function AudioSampleSystem({
{!isRecording && !file && (
<div className="flex flex-col items-center justify-center gap-4 p-4 border-2 border-dashed rounded-lg min-h-[180px]">
<Button type="button" onClick={onStart} size="lg" className="flex items-center gap-2">
<Monitor className="h-5 w-5" />
<HugeiconsIcon icon={DeskIcon} size={20} className="h-5 w-5" />
Start Capture
</Button>
<p className="text-sm text-muted-foreground text-center">
@@ -60,7 +61,7 @@ export function AudioSampleSystem({
variant="destructive"
className="flex items-center gap-2"
>
<Square className="h-4 w-4" />
<HugeiconsIcon icon={SquareIcon} size={16} className="h-4 w-4" />
Stop Capture
</Button>
<p className="text-sm text-muted-foreground text-center">
@@ -72,19 +73,13 @@ export function AudioSampleSystem({
{file && !isRecording && (
<div className="flex flex-col items-center justify-center gap-4 p-4 border-2 border-primary rounded-lg bg-primary/5 min-h-[180px]">
<div className="flex items-center gap-2">
<Monitor className="h-5 w-5 text-primary" />
<HugeiconsIcon icon={DeskIcon} size={20} className="h-5 w-5 text-primary" />
<span className="font-medium">Capture complete</span>
</div>
<p className="text-sm text-muted-foreground text-center">File: {file.name}</p>
<div className="flex gap-2">
<Button
type="button"
size="icon"
variant="outline"
onClick={onPlayPause}
aria-label={isPlaying ? 'Pause' : 'Play'}
>
{isPlaying ? <Pause className="h-4 w-4" /> : <Play className="h-4 w-4" />}
<Button type="button" size="icon" variant="outline" onClick={onPlayPause}>
{isPlaying ? <HugeiconsIcon icon={PauseIcon} size={16} className="h-4 w-4" /> : <HugeiconsIcon icon={PlayIcon} size={16} className="h-4 w-4" />}
</Button>
<Button
type="button"
@@ -93,7 +88,7 @@ export function AudioSampleSystem({
disabled={isTranscribing}
className="flex items-center gap-2"
>
<Mic className="h-4 w-4" />
<HugeiconsIcon icon={Mic01Icon} size={16} className="h-4 w-4" />
{isTranscribing ? 'Transcribing...' : 'Transcribe'}
</Button>
<Button
@@ -1,4 +1,5 @@
import { Mic, Pause, Play, Upload } from 'lucide-react';
import { HugeiconsIcon } from '@hugeicons/react';
import { Mic01Icon, PauseIcon, PlayIcon, Upload01Icon } from '@hugeicons/core-free-icons';
import { useRef, useState } from 'react';
import { Button } from '@/components/ui/button';
import { FormControl, FormItem, FormMessage } from '@/components/ui/form';
@@ -89,7 +90,7 @@ export function AudioSampleUpload({
onClick={() => fileInputRef.current?.click()}
className="flex items-center gap-2"
>
<Upload className="h-5 w-5" />
<HugeiconsIcon icon={Upload01Icon} size={20} className="h-5 w-5" />
Choose File
</Button>
<p className="text-sm text-muted-foreground text-center">
@@ -99,7 +100,7 @@ export function AudioSampleUpload({
) : (
<>
<div className="flex items-center gap-2">
<Upload className="h-5 w-5 text-primary" />
<HugeiconsIcon icon={Upload01Icon} size={20} className="h-5 w-5 text-primary" />
<span className="font-medium">File uploaded</span>
</div>
<p className="text-sm text-muted-foreground text-center">File: {file.name}</p>
@@ -110,9 +111,8 @@ export function AudioSampleUpload({
variant="outline"
onClick={onPlayPause}
disabled={isValidating}
aria-label={isPlaying ? 'Pause' : 'Play'}
>
{isPlaying ? <Pause className="h-4 w-4" /> : <Play className="h-4 w-4" />}
{isPlaying ? <HugeiconsIcon icon={PauseIcon} size={16} className="h-4 w-4" /> : <HugeiconsIcon icon={PlayIcon} size={16} className="h-4 w-4" />}
</Button>
<Button
type="button"
@@ -121,7 +121,7 @@ export function AudioSampleUpload({
disabled={isTranscribing || isValidating || isDisabled}
className="flex items-center gap-2"
>
<Mic className="h-4 w-4" />
<HugeiconsIcon icon={Mic01Icon} size={16} className="h-4 w-4" />
{isTranscribing ? 'Transcribing...' : 'Transcribe'}
</Button>
<Button
@@ -0,0 +1,80 @@
import { HugeiconsIcon } from '@hugeicons/react';
import { Mic01Icon } from '@hugeicons/core-free-icons';
import { useState } from 'react';
import { cn } from '@/lib/utils/cn';
import { useServerStore } from '@/stores/serverStore';
interface ProfileAvatarProps {
profileId: string;
avatarPath?: string | null;
size?: 'sm' | 'md' | 'lg' | 'xl';
grayscale?: boolean;
className?: string;
alt?: string;
}
const sizeClasses = {
sm: 'h-6 w-6',
md: 'h-8 w-8',
lg: 'h-10 w-10',
xl: 'h-24 w-24',
};
const iconSizes = {
sm: 14,
md: 16,
lg: 20,
xl: 40,
};
const iconClassNames = {
sm: 'h-3.5 w-3.5',
md: 'h-4 w-4',
lg: 'h-5 w-5',
xl: 'h-10 w-10',
};
export function ProfileAvatar({
profileId,
avatarPath,
size = 'md',
grayscale = false,
className,
alt = 'Profile avatar',
}: ProfileAvatarProps) {
const [avatarError, setAvatarError] = useState(false);
const serverUrl = useServerStore((state) => state.serverUrl);
// If avatarPath is explicitly null or empty string, don't try to load avatar
// Otherwise, always try to load (avatarPath might not be available in all contexts)
const avatarUrl =
avatarPath === null || avatarPath === '' ? null : `${serverUrl}/profiles/${profileId}/avatar`;
return (
<div
className={cn(
sizeClasses[size],
'rounded-full bg-muted flex items-center justify-center shrink-0 overflow-hidden',
className,
)}
>
{avatarUrl && !avatarError ? (
<img
src={avatarUrl}
alt={alt}
className={cn(
'h-full w-full object-cover transition-all duration-200',
grayscale && 'grayscale',
)}
onError={() => setAvatarError(true)}
/>
) : (
<HugeiconsIcon
icon={Mic01Icon}
size={iconSizes[size]}
className={cn(iconClassNames[size], 'text-muted-foreground')}
/>
)}
</div>
);
}
@@ -1,4 +1,5 @@
import { Download, Edit, Sparkles, Trash2 } from 'lucide-react';
import { HugeiconsIcon } from '@hugeicons/react';
import { Download01Icon, Edit01Icon, Delete01Icon } from '@hugeicons/core-free-icons';
import { useState } from 'react';
import { Badge } from '@/components/ui/badge';
import { Button } from '@/components/ui/button';
@@ -12,6 +13,7 @@ import {
DialogHeader,
DialogTitle,
} from '@/components/ui/dialog';
import { ProfileAvatar } from '@/components/VoiceProfiles/ProfileAvatar';
import type { VoiceProfileResponse } from '@/lib/api/types';
import { useDeleteProfile, useExportProfile } from '@/lib/hooks/useProfiles';
import { cn } from '@/lib/utils/cn';
@@ -23,7 +25,6 @@ interface ProfileCardProps {
export function ProfileCard({ profile }: ProfileCardProps) {
const [deleteDialogOpen, setDeleteDialogOpen] = useState(false);
const deleteProfile = useDeleteProfile();
const exportProfile = useExportProfile();
const setEditingProfileId = useUIStore((state) => state.setEditingProfileId);
@@ -57,35 +58,24 @@ export function ProfileCard({ profile }: ProfileCardProps) {
exportProfile.mutate(profile.id);
};
const handleKeyDown = (e: React.KeyboardEvent) => {
const target = e.target as HTMLElement;
if (target.closest('button')) return;
if (e.key === 'Enter' || e.key === ' ') {
e.preventDefault();
handleSelect();
}
};
const selectLabel = isSelected
? `${profile.name}, ${profile.language}. Selected as voice for generation.`
: `${profile.name}, ${profile.language}. Select as voice for generation.`;
return (
<>
<Card
className={cn(
'cursor-pointer hover:shadow-md transition-all flex flex-col h-[162px]',
isSelected && 'ring-2 ring-accent shadow-md',
'cursor-pointer hover:shadow-md transition-all flex flex-col',
isSelected && 'ring-2 ring-primary shadow-md',
)}
onClick={handleSelect}
tabIndex={0}
role="button"
aria-label={selectLabel}
aria-pressed={isSelected}
onKeyDown={handleKeyDown}
>
<CardHeader className="p-3 pb-2">
<CardTitle className="text-base font-medium">
<CardTitle className="flex items-center gap-1.5 text-base font-medium">
<ProfileAvatar
profileId={profile.id}
avatarPath={profile.avatar_path}
size="sm"
grayscale={!isSelected}
alt={`${profile.name} avatar`}
/>
<span className="break-words">{profile.name}</span>
</CardTitle>
</CardHeader>
@@ -93,23 +83,20 @@ export function ProfileCard({ profile }: ProfileCardProps) {
<p className="text-xs text-muted-foreground mb-1.5 line-clamp-2 leading-relaxed">
{profile.description || 'No description'}
</p>
<div className="mb-2 flex items-center gap-1.5">
<div className="mb-2">
<Badge variant="outline" className="text-xs h-5 px-1.5 text-muted-foreground">
{profile.language}
</Badge>
{profile.effects_chain && profile.effects_chain.length > 0 && (
<Sparkles className="h-3.5 w-3.5 text-accent fill-accent" />
)}
</div>
<div className="flex gap-0.5 justify-end items-end mt-auto">
<CircleButton
icon={Download}
icon={(props) => <HugeiconsIcon icon={Download01Icon} size={14} {...props} />}
onClick={handleExport}
disabled={exportProfile.isPending}
aria-label="Export profile"
/>
<CircleButton
icon={Edit}
icon={(props) => <HugeiconsIcon icon={Edit01Icon} size={14} {...props} />}
onClick={(e) => {
e.stopPropagation();
handleEdit();
@@ -117,7 +104,7 @@ export function ProfileCard({ profile }: ProfileCardProps) {
aria-label="Edit profile"
/>
<CircleButton
icon={Trash2}
icon={(props) => <HugeiconsIcon icon={Delete01Icon} size={14} {...props} />}
onClick={handleDeleteClick}
disabled={deleteProfile.isPending}
aria-label="Delete profile"
@@ -1,9 +1,9 @@
import { zodResolver } from '@hookform/resolvers/zod';
import { Edit2, Mic, Monitor, Upload, X } from 'lucide-react';
import { HugeiconsIcon } from '@hugeicons/react';
import { Edit02Icon, Mic01Icon, DeskIcon, Upload01Icon, Cancel01Icon } from '@hugeicons/core-free-icons';
import { useEffect, useRef, useState } from 'react';
import { useForm } from 'react-hook-form';
import * as z from 'zod';
import { EffectsChainEditor } from '@/components/Effects/EffectsChainEditor';
import { Button } from '@/components/ui/button';
import {
Dialog,
@@ -31,8 +31,6 @@ import {
import { Tabs, TabsContent, TabsList, TabsTrigger } from '@/components/ui/tabs';
import { Textarea } from '@/components/ui/textarea';
import { useToast } from '@/components/ui/use-toast';
import { apiClient } from '@/lib/api/client';
import type { EffectConfig } from '@/lib/api/types';
import { LANGUAGE_CODES, LANGUAGE_OPTIONS, type LanguageCode } from '@/lib/constants/languages';
import { useAudioPlayer } from '@/lib/hooks/useAudioPlayer';
import { useAudioRecording } from '@/lib/hooks/useAudioRecording';
@@ -46,7 +44,7 @@ import {
} from '@/lib/hooks/useProfiles';
import { useSystemAudioCapture } from '@/lib/hooks/useSystemAudioCapture';
import { useTranscription } from '@/lib/hooks/useTranscription';
import { convertToWav, formatAudioDuration, getAudioDuration } from '@/lib/utils/audio';
import { formatAudioDuration, getAudioDuration } from '@/lib/utils/audio';
import { usePlatform } from '@/platform/PlatformContext';
import { useServerStore } from '@/stores/serverStore';
import { type ProfileFormDraft, useUIStore } from '@/stores/uiStore';
@@ -128,8 +126,6 @@ export function ProfileForm() {
const { isPlaying, playPause, cleanup: cleanupAudio } = useAudioPlayer();
const isCreating = !editingProfileId;
const serverUrl = useServerStore((state) => state.serverUrl);
const [profileEffectsChain, setProfileEffectsChain] = useState<EffectConfig[]>([]);
const [effectsDirty, setEffectsDirty] = useState(false);
const form = useForm<ProfileFormValues>({
resolver: zodResolver(profileSchema),
@@ -285,8 +281,6 @@ export function ProfileForm() {
referenceText: undefined,
avatarFile: undefined,
});
setProfileEffectsChain(editingProfile.effects_chain ?? []);
setEffectsDirty(false);
} else if (profileFormDraft && open) {
// Restore from draft when opening in create mode
form.reset({
@@ -442,24 +436,6 @@ export function ProfileForm() {
}
}
// Save effects chain if changed
if (effectsDirty) {
try {
await apiClient.updateProfileEffects(
editingProfileId,
profileEffectsChain.length > 0 ? profileEffectsChain : null,
);
} catch (fxError) {
toast({
title: 'Effects update failed',
description:
fxError instanceof Error ? fxError.message : 'Failed to save effects chain',
variant: 'destructive',
});
return;
}
}
toast({
title: 'Voice updated',
description: `"${data.name}" has been updated successfully.`,
@@ -530,23 +506,10 @@ export function ProfileForm() {
language: data.language,
});
// Convert non-WAV uploads to WAV so the backend can always use soundfile.
// Recorded audio is already WAV (from useAudioRecording's convertToWav call).
let fileToUpload: File = sampleFile;
if (!sampleFile.type.includes('wav') && !sampleFile.name.toLowerCase().endsWith('.wav')) {
try {
const wavBlob = await convertToWav(sampleFile);
const wavName = sampleFile.name.replace(/\.[^.]+$/, '.wav');
fileToUpload = new File([wavBlob], wavName, { type: 'audio/wav' });
} catch {
// If browser can't decode the format, send the original and let the backend try.
}
}
try {
await addSample.mutateAsync({
profileId: profile.id,
file: fileToUpload,
file: sampleFile,
referenceText: referenceText,
});
@@ -673,7 +636,7 @@ export function ProfileForm() {
setSampleMode('record');
}}
>
<X className="h-3 w-3 mr-1" />
<HugeiconsIcon icon={Cancel01Icon} size={12} className="h-3 w-3 mr-1" />
Discard
</Button>
</div>
@@ -706,16 +669,16 @@ export function ProfileForm() {
className={`grid w-full ${platform.metadata.isTauri && isSystemAudioSupported ? 'grid-cols-3' : 'grid-cols-2'}`}
>
<TabsTrigger value="upload" className="flex items-center gap-2">
<Upload className="h-4 w-4 shrink-0" />
<HugeiconsIcon icon={Upload01Icon} size={16} className="h-4 w-4 shrink-0" />
Upload
</TabsTrigger>
<TabsTrigger value="record" className="flex items-center gap-2">
<Mic className="h-4 w-4 shrink-0" />
<HugeiconsIcon icon={Mic01Icon} size={16} className="h-4 w-4 shrink-0" />
Record
</TabsTrigger>
{platform.metadata.isTauri && isSystemAudioSupported && (
<TabsTrigger value="system" className="flex items-center gap-2">
<Monitor className="h-4 w-4 shrink-0" />
<HugeiconsIcon icon={DeskIcon} size={16} className="h-4 w-4 shrink-0" />
System Audio
</TabsTrigger>
)}
@@ -836,7 +799,7 @@ export function ProfileForm() {
className="h-full w-full object-cover"
/>
) : (
<Mic className="h-10 w-10 text-muted-foreground" />
<HugeiconsIcon icon={Mic01Icon} size={40} className="h-10 w-10 text-muted-foreground" />
)}
</div>
<button
@@ -844,7 +807,7 @@ export function ProfileForm() {
onClick={() => avatarInputRef.current?.click()}
className="absolute inset-0 rounded-full bg-accent/60 opacity-0 group-hover:opacity-100 transition-opacity flex items-center justify-center cursor-pointer"
>
<Edit2 className="h-6 w-6 text-accent-foreground" />
<HugeiconsIcon icon={Edit02Icon} size={24} className="h-6 w-6 text-accent-foreground" />
</button>
{(avatarPreview || editingProfile?.avatar_path) && (
<button
@@ -853,7 +816,7 @@ export function ProfileForm() {
disabled={deleteAvatar.isPending}
className="absolute bottom-0 right-0 h-6 w-6 rounded-full bg-background/60 backdrop-blur-sm text-muted-foreground flex items-center justify-center hover:bg-background/80 hover:text-foreground transition-colors shadow-sm border border-border/50"
>
<X className="h-3.5 w-3.5" />
<HugeiconsIcon icon={Cancel01Icon} size={14} className="h-3.5 w-3.5" />
</button>
)}
</div>
@@ -923,23 +886,6 @@ export function ProfileForm() {
</FormItem>
)}
/>
{editingProfileId && (
<div className="space-y-2">
<FormLabel>Default Effects</FormLabel>
<p className="text-xs text-muted-foreground">
Effects applied automatically to all new generations with this voice.
</p>
<EffectsChainEditor
value={profileEffectsChain}
onChange={(chain) => {
setProfileEffectsChain(chain);
setEffectsDirty(true);
}}
compact
/>
</div>
)}
</div>
</div>
@@ -1,4 +1,5 @@
import { Mic, Sparkles } from 'lucide-react';
import { HugeiconsIcon } from '@hugeicons/react';
import { Mic01Icon, SparklesIcon } from '@hugeicons/core-free-icons';
import { Button } from '@/components/ui/button';
import { Card, CardContent } from '@/components/ui/card';
import { useProfiles } from '@/lib/hooks/useProfiles';
@@ -30,22 +31,20 @@ export function ProfileList() {
{allProfiles.length === 0 ? (
<Card>
<CardContent className="flex flex-col items-center justify-center py-12">
<Mic className="h-12 w-12 text-muted-foreground mb-4" />
<HugeiconsIcon icon={Mic01Icon} size={48} className="h-12 w-12 text-muted-foreground mb-4" />
<p className="text-muted-foreground mb-4">
No voice profiles yet. Create your first profile to get started.
</p>
<Button onClick={() => setDialogOpen(true)}>
<Sparkles className="mr-2 h-4 w-4" />
<HugeiconsIcon icon={SparklesIcon} size={16} className="mr-2 h-4 w-4" />
Create Voice
</Button>
</CardContent>
</Card>
) : (
<div className="flex gap-4 overflow-x-auto p-1 pb-1 lg:grid lg:grid-cols-3 lg:auto-rows-auto lg:overflow-x-visible lg:pb-[150px]">
<div className="grid gap-4 grid-cols-3 auto-rows-auto p-1 pb-[150px]">
{allProfiles.map((profile) => (
<div key={profile.id} className="shrink-0 w-[200px] lg:w-auto lg:shrink">
<ProfileCard profile={profile} />
</div>
<ProfileCard key={profile.id} profile={profile} />
))}
</div>
)}
+11 -14
View File
@@ -1,4 +1,5 @@
import { Check, Edit, Pause, Play, Plus, Trash2, Volume2, X } from 'lucide-react';
import { HugeiconsIcon } from '@hugeicons/react';
import { CheckmarkCircle01Icon, Edit01Icon, PauseIcon, PlayIcon, Add01Icon, Delete01Icon, VolumeHighIcon, Cancel01Icon } from '@hugeicons/core-free-icons';
import { useEffect, useRef, useState } from 'react';
import { Button } from '@/components/ui/button';
import { CircleButton } from '@/components/ui/circle-button';
@@ -102,9 +103,8 @@ function MiniSamplePlayer({ audioUrl }: MiniSamplePlayerProps) {
className="h-7 w-7 shrink-0"
onClick={handlePlayPause}
disabled={isLoading}
aria-label={isPlaying ? 'Pause sample' : 'Play sample'}
>
{isPlaying ? <Pause className="h-3.5 w-3.5" /> : <Play className="h-3.5 w-3.5 ml-0.5" />}
{isPlaying ? <HugeiconsIcon icon={PauseIcon} size={14} className="h-3.5 w-3.5" /> : <HugeiconsIcon icon={PlayIcon} size={14} className="h-3.5 w-3.5 ml-0.5" />}
</Button>
<div className="flex-1 min-w-0 flex items-center gap-2">
@@ -114,8 +114,6 @@ function MiniSamplePlayer({ audioUrl }: MiniSamplePlayerProps) {
max={100}
step={0.1}
className="flex-1"
aria-label="Sample playback position"
aria-valuetext={`${formatAudioDuration(currentTime)} of ${formatAudioDuration(duration)}`}
/>
<div className="flex items-center gap-1 text-xs text-muted-foreground shrink-0 min-w-[70px]">
<span className="font-mono">{formatAudioDuration(currentTime)}</span>
@@ -131,9 +129,8 @@ function MiniSamplePlayer({ audioUrl }: MiniSamplePlayerProps) {
className="h-7 w-7 shrink-0"
onClick={handleStop}
title="Stop"
aria-label="Stop playback"
>
<X className="h-3.5 w-3.5" />
<HugeiconsIcon icon={Cancel01Icon} size={14} className="h-3.5 w-3.5" />
</Button>
</div>
</div>
@@ -213,7 +210,7 @@ export function SampleList({ profileId }: SampleListProps) {
<div className="space-y-4 pt-4">
{samples && samples.length === 0 ? (
<div className="flex flex-col items-center justify-center py-8 text-center border border-dashed rounded-lg">
<Volume2 className="h-8 w-8 text-muted-foreground/50 mb-2" />
<HugeiconsIcon icon={VolumeHighIcon} size={32} className="h-8 w-8 text-muted-foreground/50 mb-2" />
<p className="text-sm text-muted-foreground">No samples yet</p>
<p className="text-xs text-muted-foreground/70 mt-1">
Add your first audio sample to get started
@@ -236,7 +233,7 @@ export function SampleList({ profileId }: SampleListProps) {
/* Edit Mode */
<div className="p-4 space-y-3">
<div className="flex items-center gap-2 text-xs text-muted-foreground mb-2">
<Edit className="h-3 w-3" />
<HugeiconsIcon icon={Edit01Icon} size={12} className="h-3 w-3" />
<span>Editing transcription</span>
</div>
<Textarea
@@ -254,7 +251,7 @@ export function SampleList({ profileId }: SampleListProps) {
onClick={handleCancelEdit}
disabled={updateSample.isPending}
>
<X className="h-4 w-4 mr-1" />
<HugeiconsIcon icon={Cancel01Icon} size={16} className="h-4 w-4 mr-1" />
Cancel
</Button>
<Button
@@ -263,7 +260,7 @@ export function SampleList({ profileId }: SampleListProps) {
onClick={() => handleSaveEdit(sample.id)}
disabled={updateSample.isPending}
>
<Check className="h-4 w-4 mr-1" />
<HugeiconsIcon icon={CheckmarkCircle01Icon} size={16} className="h-4 w-4 mr-1" />
{updateSample.isPending ? 'Saving...' : 'Save'}
</Button>
</div>
@@ -282,12 +279,12 @@ export function SampleList({ profileId }: SampleListProps) {
{/* Action Buttons */}
<div className="shrink-0 flex items-center gap-0.5 opacity-0 group-hover:opacity-100 transition-opacity">
<CircleButton
icon={Edit}
icon={(props) => <HugeiconsIcon icon={Edit01Icon} size={14} {...props} />}
title="Edit transcription"
onClick={() => handleStartEdit(sample.id, sample.reference_text)}
/>
<CircleButton
icon={Trash2}
icon={(props) => <HugeiconsIcon icon={Delete01Icon} size={14} {...props} />}
title="Delete sample"
onClick={() => handleDeleteClick(sample.id)}
disabled={deleteSample.isPending}
@@ -316,7 +313,7 @@ export function SampleList({ profileId }: SampleListProps) {
className="w-full"
onClick={() => setUploadOpen(true)}
>
<Plus className="mr-2 h-4 w-4" />
<HugeiconsIcon icon={Add01Icon} size={16} className="mr-2 h-4 w-4" />
Add Sample
</Button>
@@ -1,5 +1,6 @@
import { zodResolver } from '@hookform/resolvers/zod';
import { Mic, Monitor, Upload } from 'lucide-react';
import { HugeiconsIcon } from '@hugeicons/react';
import { Mic01Icon, DeskIcon, Upload01Icon } from '@hugeicons/core-free-icons';
import { useEffect, useState } from 'react';
import { useForm } from 'react-hook-form';
import * as z from 'zod';
@@ -236,16 +237,16 @@ export function SampleUpload({ profileId, open, onOpenChange }: SampleUploadProp
className={`grid w-full ${platform.metadata.isTauri && isSystemAudioSupported ? 'grid-cols-3' : 'grid-cols-2'}`}
>
<TabsTrigger value="upload" className="flex items-center gap-2">
<Upload className="h-4 w-4 shrink-0" />
<HugeiconsIcon icon={Upload01Icon} size={16} className="h-4 w-4 shrink-0" />
Upload
</TabsTrigger>
<TabsTrigger value="record" className="flex items-center gap-2">
<Mic className="h-4 w-4 shrink-0" />
<HugeiconsIcon icon={Mic01Icon} size={16} className="h-4 w-4 shrink-0" />
Record
</TabsTrigger>
{platform.metadata.isTauri && isSystemAudioSupported && (
<TabsTrigger value="system" className="flex items-center gap-2">
<Monitor className="h-4 w-4 shrink-0" />
<HugeiconsIcon icon={DeskIcon} size={16} className="h-4 w-4 shrink-0" />
System Audio
</TabsTrigger>
)}
@@ -1,340 +0,0 @@
import { zodResolver } from '@hookform/resolvers/zod';
import { Edit2, Mic, X } from 'lucide-react';
import { useEffect, useRef, useState } from 'react';
import { useForm } from 'react-hook-form';
import * as z from 'zod';
import { EffectsChainEditor } from '@/components/Effects/EffectsChainEditor';
import { Button } from '@/components/ui/button';
import {
Form,
FormControl,
FormField,
FormItem,
FormLabel,
FormMessage,
} from '@/components/ui/form';
import { Input } from '@/components/ui/input';
import {
Select,
SelectContent,
SelectItem,
SelectTrigger,
SelectValue,
} from '@/components/ui/select';
import { Textarea } from '@/components/ui/textarea';
import { useToast } from '@/components/ui/use-toast';
import { SampleList } from '@/components/VoiceProfiles/SampleList';
import { apiClient } from '@/lib/api/client';
import type { EffectConfig } from '@/lib/api/types';
import { LANGUAGE_CODES, LANGUAGE_OPTIONS, type LanguageCode } from '@/lib/constants/languages';
import { BOTTOM_SAFE_AREA_PADDING } from '@/lib/constants/ui';
import {
useDeleteAvatar,
useProfile,
useUpdateProfile,
useUploadAvatar,
} from '@/lib/hooks/useProfiles';
import { cn } from '@/lib/utils/cn';
import { usePlayerStore } from '@/stores/playerStore';
import { useServerStore } from '@/stores/serverStore';
const profileSchema = z.object({
name: z.string().min(1, 'Name is required').max(100),
description: z.string().max(500).optional(),
language: z.enum(LANGUAGE_CODES as [LanguageCode, ...LanguageCode[]]),
});
type ProfileFormValues = z.infer<typeof profileSchema>;
interface VoiceInspectorProps {
profileId: string;
}
export function VoiceInspector({ profileId }: VoiceInspectorProps) {
const { data: profile } = useProfile(profileId);
const audioUrl = usePlayerStore((state) => state.audioUrl);
const isPlayerVisible = !!audioUrl;
const updateProfile = useUpdateProfile();
const uploadAvatar = useUploadAvatar();
const deleteAvatar = useDeleteAvatar();
const serverUrl = useServerStore((state) => state.serverUrl);
const { toast } = useToast();
const [avatarPreview, setAvatarPreview] = useState<string | null>(null);
const [avatarError, setAvatarError] = useState(false);
const avatarInputRef = useRef<HTMLInputElement>(null);
const [effectsChain, setEffectsChain] = useState<EffectConfig[]>([]);
const [effectsDirty, setEffectsDirty] = useState(false);
const form = useForm<ProfileFormValues>({
resolver: zodResolver(profileSchema),
defaultValues: {
name: '',
description: '',
language: 'en',
},
});
// Populate form when profile loads
useEffect(() => {
if (profile) {
form.reset({
name: profile.name,
description: profile.description || '',
language: profile.language as LanguageCode,
});
setEffectsChain(profile.effects_chain ?? []);
setEffectsDirty(false);
}
}, [profile, form]);
// Avatar preview
useEffect(() => {
if (profile?.avatar_path) {
setAvatarPreview(`${serverUrl}/profiles/${profile.id}/avatar`);
} else {
setAvatarPreview(null);
}
setAvatarError(false);
}, [profile, serverUrl]);
function handleAvatarFileChange(e: React.ChangeEvent<HTMLInputElement>) {
const file = e.target.files?.[0];
if (!file) return;
if (!file.type.startsWith('image/')) {
toast({
title: 'Invalid file type',
description: 'Please select PNG, JPG, or WebP',
variant: 'destructive',
});
return;
}
if (file.size > 5 * 1024 * 1024) {
toast({
title: 'File too large',
description: 'Image must be less than 5MB',
variant: 'destructive',
});
return;
}
// Upload immediately
uploadAvatar.mutate(
{ profileId, file },
{
onSuccess: () => {
setAvatarPreview(URL.createObjectURL(file));
toast({ title: 'Avatar updated' });
},
onError: (err) => {
toast({
title: 'Avatar upload failed',
description: err instanceof Error ? err.message : 'Unknown error',
variant: 'destructive',
});
},
},
);
}
async function handleRemoveAvatar() {
if (profile?.avatar_path) {
try {
await deleteAvatar.mutateAsync(profileId);
toast({ title: 'Avatar removed' });
} catch (err) {
toast({
title: 'Failed to remove avatar',
description: err instanceof Error ? err.message : 'Unknown error',
variant: 'destructive',
});
}
}
setAvatarPreview(null);
if (avatarInputRef.current) avatarInputRef.current.value = '';
}
async function onSubmit(data: ProfileFormValues) {
try {
await updateProfile.mutateAsync({
profileId,
data: {
name: data.name,
description: data.description,
language: data.language,
},
});
if (effectsDirty) {
try {
await apiClient.updateProfileEffects(
profileId,
effectsChain.length > 0 ? effectsChain : null,
);
setEffectsDirty(false);
} catch (fxError) {
toast({
title: 'Effects update failed',
description: fxError instanceof Error ? fxError.message : 'Failed to save effects',
variant: 'destructive',
});
return;
}
}
toast({ title: 'Voice updated', description: `"${data.name}" saved.` });
} catch (error) {
toast({
title: 'Error',
description: error instanceof Error ? error.message : 'Failed to save profile',
variant: 'destructive',
});
}
}
if (!profile) {
return (
<div className="flex items-center justify-center h-full text-muted-foreground text-sm">
Loading...
</div>
);
}
const isDirty = form.formState.isDirty || effectsDirty;
return (
<div className="h-full flex flex-col overflow-hidden">
<div className={cn('flex-1 overflow-y-auto', isPlayerVisible && BOTTOM_SAFE_AREA_PADDING)}>
<Form {...form}>
<form onSubmit={form.handleSubmit(onSubmit)} className="space-y-0">
{/* Avatar */}
<div className="flex justify-center pt-5 pb-3">
<div className="relative group">
<div className="h-20 w-20 rounded-full bg-muted flex items-center justify-center shrink-0 overflow-hidden border-2 border-border">
{avatarPreview && !avatarError ? (
<img
src={avatarPreview}
alt={profile.name}
className="h-full w-full object-cover"
onError={() => setAvatarError(true)}
/>
) : (
<Mic className="h-8 w-8 text-muted-foreground" />
)}
</div>
<button
type="button"
onClick={() => avatarInputRef.current?.click()}
className="absolute inset-0 rounded-full bg-accent/60 opacity-0 group-hover:opacity-100 transition-opacity flex items-center justify-center cursor-pointer"
>
<Edit2 className="h-5 w-5 text-accent-foreground" />
</button>
{avatarPreview && (
<button
type="button"
onClick={handleRemoveAvatar}
disabled={deleteAvatar.isPending}
className="absolute bottom-0 right-0 h-5 w-5 rounded-full bg-background/60 backdrop-blur-sm text-muted-foreground flex items-center justify-center hover:bg-background/80 hover:text-foreground transition-colors shadow-sm border border-border/50"
>
<X className="h-3 w-3" />
</button>
)}
</div>
<input
ref={avatarInputRef}
type="file"
accept="image/png,image/jpeg,image/webp"
onChange={handleAvatarFileChange}
className="hidden"
/>
</div>
{/* Fields */}
<div className="space-y-3 px-5">
<FormField
control={form.control}
name="name"
render={({ field }) => (
<FormItem>
<FormLabel>Name</FormLabel>
<FormControl>
<Input placeholder="My Voice" {...field} />
</FormControl>
<FormMessage />
</FormItem>
)}
/>
<FormField
control={form.control}
name="description"
render={({ field }) => (
<FormItem>
<FormLabel>Description</FormLabel>
<FormControl>
<Textarea placeholder="Describe this voice..." rows={2} {...field} />
</FormControl>
<FormMessage />
</FormItem>
)}
/>
<FormField
control={form.control}
name="language"
render={({ field }) => (
<FormItem>
<FormLabel>Language</FormLabel>
<Select onValueChange={field.onChange} value={field.value}>
<FormControl>
<SelectTrigger>
<SelectValue />
</SelectTrigger>
</FormControl>
<SelectContent>
{LANGUAGE_OPTIONS.map((lang) => (
<SelectItem key={lang.value} value={lang.value}>
{lang.label}
</SelectItem>
))}
</SelectContent>
</Select>
<FormMessage />
</FormItem>
)}
/>
{/* Effects */}
<div className="space-y-2">
<FormLabel>Default Effects</FormLabel>
<p className="text-xs text-muted-foreground">
Applied automatically to new generations with this voice.
</p>
<EffectsChainEditor
value={effectsChain}
onChange={(chain) => {
setEffectsChain(chain);
setEffectsDirty(true);
}}
compact
/>
</div>
{/* Save */}
{isDirty && (
<Button type="submit" className="w-full" disabled={updateProfile.isPending}>
{updateProfile.isPending ? 'Saving...' : 'Save Changes'}
</Button>
)}
</div>
{/* Samples */}
<div className="px-5 pb-5">
<SampleList profileId={profileId} />
</div>
</form>
</Form>
</div>
</div>
);
}
+122 -145
View File
@@ -1,9 +1,14 @@
import { useQuery, useQueryClient } from '@tanstack/react-query';
import { Mic, Plus, Search, Sparkles } from 'lucide-react';
import { useEffect, useMemo, useRef, useState } from 'react';
import { HugeiconsIcon } from '@hugeicons/react';
import { Edit01Icon, MoreHorizontalIcon, Add01Icon, Delete01Icon } from '@hugeicons/core-free-icons';
import { useMemo, useRef } from 'react';
import { Button } from '@/components/ui/button';
import { Input } from '@/components/ui/input';
import {
DropdownMenu,
DropdownMenuContent,
DropdownMenuItem,
DropdownMenuTrigger,
} from '@/components/ui/dropdown-menu';
import { MultiSelect } from '@/components/ui/multi-select';
import {
Table,
@@ -14,49 +19,37 @@ import {
TableRow,
} from '@/components/ui/table';
import { ProfileForm } from '@/components/VoiceProfiles/ProfileForm';
import { ProfileAvatar } from '@/components/VoiceProfiles/ProfileAvatar';
import { apiClient } from '@/lib/api/client';
import type { VoiceProfileResponse } from '@/lib/api/types';
import { BOTTOM_SAFE_AREA_PADDING } from '@/lib/constants/ui';
import { useProfiles } from '@/lib/hooks/useProfiles';
import { useHistory } from '@/lib/hooks/useHistory';
import { useDeleteProfile, useProfileSamples, useProfiles } from '@/lib/hooks/useProfiles';
import { cn } from '@/lib/utils/cn';
import { usePlayerStore } from '@/stores/playerStore';
import { useServerStore } from '@/stores/serverStore';
import { useUIStore } from '@/stores/uiStore';
import { VoiceInspector } from './VoiceInspector';
export function VoicesTab() {
const { data: profiles, isLoading } = useProfiles();
const { data: historyData } = useHistory({ limit: 1000 });
const queryClient = useQueryClient();
const setDialogOpen = useUIStore((state) => state.setProfileDialogOpen);
const selectedVoiceId = useUIStore((state) => state.selectedVoiceId);
const setSelectedVoiceId = useUIStore((state) => state.setSelectedVoiceId);
const setEditingProfileId = useUIStore((state) => state.setEditingProfileId);
const deleteProfile = useDeleteProfile();
const scrollRef = useRef<HTMLDivElement>(null);
const audioUrl = usePlayerStore((state) => state.audioUrl);
const isPlayerVisible = !!audioUrl;
const [search, setSearch] = useState('');
const filteredProfiles = useMemo(() => {
if (!profiles) return [];
if (!search.trim()) return profiles;
const q = search.toLowerCase();
return profiles.filter(
(p) =>
p.name.toLowerCase().includes(q) ||
p.description?.toLowerCase().includes(q) ||
p.language.toLowerCase().includes(q),
);
}, [profiles, search]);
// Auto-select first profile if none selected
useEffect(() => {
if (!selectedVoiceId && profiles && profiles.length > 0) {
setSelectedVoiceId(profiles[0].id);
// Get generation counts per profile
const generationCounts = useMemo(() => {
const counts: Record<string, number> = {};
if (historyData?.items) {
historyData.items.forEach((item) => {
counts[item.profile_id] = (counts[item.profile_id] || 0) + 1;
});
}
// Clear selection if selected profile was deleted
if (selectedVoiceId && profiles && !profiles.find((p) => p.id === selectedVoiceId)) {
setSelectedVoiceId(profiles.length > 0 ? profiles[0].id : null);
}
}, [profiles, selectedVoiceId, setSelectedVoiceId]);
return counts;
}, [historyData]);
// Get channel assignments for each profile
const { data: channelAssignments } = useQuery({
@@ -83,6 +76,17 @@ export function VoicesTab() {
queryFn: () => apiClient.listChannels(),
});
const handleEdit = (profileId: string) => {
setEditingProfileId(profileId);
setDialogOpen(true);
};
const handleDelete = (profileId: string) => {
if (confirm('Are you sure you want to delete this profile?')) {
deleteProfile.mutate(profileId);
}
};
const handleChannelChange = async (profileId: string, channelIds: string[]) => {
try {
await apiClient.setProfileChannels(profileId, channelIds);
@@ -101,76 +105,56 @@ export function VoicesTab() {
}
return (
<div className="h-full flex gap-0 overflow-hidden -mx-8">
{/* Left: Table */}
<div className="flex-1 min-w-0 flex flex-col relative overflow-hidden">
{/* Scroll Mask */}
<div className="absolute top-0 left-0 right-0 h-16 bg-gradient-to-b from-background to-transparent z-10 pointer-events-none" />
<div className="h-full flex flex-col relative overflow-hidden">
{/* Scroll Mask - Always visible, behind content */}
<div className="absolute top-0 left-0 right-0 h-16 bg-gradient-to-b from-background to-transparent z-10 pointer-events-none" />
{/* Fixed Header */}
<div className="absolute top-0 left-0 right-0 z-20 pl-8 pr-8">
<div className="flex items-center gap-3 mb-6">
<h1 className="text-2xl font-bold">Voices</h1>
<div className="flex-1" />
<div className="relative w-[240px]">
<Search className="absolute left-2.5 top-1/2 -translate-y-1/2 h-3.5 w-3.5 text-muted-foreground" />
<Input
placeholder="Search voices..."
value={search}
onChange={(e) => setSearch(e.target.value)}
className="h-10 pl-8 text-sm rounded-full focus-visible:ring-0 focus-visible:ring-offset-0"
/>
</div>
<Button onClick={() => setDialogOpen(true)}>
<Plus className="h-4 w-4 mr-2" />
New Voice
</Button>
</div>
</div>
{/* Scrollable Content */}
<div
ref={scrollRef}
className={cn(
'flex-1 overflow-y-auto overflow-x-hidden pt-16 relative z-0',
isPlayerVisible && BOTTOM_SAFE_AREA_PADDING,
)}
>
<Table className="table-fixed [&_td:first-child]:pl-8 [&_th:first-child]:pl-8">
<TableHeader>
<TableRow>
<TableHead className="w-[30%]">Name</TableHead>
<TableHead className="w-[10%]">Language</TableHead>
<TableHead className="w-[10%]">Generations</TableHead>
<TableHead className="w-[8%]">Samples</TableHead>
<TableHead className="w-[8%]">Effects</TableHead>
<TableHead className="w-[24%]">Channels</TableHead>
<TableHead className="w-6"></TableHead>
</TableRow>
</TableHeader>
<TableBody>
{filteredProfiles.map((profile) => (
<VoiceRow
key={profile.id}
profile={profile}
isSelected={selectedVoiceId === profile.id}
onSelect={() => setSelectedVoiceId(profile.id)}
channelIds={channelAssignments?.[profile.id] || []}
channels={channels || []}
onChannelChange={(channelIds) => handleChannelChange(profile.id, channelIds)}
/>
))}
</TableBody>
</Table>
{/* Fixed Header */}
<div className="absolute top-0 left-0 right-0 z-20">
<div className="flex items-center justify-between mb-6">
<h1 className="text-2xl font-bold">Voices</h1>
<Button onClick={() => setDialogOpen(true)}>
<HugeiconsIcon icon={Add01Icon} size={16} className="h-4 w-4 mr-2" />
New Voice
</Button>
</div>
</div>
{/* Right: Inspector */}
{selectedVoiceId && (
<div className="w-[340px] shrink-0 border-l border-t rounded-tl-xl bg-muted/30">
<VoiceInspector key={selectedVoiceId} profileId={selectedVoiceId} />
</div>
)}
{/* Scrollable Content */}
<div
ref={scrollRef}
className={cn(
'flex-1 overflow-y-auto pt-16 relative z-0',
isPlayerVisible && BOTTOM_SAFE_AREA_PADDING,
)}
>
<Table>
<TableHeader>
<TableRow>
<TableHead>Name</TableHead>
<TableHead>Language</TableHead>
<TableHead>Generations</TableHead>
<TableHead>Samples</TableHead>
<TableHead>Channels</TableHead>
<TableHead className="w-[50px]"></TableHead>
</TableRow>
</TableHeader>
<TableBody>
{profiles?.map((profile) => (
<VoiceRow
key={profile.id}
profile={profile}
generationCount={generationCounts[profile.id] || 0}
channelIds={channelAssignments?.[profile.id] || []}
channels={channels || []}
onChannelChange={(channelIds) => handleChannelChange(profile.id, channelIds)}
onEdit={() => handleEdit(profile.id)}
onDelete={() => handleDelete(profile.id)}
/>
))}
</TableBody>
</Table>
</div>
<ProfileForm />
</div>
@@ -179,71 +163,46 @@ export function VoicesTab() {
interface VoiceRowProps {
profile: VoiceProfileResponse;
isSelected: boolean;
onSelect: () => void;
generationCount: number;
channelIds: string[];
channels: Array<{ id: string; name: string; is_default: boolean }>;
onChannelChange: (channelIds: string[]) => void;
onEdit: () => void;
onDelete: () => void;
}
function VoiceRow({
profile,
isSelected,
onSelect,
generationCount,
channelIds,
channels,
onChannelChange,
onEdit,
onDelete,
}: VoiceRowProps) {
const serverUrl = useServerStore((state) => state.serverUrl);
const [avatarError, setAvatarError] = useState(false);
const avatarUrl = profile.avatar_path ? `${serverUrl}/profiles/${profile.id}/avatar` : null;
const enabledEffects = profile.effects_chain?.filter((e) => e.enabled) ?? [];
const effectsSummary = enabledEffects.map((e) => e.type).join(' → ');
const { data: samples } = useProfileSamples(profile.id);
return (
<TableRow
className={cn('cursor-pointer', isSelected ? 'bg-muted/50' : 'hover:bg-muted/50')}
onClick={onSelect}
>
<TableRow className="cursor-pointer" onClick={onEdit}>
<TableCell>
<div className="flex w-full min-w-0 items-center gap-2">
<div className="h-8 w-8 rounded-full bg-muted flex items-center justify-center shrink-0 overflow-hidden">
{avatarUrl && !avatarError ? (
<img
src={avatarUrl}
alt={`${profile.name} avatar`}
className="h-full w-full object-cover"
onError={() => setAvatarError(true)}
/>
) : (
<Mic className="h-4 w-4 text-muted-foreground" />
)}
</div>
<div className="min-w-0">
<div className="font-medium truncate">{profile.name}</div>
<div className="flex items-center gap-2">
<ProfileAvatar
profileId={profile.id}
avatarPath={profile.avatar_path}
size="md"
alt={`${profile.name} avatar`}
/>
<div>
<div className="font-medium">{profile.name}</div>
{profile.description && (
<div className="text-sm text-muted-foreground truncate">{profile.description}</div>
<div className="text-sm text-muted-foreground">{profile.description}</div>
)}
</div>
</div>
</TableCell>
<TableCell>{profile.language}</TableCell>
<TableCell>{profile.generation_count}</TableCell>
<TableCell>{profile.sample_count}</TableCell>
<TableCell>
{enabledEffects.length > 0 ? (
<span
className="inline-flex items-center gap-1 text-xs text-accent"
title={effectsSummary}
>
<Sparkles className="h-3 w-3 fill-accent" />
{enabledEffects.length}
</span>
) : (
<span className="text-xs text-muted-foreground">—</span>
)}
</TableCell>
<TableCell onClick={(e) => e.stopPropagation()}>{profile.language}</TableCell>
<TableCell onClick={(e) => e.stopPropagation()}>{generationCount}</TableCell>
<TableCell onClick={(e) => e.stopPropagation()}>{samples?.length || 0}</TableCell>
<TableCell onClick={(e) => e.stopPropagation()}>
<MultiSelect
options={channels.map((ch) => ({
@@ -253,10 +212,28 @@ function VoiceRow({
value={channelIds}
onChange={onChannelChange}
placeholder="Select channels..."
className="w-full"
className="min-w-[200px]"
/>
</TableCell>
<TableCell />
<TableCell onClick={(e) => e.stopPropagation()}>
<DropdownMenu>
<DropdownMenuTrigger asChild>
<Button variant="ghost" size="icon">
<HugeiconsIcon icon={MoreHorizontalIcon} size={16} className="h-4 w-4" />
</Button>
</DropdownMenuTrigger>
<DropdownMenuContent>
<DropdownMenuItem onClick={onEdit}>
<HugeiconsIcon icon={Edit01Icon} size={16} className="h-4 w-4 mr-2" />
Edit
</DropdownMenuItem>
<DropdownMenuItem onClick={onDelete} className="text-destructive">
<HugeiconsIcon icon={Delete01Icon} size={16} className="h-4 w-4 mr-2" />
Delete
</DropdownMenuItem>
</DropdownMenuContent>
</DropdownMenu>
</TableCell>
</TableRow>
);
}
+3 -2
View File
@@ -1,5 +1,6 @@
import { Check } from 'lucide-react';
import * as React from 'react';
import { HugeiconsIcon } from '@hugeicons/react';
import { CheckmarkCircle01Icon } from '@hugeicons/core-free-icons';
import { cn } from '@/lib/utils/cn';
export interface CheckboxProps {
@@ -34,7 +35,7 @@ const Checkbox = React.forwardRef<HTMLButtonElement, CheckboxProps>(
)}
{...props}
>
{checked && <Check className="h-3 w-3 text-accent-foreground" />}
{checked && <HugeiconsIcon icon={CheckmarkCircle01Icon} size={12} className="h-3 w-3 text-accent-foreground" />}
</button>
);
},
+3 -2
View File
@@ -1,5 +1,6 @@
import * as DialogPrimitive from '@radix-ui/react-dialog';
import { X } from 'lucide-react';
import { HugeiconsIcon } from '@hugeicons/react';
import { Cancel01Icon } from '@hugeicons/core-free-icons';
import * as React from 'react';
import { cn } from '@/lib/utils/cn';
@@ -42,7 +43,7 @@ const DialogContent = React.forwardRef<
>
{children}
<DialogPrimitive.Close className="absolute right-4 top-4 rounded-sm opacity-70 ring-offset-background transition-opacity hover:opacity-100 focus:outline-none focus:ring-2 focus:ring-ring focus:ring-offset-2 disabled:pointer-events-none data-[state=open]:bg-accent data-[state=open]:text-muted-foreground">
<X className="h-4 w-4" />
<HugeiconsIcon icon={Cancel01Icon} size={16} className="h-4 w-4" />
<span className="sr-only">Close</span>
</DialogPrimitive.Close>
</DialogPrimitive.Content>
+5 -4
View File
@@ -1,5 +1,6 @@
import { MoreHorizontalIcon } from '@hugeicons/core-free-icons';
import { HugeiconsIcon } from '@hugeicons/react';
import * as DropdownMenuPrimitive from '@radix-ui/react-dropdown-menu';
import { MoreHorizontal } from 'lucide-react';
import * as React from 'react';
import { cn } from '@/lib/utils/cn';
@@ -26,7 +27,7 @@ const DropdownMenuSubTrigger = React.forwardRef<
{...props}
>
{children}
<MoreHorizontal className="ml-auto h-4 w-4" />
<HugeiconsIcon icon={MoreHorizontalIcon} size={16} className="ml-auto h-4 w-4" />
</DropdownMenuPrimitive.SubTrigger>
));
DropdownMenuSubTrigger.displayName = DropdownMenuPrimitive.SubTrigger.displayName;
@@ -97,7 +98,7 @@ const DropdownMenuCheckboxItem = React.forwardRef<
>
<span className="absolute left-2 flex h-3.5 w-3.5 items-center justify-center">
<DropdownMenuPrimitive.ItemIndicator>
<MoreHorizontal className="h-4 w-4" />
<HugeiconsIcon icon={MoreHorizontalIcon} size={16} className="h-4 w-4" />
</DropdownMenuPrimitive.ItemIndicator>
</span>
{children}
@@ -119,7 +120,7 @@ const DropdownMenuRadioItem = React.forwardRef<
>
<span className="absolute left-2 flex h-3.5 w-3.5 items-center justify-center">
<DropdownMenuPrimitive.ItemIndicator>
<MoreHorizontal className="h-2 w-2 fill-current" />
<HugeiconsIcon icon={MoreHorizontalIcon} size={8} className="h-2 w-2 fill-current" />
</DropdownMenuPrimitive.ItemIndicator>
</span>
{children}
+4 -3
View File
@@ -1,5 +1,6 @@
import * as React from 'react';
import { ChevronDown, Check } from 'lucide-react';
import { HugeiconsIcon } from '@hugeicons/react';
import { ArrowDown01Icon, CheckmarkCircle01Icon } from '@hugeicons/core-free-icons';
import { cn } from '@/lib/utils/cn';
import {
DropdownMenu,
@@ -36,7 +37,7 @@ const MultiSelectCheckboxItem = React.forwardRef<
>
<span className="absolute left-2 flex h-3.5 w-3.5 items-center justify-center">
<DropdownMenuPrimitive.ItemIndicator>
<Check className="h-4 w-4" />
<HugeiconsIcon icon={CheckmarkCircle01Icon} size={16} className="h-4 w-4" />
</DropdownMenuPrimitive.ItemIndicator>
</span>
{children}
@@ -78,7 +79,7 @@ export function MultiSelect({
)}
>
<span className="line-clamp-1">{displayText}</span>
<ChevronDown className="h-4 w-4 opacity-50" />
<HugeiconsIcon icon={ArrowDown01Icon} size={16} className="h-4 w-4 opacity-50" />
</button>
</DropdownMenuTrigger>
<DropdownMenuContent
+43
View File
@@ -0,0 +1,43 @@
'use client';
import { CircleIcon } from '@hugeicons/core-free-icons';
import { HugeiconsIcon } from '@hugeicons/react';
import * as RadioGroupPrimitive from '@radix-ui/react-radio-group';
import * as React from 'react';
import { cn } from '@/lib/utils/cn';
const RadioGroup = React.forwardRef<
React.ElementRef<typeof RadioGroupPrimitive.Root>,
React.ComponentPropsWithoutRef<typeof RadioGroupPrimitive.Root>
>(({ className, ...props }, ref) => {
return <RadioGroupPrimitive.Root className={cn('grid gap-2', className)} {...props} ref={ref} />;
});
RadioGroup.displayName = RadioGroupPrimitive.Root.displayName;
const RadioGroupItem = React.forwardRef<
React.ElementRef<typeof RadioGroupPrimitive.Item>,
React.ComponentPropsWithoutRef<typeof RadioGroupPrimitive.Item>
>(({ className, ...props }, ref) => {
return (
<RadioGroupPrimitive.Item
ref={ref}
className={cn(
'aspect-square h-4 w-4 rounded-full border border-accent text-accent ring-offset-background focus:outline-none focus-visible:ring-2 focus-visible:ring-ring focus-visible:ring-offset-2 disabled:cursor-not-allowed disabled:opacity-50',
className,
)}
{...props}
>
<RadioGroupPrimitive.Indicator className="flex items-center justify-center">
<HugeiconsIcon
icon={CircleIcon}
size={10}
className="h-2.5 w-2.5 fill-current text-current"
/>
</RadioGroupPrimitive.Indicator>
</RadioGroupPrimitive.Item>
);
});
RadioGroupItem.displayName = RadioGroupPrimitive.Item.displayName;
export { RadioGroup, RadioGroupItem };
+8 -7
View File
@@ -1,5 +1,6 @@
import * as SelectPrimitive from '@radix-ui/react-select';
import { Check, ChevronDown, ChevronUp } from 'lucide-react';
import { HugeiconsIcon } from '@hugeicons/react';
import { CheckmarkCircle01Icon, ArrowDown01Icon, ArrowUp01Icon } from '@hugeicons/core-free-icons';
import * as React from 'react';
import { cn } from '@/lib/utils/cn';
@@ -16,14 +17,14 @@ const SelectTrigger = React.forwardRef<
<SelectPrimitive.Trigger
ref={ref}
className={cn(
'flex h-10 w-full items-center justify-between rounded-md border border-input bg-background px-3 py-2 text-sm ring-offset-background placeholder:text-muted-foreground focus:outline-none focus:bg-muted/50 disabled:cursor-not-allowed disabled:opacity-50 [&>span]:line-clamp-1',
'flex h-10 w-full items-center justify-between rounded-md border border-input bg-background px-3 py-2 text-sm ring-offset-background placeholder:text-muted-foreground focus:outline-none focus:ring-2 focus:ring-ring focus:ring-offset-2 disabled:cursor-not-allowed disabled:opacity-50 [&>span]:line-clamp-1',
className,
)}
{...props}
>
{children}
<SelectPrimitive.Icon asChild>
<ChevronDown className="h-4 w-4 opacity-50" />
<HugeiconsIcon icon={ArrowDown01Icon} size={16} className="h-4 w-4 opacity-50" />
</SelectPrimitive.Icon>
</SelectPrimitive.Trigger>
));
@@ -38,7 +39,7 @@ const SelectScrollUpButton = React.forwardRef<
className={cn('flex cursor-default items-center justify-center py-1', className)}
{...props}
>
<ChevronUp className="h-4 w-4" />
<HugeiconsIcon icon={ArrowUp01Icon} size={16} className="h-4 w-4" />
</SelectPrimitive.ScrollUpButton>
));
SelectScrollUpButton.displayName = SelectPrimitive.ScrollUpButton.displayName;
@@ -52,7 +53,7 @@ const SelectScrollDownButton = React.forwardRef<
className={cn('flex cursor-default items-center justify-center py-1', className)}
{...props}
>
<ChevronDown className="h-4 w-4" />
<HugeiconsIcon icon={ArrowDown01Icon} size={16} className="h-4 w-4" />
</SelectPrimitive.ScrollDownButton>
));
SelectScrollDownButton.displayName = SelectPrimitive.ScrollDownButton.displayName;
@@ -108,14 +109,14 @@ const SelectItem = React.forwardRef<
<SelectPrimitive.Item
ref={ref}
className={cn(
'relative flex w-full cursor-default select-none items-center rounded-sm py-1.5 pl-8 pr-2 text-sm outline-none focus:bg-accent focus:text-accent-foreground focus:[&_*]:text-accent-foreground data-[disabled]:pointer-events-none data-[disabled]:opacity-50',
'relative flex w-full cursor-default select-none items-center rounded-sm py-1.5 pl-8 pr-2 text-sm outline-none focus:bg-accent focus:text-accent-foreground data-[disabled]:pointer-events-none data-[disabled]:opacity-50',
className,
)}
{...props}
>
<span className="absolute left-2 flex h-3.5 w-3.5 items-center justify-center">
<SelectPrimitive.ItemIndicator>
<Check className="h-4 w-4" />
<HugeiconsIcon icon={CheckmarkCircle01Icon} size={16} className="h-4 w-4" />
</SelectPrimitive.ItemIndicator>
</span>
+5 -6
View File
@@ -14,11 +14,7 @@ const TableHeader = React.forwardRef<
HTMLTableSectionElement,
React.HTMLAttributes<HTMLTableSectionElement>
>(({ className, ...props }, ref) => (
<thead
ref={ref}
className={cn('[&_tr]:border-b [&_tr]:hover:bg-transparent', className)}
{...props}
/>
<thead ref={ref} className={cn('[&_tr]:border-b', className)} {...props} />
));
TableHeader.displayName = 'TableHeader';
@@ -46,7 +42,10 @@ const TableRow = React.forwardRef<HTMLTableRowElement, React.HTMLAttributes<HTML
({ className, ...props }, ref) => (
<tr
ref={ref}
className={cn('border-b hover:bg-muted/50 data-[state=selected]:bg-muted', className)}
className={cn(
'border-b transition-colors hover:bg-muted/50 data-[state=selected]:bg-muted',
className,
)}
{...props}
/>
),
+3 -2
View File
@@ -1,6 +1,7 @@
import * as ToastPrimitives from '@radix-ui/react-toast';
import { cva, type VariantProps } from 'class-variance-authority';
import { X } from 'lucide-react';
import { HugeiconsIcon } from '@hugeicons/react';
import { Cancel01Icon } from '@hugeicons/core-free-icons';
import * as React from 'react';
import { cn } from '@/lib/utils/cn';
@@ -79,7 +80,7 @@ const ToastClose = React.forwardRef<
toast-close=""
{...props}
>
<X className="h-4 w-4" />
<HugeiconsIcon icon={Cancel01Icon} size={16} className="h-4 w-4" />
</ToastPrimitives.Close>
));
ToastClose.displayName = ToastPrimitives.Close.displayName;
-52
View File
@@ -1,52 +0,0 @@
import { useCallback, useEffect, useRef, useState } from 'react';
import { usePlatform } from '@/platform/PlatformContext';
import type { UpdateStatus } from '@/platform/types';
// Re-export UpdateStatus for backwards compatibility
export type { UpdateStatus };
export function useAutoUpdater(checkOnMount = false) {
const platform = usePlatform();
const [status, setStatus] = useState<UpdateStatus>(platform.updater.getStatus());
const hasCheckedRef = useRef(false);
// Subscribe to updater status changes
useEffect(() => {
const unsubscribe = platform.updater.subscribe((newStatus) => {
setStatus(newStatus);
});
return unsubscribe;
// Empty dependency array - platform is stable from context
// eslint-disable-next-line react-hooks/exhaustive-deps
}, [platform.updater.subscribe]);
const checkForUpdates = useCallback(async () => {
await platform.updater.checkForUpdates();
// eslint-disable-next-line react-hooks/exhaustive-deps
}, [platform.updater.checkForUpdates]);
const downloadAndInstall = useCallback(async () => {
await platform.updater.downloadAndInstall();
// eslint-disable-next-line react-hooks/exhaustive-deps
}, [platform.updater.downloadAndInstall]);
const restartAndInstall = useCallback(async () => {
await platform.updater.restartAndInstall();
// eslint-disable-next-line react-hooks/exhaustive-deps
}, [platform.updater.restartAndInstall]);
useEffect(() => {
if (checkOnMount && platform.metadata.isTauri && !hasCheckedRef.current) {
hasCheckedRef.current = true;
checkForUpdates();
}
// eslint-disable-next-line react-hooks/exhaustive-deps
}, [platform.metadata.isTauricheckOnMountcheckForUpdates]);
return {
status,
checkForUpdates,
downloadAndInstall,
restartAndInstall,
};
}
+5 -4
View File
@@ -1,4 +1,5 @@
import { Download, RefreshCw } from 'lucide-react';
import { Download01Icon, Refresh01Icon } from '@hugeicons/core-free-icons';
import { HugeiconsIcon } from '@hugeicons/react';
import { useCallback, useEffect, useRef, useState } from 'react';
import { Progress } from '@/components/ui/progress';
import { ToastAction } from '@/components/ui/toast';
@@ -73,7 +74,7 @@ export function useAutoUpdater(options: boolean | UseAutoUpdaterOptions = false)
}
// Empty dependency array - only run once on mount
// eslint-disable-next-line react-hooks/exhaustive-deps
}, [platform.metadata.isTauricheckOnMountcheckForUpdates]);
}, [platform.metadata.isTauri, checkOnMount, checkForUpdates]);
// Show toast when update is available
useEffect(() => {
@@ -132,7 +133,7 @@ export function useAutoUpdater(options: boolean | UseAutoUpdaterOptions = false)
toastUpdateRef.current({
title: (
<div className="flex items-center gap-2">
<Download className="h-4 w-4 animate-pulse" />
<HugeiconsIcon icon={Download01Icon} size={16} className="h-4 w-4 animate-pulse" />
<span>Downloading Update</span>
</div>
),
@@ -174,7 +175,7 @@ export function useAutoUpdater(options: boolean | UseAutoUpdaterOptions = false)
duration: Infinity,
action: (
<ToastAction altText="Restart now" onClick={handleRestartNow}>
<RefreshCw className="h-3 w-3 mr-1" />
<HugeiconsIcon icon={Refresh01Icon} size={12} className="h-3 w-3 mr-1" />
Restart Now
</ToastAction>
),
-16
View File
@@ -1,5 +1,4 @@
@import "tailwindcss" source(".");
@import "loaders.css/loaders.min.css";
@theme {
--radius-sm: calc(var(--radius) - 4px);
@@ -156,18 +155,3 @@
animation: fadeIn 0.5s ease-out 0.15s forwards;
opacity: 0;
}
/* react-loaders */
.line-scale-pulse-out-rapid > div,
.line-scale > div {
background-color: hsl(var(--accent)) !important;
}
.loader-hidden {
display: block;
}
.loader-hidden > div > div {
animation-play-state: paused !important;
background-color: hsl(var(--muted-foreground)) !important;
}
+69 -178
View File
@@ -2,15 +2,9 @@ import type { LanguageCode } from '@/lib/constants/languages';
import { useServerStore } from '@/stores/serverStore';
import type {
ActiveTasksResponse,
ApplyEffectsRequest,
AvailableEffectsResponse,
CudaStatus,
EffectConfig,
EffectPresetCreate,
EffectPresetResponse,
FolderPathsResponse,
GenerationRequest,
GenerationResponse,
GenerationVersionResponse,
HealthResponse,
HistoryListResponse,
HistoryQuery,
@@ -27,7 +21,6 @@ import type {
StoryItemReorder,
StoryItemSplit,
StoryItemTrim,
StoryItemVersionUpdate,
StoryResponse,
TranscriptionResponse,
VoiceProfileCreate,
@@ -65,6 +58,11 @@ class ApiClient {
return this.request<HealthResponse>('/health');
}
// System
async getSystemFolders(): Promise<FolderPathsResponse> {
return this.request<FolderPathsResponse>('/system/folders');
}
// Profiles
async createProfile(data: VoiceProfileCreate): Promise<VoiceProfileResponse> {
return this.request<VoiceProfileResponse>('/profiles', {
@@ -207,21 +205,74 @@ class ApiClient {
});
}
async retryGeneration(generationId: string): Promise<GenerationResponse> {
return this.request<GenerationResponse>(`/generate/${generationId}/retry`, {
// Providers
async listProviders(): Promise<{
providers: Array<{
type: string;
name: string;
installed: boolean;
size_mb: number | null;
}>;
installed: string[];
}> {
return this.request('/providers');
}
async getActiveProvider(): Promise<{
provider: string;
health: {
status: string;
provider: string;
version: string | null;
model: string | null;
device: string | null;
};
status: {
model_loaded: boolean;
model_size: string | null;
available_sizes: string[];
gpu_available: boolean | null;
vram_used_mb: number | null;
};
}> {
return this.request('/providers/active');
}
async startProvider(providerType: string): Promise<{
message: string;
provider: {
status: string;
provider: string;
version: string | null;
model: string | null;
device: string | null;
};
}> {
return this.request('/providers/start', {
method: 'POST',
body: JSON.stringify({ provider_type: providerType }),
});
}
async stopProvider(): Promise<{ message: string }> {
return this.request('/providers/stop', {
method: 'POST',
});
}
async regenerateGeneration(generationId: string): Promise<GenerationResponse> {
return this.request<GenerationResponse>(`/generate/${generationId}/regenerate`, {
async downloadProvider(providerType: string): Promise<{
message: string;
provider_type: string;
}> {
return this.request('/providers/download', {
method: 'POST',
body: JSON.stringify({ provider_type: providerType }),
});
}
async toggleFavorite(generationId: string): Promise<{ is_favorited: boolean }> {
return this.request<{ is_favorited: boolean }>(`/history/${generationId}/favorite`, {
method: 'POST',
async deleteProvider(providerType: string): Promise<{ message: string }> {
return this.request(`/providers/${providerType}`, {
method: 'DELETE',
});
}
@@ -277,7 +328,9 @@ class ApiClient {
return response.blob();
}
async importGeneration(file: File): Promise<{
async importGeneration(
file: File,
): Promise<{
id: string;
profile_id: string;
profile_name: string;
@@ -303,11 +356,6 @@ class ApiClient {
return response.json();
}
// Generation status SSE
getGenerationStatusUrl(generationId: string): string {
return `${this.getBaseUrl()}/generate/${generationId}/status`;
}
// Audio
getAudioUrl(audioId: string): string {
return `${this.getBaseUrl()}/audio/${audioId}`;
@@ -346,21 +394,6 @@ class ApiClient {
return this.request<ModelStatusListResponse>('/models/status');
}
async getModelsCacheDir(): Promise<{ path: string }> {
return this.request<{ path: string }>('/models/cache-dir');
}
async migrateModels(destination: string): Promise<{ source: string; destination: string }> {
return this.request('/models/migrate', {
method: 'POST',
body: JSON.stringify({ destination }),
});
}
getMigrationProgressUrl(): string {
return `${this.getBaseUrl()}/models/migrate/progress`;
}
async triggerModelDownload(modelName: string): Promise<{ message: string }> {
console.log(
'[API] triggerModelDownload called for:',
@@ -382,28 +415,11 @@ class ApiClient {
});
}
async unloadModel(modelName: string): Promise<{ message: string }> {
return this.request<{ message: string }>(`/models/${modelName}/unload`, {
method: 'POST',
});
}
async cancelDownload(modelName: string): Promise<{ message: string }> {
return this.request<{ message: string }>('/models/download/cancel', {
method: 'POST',
body: JSON.stringify({ model_name: modelName } as ModelDownloadRequest),
});
}
// Task Management
async getActiveTasks(): Promise<ActiveTasksResponse> {
return this.request<ActiveTasksResponse>('/tasks/active');
}
async clearAllTasks(): Promise<{ message: string }> {
return this.request<{ message: string }>('/tasks/clear', { method: 'POST' });
}
// Audio Channels
async listChannels(): Promise<
Array<{
@@ -477,23 +493,6 @@ class ApiClient {
});
}
// CUDA Backend Management
async getCudaStatus(): Promise<CudaStatus> {
return this.request<CudaStatus>('/backend/cuda-status');
}
async downloadCudaBackend(): Promise<{ message: string; progress_key: string }> {
return this.request<{ message: string; progress_key: string }>('/backend/download-cuda', {
method: 'POST',
});
}
async deleteCudaBackend(): Promise<{ message: string }> {
return this.request<{ message: string }>('/backend/cuda', {
method: 'DELETE',
});
}
// Stories
async listStories(): Promise<StoryResponse[]> {
return this.request<StoryResponse[]>('/stories');
@@ -589,17 +588,6 @@ class ApiClient {
});
}
async setStoryItemVersion(
storyId: string,
itemId: string,
data: StoryItemVersionUpdate,
): Promise<StoryItemDetail> {
return this.request<StoryItemDetail>(`/stories/${storyId}/items/${itemId}/version`, {
method: 'PUT',
body: JSON.stringify(data),
});
}
async exportStoryAudio(storyId: string): Promise<Blob> {
const url = `${this.getBaseUrl()}/stories/${storyId}/export-audio`;
const response = await fetch(url);
@@ -613,103 +601,6 @@ class ApiClient {
return response.blob();
}
// Effects & Versions
async getAvailableEffects(): Promise<AvailableEffectsResponse> {
return this.request<AvailableEffectsResponse>('/effects/available');
}
async listEffectPresets(): Promise<EffectPresetResponse[]> {
return this.request<EffectPresetResponse[]>('/effects/presets');
}
async createEffectPreset(data: EffectPresetCreate): Promise<EffectPresetResponse> {
return this.request<EffectPresetResponse>('/effects/presets', {
method: 'POST',
body: JSON.stringify(data),
});
}
async updateEffectPreset(
presetId: string,
data: { name?: string; description?: string; effects_chain?: EffectConfig[] },
): Promise<EffectPresetResponse> {
return this.request<EffectPresetResponse>(`/effects/presets/${presetId}`, {
method: 'PUT',
body: JSON.stringify(data),
});
}
async deleteEffectPreset(presetId: string): Promise<void> {
await this.request<void>(`/effects/presets/${presetId}`, {
method: 'DELETE',
});
}
async listGenerationVersions(generationId: string): Promise<GenerationVersionResponse[]> {
return this.request<GenerationVersionResponse[]>(`/generations/${generationId}/versions`);
}
async applyEffectsToGeneration(
generationId: string,
data: ApplyEffectsRequest,
): Promise<GenerationVersionResponse> {
return this.request<GenerationVersionResponse>(
`/generations/${generationId}/versions/apply-effects`,
{
method: 'POST',
body: JSON.stringify(data),
},
);
}
async setDefaultVersion(
generationId: string,
versionId: string,
): Promise<GenerationVersionResponse> {
return this.request<GenerationVersionResponse>(
`/generations/${generationId}/versions/${versionId}/set-default`,
{ method: 'PUT' },
);
}
async deleteGenerationVersion(generationId: string, versionId: string): Promise<void> {
await this.request<void>(`/generations/${generationId}/versions/${versionId}`, {
method: 'DELETE',
});
}
getVersionAudioUrl(versionId: string): string {
return `${this.getBaseUrl()}/audio/version/${versionId}`;
}
async updateProfileEffects(
profileId: string,
effectsChain: EffectConfig[] | null,
): Promise<VoiceProfileResponse> {
return this.request<VoiceProfileResponse>(`/profiles/${profileId}/effects`, {
method: 'PUT',
body: JSON.stringify({ effects_chain: effectsChain }),
});
}
async previewEffects(generationId: string, effectsChain: EffectConfig[]): Promise<Blob> {
const url = `${this.getBaseUrl()}/effects/preview/${generationId}`;
const response = await fetch(url, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ effects_chain: effectsChain }),
});
if (!response.ok) {
const error = await response.json().catch(() => ({
detail: response.statusText,
}));
throw new Error(error.detail || `HTTP error! status: ${response.status}`);
}
return response.blob();
}
}
export const apiClient = new ApiClient();
+9 -138
View File
@@ -13,9 +13,6 @@ export interface VoiceProfileResponse {
description?: string;
language: string;
avatar_path?: string;
effects_chain?: EffectConfig[];
generation_count: number;
sample_count: number;
created_at: string;
updated_at: string;
}
@@ -31,35 +28,13 @@ export interface ProfileSampleResponse {
reference_text: string;
}
export interface EffectConfig {
type: string;
enabled: boolean;
params: Record<string, number>;
}
export interface GenerationRequest {
profile_id: string;
text: string;
language: LanguageCode;
seed?: number;
model_size?: '1.7B' | '0.6B';
engine?: 'qwen' | 'luxtts' | 'chatterbox' | 'chatterbox_turbo';
instruct?: string;
max_chunk_chars?: number;
crossfade_ms?: number;
normalize?: boolean;
effects_chain?: EffectConfig[];
}
export interface GenerationVersionResponse {
id: string;
generation_id: string;
label: string;
audio_path: string;
effects_chain?: EffectConfig[];
source_version_id?: string;
is_default: boolean;
created_at: string;
}
export interface GenerationResponse {
@@ -67,18 +42,10 @@ export interface GenerationResponse {
profile_id: string;
text: string;
language: string;
audio_path?: string;
duration?: number;
audio_path: string;
duration: number;
seed?: number;
instruct?: string;
engine?: string;
model_size?: string;
status: 'loading_model' | 'generating' | 'completed' | 'failed';
error?: string;
is_favorited?: boolean;
created_at: string;
versions?: GenerationVersionResponse[];
active_version_id?: string;
}
export interface HistoryQuery {
@@ -90,8 +57,6 @@ export interface HistoryQuery {
export interface HistoryResponse extends GenerationResponse {
profile_name: string;
versions?: GenerationVersionResponse[];
active_version_id?: string;
}
export interface HistoryListResponse {
@@ -114,29 +79,7 @@ export interface HealthResponse {
model_downloaded?: boolean;
model_size?: string;
gpu_available: boolean;
gpu_type?: string;
vram_used_mb?: number;
backend_type?: string;
backend_variant?: string; // "cpu" or "cuda"
}
export interface CudaDownloadProgress {
model_name: string;
current: number;
total: number;
progress: number;
filename?: string;
status: 'downloading' | 'extracting' | 'complete' | 'error';
timestamp: string;
error?: string;
}
export interface CudaStatus {
available: boolean; // CUDA binary exists on disk
active: boolean; // Currently running the CUDA binary
binary_path?: string;
downloading: boolean; // Download in progress
download_progress?: CudaDownloadProgress;
}
export interface ModelProgress {
@@ -153,29 +96,12 @@ export interface ModelProgress {
export interface ModelStatus {
model_name: string;
display_name: string;
hf_repo_id?: string; // HuggingFace repository ID
downloaded: boolean;
downloading: boolean; // True if download is in progress
downloading: boolean; // True if download is in progress
size_mb?: number;
loaded: boolean;
}
export interface HuggingFaceModelInfo {
id: string;
author: string;
lastModified: string;
pipeline_tag?: string;
library_name?: string;
downloads: number;
likes: number;
tags: string[];
cardData?: {
license?: string;
language?: string[];
pipeline_tag?: string;
};
}
export interface ModelStatusListResponse {
models: ModelStatus[];
}
@@ -188,11 +114,6 @@ export interface ActiveDownloadTask {
model_name: string;
status: string;
started_at: string;
error?: string;
progress?: number; // 0-100 percentage
current?: number; // bytes downloaded
total?: number; // total bytes
filename?: string; // current file being downloaded
}
export interface ActiveGenerationTask {
@@ -207,6 +128,12 @@ export interface ActiveTasksResponse {
generations: ActiveGenerationTask[];
}
export interface FolderPathsResponse {
data_dir: string;
models_dir: string;
providers_dir: string;
}
export interface StoryCreate {
name: string;
description?: string;
@@ -225,7 +152,6 @@ export interface StoryItemDetail {
id: string;
story_id: string;
generation_id: string;
version_id?: string;
start_time_ms: number;
track: number;
trim_start_ms: number;
@@ -240,12 +166,6 @@ export interface StoryItemDetail {
seed?: number;
instruct?: string;
generation_created_at: string;
versions?: GenerationVersionResponse[];
active_version_id?: string;
}
export interface StoryItemVersionUpdate {
version_id: string | null;
}
export interface StoryDetailResponse {
@@ -289,52 +209,3 @@ export interface StoryItemTrim {
export interface StoryItemSplit {
split_time_ms: number;
}
// Effects
export interface EffectPresetResponse {
id: string;
name: string;
description?: string;
effects_chain: EffectConfig[];
is_builtin: boolean;
created_at: string;
}
export interface EffectPresetCreate {
name: string;
description?: string;
effects_chain: EffectConfig[];
}
export interface EffectPresetUpdate {
name?: string;
description?: string;
effects_chain?: EffectConfig[];
}
export interface AvailableEffectParam {
default: number;
min: number;
max: number;
step: number;
description: string;
}
export interface AvailableEffect {
type: string;
label: string;
description: string;
params: Record<string, AvailableEffectParam>;
}
export interface AvailableEffectsResponse {
effects: AvailableEffect[];
}
export interface ApplyEffectsRequest {
effects_chain: EffectConfig[];
source_version_id?: string;
label?: string;
set_as_default?: boolean;
}
+12 -72
View File
@@ -1,86 +1,26 @@
/**
* Supported languages for voice generation, per engine.
*
* Qwen3-TTS supports 10 languages.
* LuxTTS is English-only.
* Chatterbox Multilingual supports 23 languages.
* Chatterbox Turbo is English-only.
* Supported languages for Qwen3-TTS
* Based on: https://github.com/QwenLM/Qwen3-TTS
*/
/** All languages that any engine supports. */
export const ALL_LANGUAGES = {
ar: 'Arabic',
da: 'Danish',
de: 'German',
el: 'Greek',
export const SUPPORTED_LANGUAGES = {
zh: 'Chinese',
en: 'English',
es: 'Spanish',
fi: 'Finnish',
fr: 'French',
he: 'Hebrew',
hi: 'Hindi',
it: 'Italian',
ja: 'Japanese',
ko: 'Korean',
ms: 'Malay',
nl: 'Dutch',
no: 'Norwegian',
pl: 'Polish',
pt: 'Portuguese',
de: 'German',
fr: 'French',
ru: 'Russian',
sv: 'Swedish',
sw: 'Swahili',
tr: 'Turkish',
zh: 'Chinese',
pt: 'Portuguese',
es: 'Spanish',
it: 'Italian',
} as const;
export type LanguageCode = keyof typeof ALL_LANGUAGES;
export type LanguageCode = keyof typeof SUPPORTED_LANGUAGES;
/** Per-engine supported language codes. */
export const ENGINE_LANGUAGES: Record<string, readonly LanguageCode[]> = {
qwen: ['zh', 'en', 'ja', 'ko', 'de', 'fr', 'ru', 'pt', 'es', 'it'],
luxtts: ['en'],
chatterbox: [
'ar',
'da',
'de',
'el',
'en',
'es',
'fi',
'fr',
'he',
'hi',
'it',
'ja',
'ko',
'ms',
'nl',
'no',
'pl',
'pt',
'ru',
'sv',
'sw',
'tr',
'zh',
],
chatterbox_turbo: ['en'],
} as const;
export const LANGUAGE_CODES = Object.keys(SUPPORTED_LANGUAGES) as LanguageCode[];
/** Helper: get language options for a given engine. */
export function getLanguageOptionsForEngine(engine: string) {
const codes = ENGINE_LANGUAGES[engine] ?? ENGINE_LANGUAGES.qwen;
return codes.map((code) => ({
value: code,
label: ALL_LANGUAGES[code],
}));
}
// ── Backwards-compatible exports used elsewhere ──────────────────────
export const SUPPORTED_LANGUAGES = ALL_LANGUAGES;
export const LANGUAGE_CODES = Object.keys(ALL_LANGUAGES) as LanguageCode[];
export const LANGUAGE_OPTIONS = LANGUAGE_CODES.map((code) => ({
value: code,
label: ALL_LANGUAGES[code],
label: SUPPORTED_LANGUAGES[code],
}));
+2 -5
View File
@@ -2,14 +2,11 @@
* UI layout constants for safe area padding
*/
const isWindows = typeof navigator !== 'undefined' && navigator.userAgent.includes('Windows');
/**
* Top safe area padding - height of the drag region bar
* On macOS this accounts for the overlay titlebar (48px).
* On Windows the native title bar is outside the webview, so no padding is needed.
* Corresponds to Tailwind's pt-12 (3rem / 48px)
*/
export const TOP_SAFE_AREA_PADDING = isWindows ? 'pt-8' : 'pt-12';
export const TOP_SAFE_AREA_PADDING = 'pt-12';
/**
* Bottom safe area padding - height of the audio player
+15 -21
View File
@@ -20,13 +20,11 @@ export function useAudioRecording({
const streamRef = useRef<MediaStream | null>(null);
const timerRef = useRef<number | null>(null);
const startTimeRef = useRef<number | null>(null);
const cancelledRef = useRef<boolean>(false);
const startRecording = useCallback(async () => {
try {
setError(null);
chunksRef.current = [];
cancelledRef.current = false;
setDuration(0);
// Check if getUserMedia is available
@@ -89,34 +87,31 @@ export function useAudioRecording({
};
mediaRecorder.onstop = async () => {
// Snapshot the cancellation flag and recorded duration immediately —
// cancelRecording() clears chunks and sets cancelledRef synchronously
// before this async handler runs, so we must check it first.
const wasCancelled = cancelledRef.current;
const recordedDuration = startTimeRef.current
? (Date.now() - startTimeRef.current) / 1000
: undefined;
const webmBlob = new Blob(chunksRef.current, { type: 'audio/webm' });
// Stop all tracks now that we have the data
streamRef.current?.getTracks().forEach((track) => {
track.stop();
});
streamRef.current = null;
// Don't fire completion callback if the recording was cancelled
if (wasCancelled) return;
// Convert to WAV format to avoid needing ffmpeg on backend
try {
const wavBlob = await convertToWav(webmBlob);
// Pass the actual recorded duration
const recordedDuration = startTimeRef.current
? (Date.now() - startTimeRef.current) / 1000
: undefined;
onRecordingComplete?.(wavBlob, recordedDuration);
} catch (err) {
console.error('Error converting audio to WAV:', err);
// Fallback to original blob if conversion fails
const recordedDuration = startTimeRef.current
? (Date.now() - startTimeRef.current) / 1000
: undefined;
onRecordingComplete?.(webmBlob, recordedDuration);
}
// Stop all tracks
streamRef.current?.getTracks().forEach((track) => {
track.stop();
});
streamRef.current = null;
};
mediaRecorder.onerror = (event) => {
@@ -172,10 +167,9 @@ export function useAudioRecording({
const cancelRecording = useCallback(() => {
if (mediaRecorderRef.current) {
cancelledRef.current = true; // Must be set before stop() triggers onstop
chunksRef.current = [];
mediaRecorderRef.current.stop();
setIsRecording(false);
chunksRef.current = [];
setDuration(0);
}
+14 -51
View File
@@ -4,20 +4,18 @@ import { useForm } from 'react-hook-form';
import * as z from 'zod';
import { useToast } from '@/components/ui/use-toast';
import { apiClient } from '@/lib/api/client';
import type { EffectConfig } from '@/lib/api/types';
import { LANGUAGE_CODES, type LanguageCode } from '@/lib/constants/languages';
import { useGeneration } from '@/lib/hooks/useGeneration';
import { useModelDownloadToast } from '@/lib/hooks/useModelDownloadToast';
import { useGenerationStore } from '@/stores/generationStore';
import { useServerStore } from '@/stores/serverStore';
import { usePlayerStore } from '@/stores/playerStore';
const generationSchema = z.object({
text: z.string().min(1, '').max(50000),
text: z.string().min(1, 'Text is required').max(5000),
language: z.enum(LANGUAGE_CODES as [LanguageCode, ...LanguageCode[]]),
seed: z.number().int().optional(),
modelSize: z.enum(['1.7B', '0.6B']).optional(),
instruct: z.string().max(500).optional(),
engine: z.enum(['qwen', 'luxtts', 'chatterbox', 'chatterbox_turbo']).optional(),
});
export type GenerationFormValues = z.infer<typeof generationSchema>;
@@ -25,16 +23,13 @@ export type GenerationFormValues = z.infer<typeof generationSchema>;
interface UseGenerationFormOptions {
onSuccess?: (generationId: string) => void;
defaultValues?: Partial<GenerationFormValues>;
getEffectsChain?: () => EffectConfig[] | undefined;
}
export function useGenerationForm(options: UseGenerationFormOptions = {}) {
const { toast } = useToast();
const generation = useGeneration();
const addPendingGeneration = useGenerationStore((state) => state.addPendingGeneration);
const maxChunkChars = useServerStore((state) => state.maxChunkChars);
const crossfadeMs = useServerStore((state) => state.crossfadeMs);
const normalizeAudio = useServerStore((state) => state.normalizeAudio);
const setAudioWithAutoPlay = usePlayerStore((state) => state.setAudioWithAutoPlay);
const setIsGenerating = useGenerationStore((state) => state.setIsGenerating);
const [downloadingModelName, setDownloadingModelName] = useState<string | null>(null);
const [downloadingDisplayName, setDownloadingDisplayName] = useState<string | null>(null);
@@ -52,7 +47,6 @@ export function useGenerationForm(options: UseGenerationFormOptions = {}) {
seed: undefined,
modelSize: '1.7B',
instruct: '',
engine: 'qwen',
...options.defaultValues,
},
});
@@ -71,27 +65,11 @@ export function useGenerationForm(options: UseGenerationFormOptions = {}) {
}
try {
const engine = data.engine || 'qwen';
const modelName =
engine === 'luxtts'
? 'luxtts'
: engine === 'chatterbox'
? 'chatterbox-tts'
: engine === 'chatterbox_turbo'
? 'chatterbox-turbo'
: `qwen-tts-${data.modelSize}`;
const displayName =
engine === 'luxtts'
? 'LuxTTS'
: engine === 'chatterbox'
? 'Chatterbox TTS'
: engine === 'chatterbox_turbo'
? 'Chatterbox Turbo'
: data.modelSize === '1.7B'
? 'Qwen TTS 1.7B'
: 'Qwen TTS 0.6B';
setIsGenerating(true);
const modelName = `qwen-tts-${data.modelSize}`;
const displayName = data.modelSize === '1.7B' ? 'Qwen TTS 1.7B' : 'Qwen TTS 0.6B';
// Check if model needs downloading
try {
const modelStatus = await apiClient.getModelStatus();
const model = modelStatus.models.find((m) => m.model_name === modelName);
@@ -104,35 +82,19 @@ export function useGenerationForm(options: UseGenerationFormOptions = {}) {
console.error('Failed to check model status:', error);
}
const isQwen = engine === 'qwen';
const effectsChain = options.getEffectsChain?.();
// This now returns immediately with status="generating"
const result = await generation.mutateAsync({
profile_id: selectedProfileId,
text: data.text,
language: data.language,
seed: data.seed,
model_size: isQwen ? data.modelSize : undefined,
engine,
instruct: isQwen ? data.instruct || undefined : undefined,
max_chunk_chars: maxChunkChars,
crossfade_ms: crossfadeMs,
normalize: normalizeAudio,
effects_chain: effectsChain?.length ? effectsChain : undefined,
model_size: data.modelSize,
instruct: data.instruct || undefined,
});
// Track this generation for SSE status updates
addPendingGeneration(result.id);
const audioUrl = apiClient.getAudioUrl(result.id);
setAudioWithAutoPlay(audioUrl, result.id, selectedProfileId, data.text.substring(0, 50));
// Reset form immediately — user can start typing again
form.reset({
text: '',
language: data.language,
seed: undefined,
modelSize: data.modelSize,
instruct: '',
engine: data.engine,
});
form.reset();
options.onSuccess?.(result.id);
} catch (error) {
toast({
@@ -141,6 +103,7 @@ export function useGenerationForm(options: UseGenerationFormOptions = {}) {
variant: 'destructive',
});
} finally {
setIsGenerating(false);
setDownloadingModelName(null);
setDownloadingDisplayName(null);
}
-154
View File
@@ -1,154 +0,0 @@
import { useQueryClient } from '@tanstack/react-query';
import { useEffect, useRef } from 'react';
import { useToast } from '@/components/ui/use-toast';
import { apiClient } from '@/lib/api/client';
import { useGenerationStore } from '@/stores/generationStore';
import { usePlayerStore } from '@/stores/playerStore';
import { useServerStore } from '@/stores/serverStore';
interface GenerationStatusEvent {
id: string;
status: 'loading_model' | 'generating' | 'completed' | 'failed' | 'not_found';
duration?: number;
error?: string;
}
/**
* Subscribes to SSE for all pending generations. When a generation completes,
* invalidates the history query, removes it from pending, and auto-plays
* if the player is idle.
*/
export function useGenerationProgress() {
const queryClient = useQueryClient();
const { toast } = useToast();
const pendingIds = useGenerationStore((s) => s.pendingGenerationIds);
const removePendingGeneration = useGenerationStore((s) => s.removePendingGeneration);
const removePendingStoryAdd = useGenerationStore((s) => s.removePendingStoryAdd);
const isPlaying = usePlayerStore((s) => s.isPlaying);
const setAudioWithAutoPlay = usePlayerStore((s) => s.setAudioWithAutoPlay);
const autoplayOnGenerate = useServerStore((s) => s.autoplayOnGenerate);
// Keep refs to avoid stale closures in EventSource handlers
const isPlayingRef = useRef(isPlaying);
const autoplayRef = useRef(autoplayOnGenerate);
isPlayingRef.current = isPlaying;
autoplayRef.current = autoplayOnGenerate;
// Track active EventSource instances
const eventSourcesRef = useRef<Map<string, EventSource>>(new Map());
// Unmount-only cleanup — close all SSE connections when the hook is torn down
useEffect(() => {
const sources = eventSourcesRef.current;
return () => {
for (const source of sources.values()) {
source.close();
}
sources.clear();
};
}, []);
useEffect(() => {
const currentSources = eventSourcesRef.current;
// Close SSE connections for IDs no longer pending
for (const [id, source] of currentSources.entries()) {
if (!pendingIds.has(id)) {
source.close();
currentSources.delete(id);
}
}
// Open SSE connections for new pending IDs
for (const id of pendingIds) {
if (currentSources.has(id)) continue;
const url = apiClient.getGenerationStatusUrl(id);
const source = new EventSource(url);
source.onmessage = (event) => {
try {
const data: GenerationStatusEvent = JSON.parse(event.data);
if (data.status === 'completed') {
source.close();
currentSources.delete(id);
removePendingGeneration(id);
// Refresh history to pick up the completed generation
queryClient.invalidateQueries({ queryKey: ['history'] });
// If this generation was queued for a story, add it now
const storyId = removePendingStoryAdd(id);
if (storyId) {
apiClient
.addStoryItem(storyId, { generation_id: id })
.then(() => {
queryClient.invalidateQueries({ queryKey: ['stories'] });
queryClient.invalidateQueries({ queryKey: ['stories', storyId] });
toast({
title: 'Added to story',
description: data.duration
? `Audio generated (${data.duration.toFixed(2)}s) and added to story`
: 'Audio generated and added to story',
});
})
.catch(() => {
toast({
title: 'Generation complete',
description: 'Audio generated but failed to add to story',
variant: 'destructive',
});
});
} else {
// toast({
// title: 'Generation complete!',
// description: data.duration
// ? `Audio generated (${data.duration.toFixed(2)}s)`
// : 'Audio generated',
// });
}
// Auto-play if enabled and nothing is currently playing
if (autoplayRef.current && !isPlayingRef.current) {
const genAudioUrl = apiClient.getAudioUrl(id);
setAudioWithAutoPlay(genAudioUrl, id, '', '');
}
} else if (data.status === 'failed' || data.status === 'not_found') {
source.close();
currentSources.delete(id);
removePendingGeneration(id);
removePendingStoryAdd(id);
queryClient.invalidateQueries({ queryKey: ['history'] });
toast({
title: data.status === 'not_found' ? 'Generation not found' : 'Generation failed',
description: data.error || 'An error occurred during generation',
variant: 'destructive',
});
}
} catch {
// Ignore parse errors from heartbeats etc
}
};
source.onerror = () => {
// EventSource auto-reconnects, but if we get repeated errors
// just clean up
source.close();
currentSources.delete(id);
removePendingGeneration(id);
};
currentSources.set(id, source);
}
}, [
pendingIds,
removePendingGeneration,
removePendingStoryAdd,
queryClient,
toast,
setAudioWithAutoPlay,
]);
}
+34 -11
View File
@@ -1,4 +1,6 @@
import { CheckCircle2, Loader2, XCircle } from 'lucide-react';
import { CancelCircleIcon, CheckmarkCircle02Icon } from '@hugeicons/core-free-icons';
import { HugeiconsIcon } from '@hugeicons/react';
import { Icon } from '@iconify/react';
import { useCallback, useEffect, useRef } from 'react';
import { Progress } from '@/components/ui/progress';
import { useToast } from '@/components/ui/use-toast';
@@ -10,7 +12,7 @@ interface UseModelDownloadToastOptions {
displayName: string;
enabled?: boolean;
onComplete?: () => void;
onError?: (error: string) => void;
onError?: () => void;
}
/**
@@ -59,7 +61,7 @@ export function useModelDownloadToast({
title: displayName,
description: (
<div className="flex items-center gap-2">
<Loader2 className="h-4 w-4 animate-spin" />
<Icon icon="svg-spinners:ring-resize" className="h-4 w-4 animate-spin" />
<span>Connecting to download...</span>
</div>
),
@@ -96,19 +98,35 @@ export function useModelDownloadToast({
switch (progress.status) {
case 'complete':
statusIcon = <CheckCircle2 className="h-4 w-4 text-green-500" />;
statusIcon = (
<HugeiconsIcon
icon={CheckmarkCircle02Icon}
size={16}
className="h-4 w-4 text-green-500"
/>
);
statusText = 'Download complete';
break;
case 'error':
statusIcon = <XCircle className="h-4 w-4 text-destructive" />;
statusText = 'Download failed. See Problems panel for details.';
statusIcon = (
<HugeiconsIcon
icon={CancelCircleIcon}
size={16}
className="h-4 w-4 text-destructive"
/>
);
statusText = `Error: ${progress.error || 'Unknown error'}`;
break;
case 'downloading':
statusIcon = <Loader2 className="h-4 w-4 animate-spin" />;
statusIcon = (
<Icon icon="svg-spinners:ring-resize" className="h-4 w-4 animate-spin" />
);
statusText = progress.filename || 'Downloading...';
break;
case 'extracting':
statusIcon = <Loader2 className="h-4 w-4 animate-spin" />;
statusIcon = (
<Icon icon="svg-spinners:ring-resize" className="h-4 w-4 animate-spin" />
);
statusText = 'Extracting...';
break;
}
@@ -131,7 +149,8 @@ export function useModelDownloadToast({
)}
</div>
),
duration: progress.status === 'complete' || progress.status === 'error' ? 5000 : Infinity,
duration: progress.status === 'complete' ? 5000 : Infinity,
variant: progress.status === 'error' ? 'destructive' : 'default',
});
// Close connection and dismiss toast on completion or error
@@ -153,7 +172,11 @@ export function useModelDownloadToast({
toastUpdateRef.current({
title: (
<div className="flex items-center gap-2">
<CheckCircle2 className="h-4 w-4 text-green-500" />
<HugeiconsIcon
icon={CheckmarkCircle02Icon}
size={16}
className="h-4 w-4 text-green-500"
/>
<span>{displayName}</span>
</div>
),
@@ -168,7 +191,7 @@ export function useModelDownloadToast({
onComplete();
} else if (isError && onError) {
console.log('[useModelDownloadToast] Download error, calling onError callback');
onError(progress.error || 'Unknown error');
onError();
}
}
}
+12 -12
View File
@@ -1,23 +1,23 @@
import { useCallback, useEffect, useRef, useState } from 'react';
import { apiClient } from '@/lib/api/client';
import type { ActiveDownloadTask } from '@/lib/api/types';
import { useGenerationStore } from '@/stores/generationStore';
import type { ActiveDownloadTask } from '@/lib/api/types';
// Polling interval in milliseconds
const POLL_INTERVAL = 30000;
const POLL_INTERVAL = 2000;
/**
* Hook to monitor active tasks (downloads and generations).
* Polls the server periodically to catch downloads triggered from anywhere
* (transcription, generation, explicit download, etc.).
*
*
* Returns the active downloads so components can render download toasts.
*/
export function useRestoreActiveTasks() {
const [activeDownloads, setActiveDownloads] = useState<ActiveDownloadTask[]>([]);
const setIsGenerating = useGenerationStore((state) => state.setIsGenerating);
const setActiveGenerationId = useGenerationStore((state) => state.setActiveGenerationId);
const addPendingGeneration = useGenerationStore((state) => state.addPendingGeneration);
// Track which downloads we've seen to detect new ones
const seenDownloadsRef = useRef<Set<string>>(new Set());
@@ -25,15 +25,15 @@ export function useRestoreActiveTasks() {
try {
const tasks = await apiClient.getActiveTasks();
// Restore pending generations (e.g., after page refresh)
// Update generation state
if (tasks.generations.length > 0) {
setIsGenerating(true);
setActiveGenerationId(tasks.generations[0].task_id);
for (const gen of tasks.generations) {
addPendingGeneration(gen.task_id);
}
} else {
// Only clear if we were tracking a generation
const currentId = useGenerationStore.getState().activeGenerationId;
if (currentId) {
setIsGenerating(false);
setActiveGenerationId(null);
}
}
@@ -41,14 +41,14 @@ export function useRestoreActiveTasks() {
// Update active downloads
// Keep track of all active downloads (including new ones)
const currentDownloadNames = new Set(tasks.downloads.map((d) => d.model_name));
// Remove completed downloads from our seen set
for (const name of seenDownloadsRef.current) {
if (!currentDownloadNames.has(name)) {
seenDownloadsRef.current.delete(name);
}
}
// Add new downloads to seen set
for (const download of tasks.downloads) {
seenDownloadsRef.current.add(download.model_name);
@@ -59,7 +59,7 @@ export function useRestoreActiveTasks() {
// Silently fail - server might be temporarily unavailable
console.debug('Failed to fetch active tasks:', error);
}
}, [setActiveGenerationId, addPendingGeneration]);
}, [setIsGenerating, setActiveGenerationId]);
useEffect(() => {
// Fetch immediately on mount
+8 -61
View File
@@ -1,15 +1,6 @@
import { useMutation, useQuery, useQueryClient } from '@tanstack/react-query';
import { apiClient } from '@/lib/api/client';
import type {
StoryCreate,
StoryItemBatchUpdate,
StoryItemCreate,
StoryItemMove,
StoryItemReorder,
StoryItemSplit,
StoryItemTrim,
StoryItemVersionUpdate,
} from '@/lib/api/types';
import type { StoryCreate, StoryItemCreate, StoryItemBatchUpdate, StoryItemReorder, StoryItemMove, StoryItemTrim, StoryItemSplit } from '@/lib/api/types';
import { usePlatform } from '@/platform/PlatformContext';
export function useStories() {
@@ -118,15 +109,8 @@ export function useMoveStoryItem() {
const queryClient = useQueryClient();
return useMutation({
mutationFn: ({
storyId,
itemId,
data,
}: {
storyId: string;
itemId: string;
data: StoryItemMove;
}) => apiClient.moveStoryItem(storyId, itemId, data),
mutationFn: ({ storyId, itemId, data }: { storyId: string; itemId: string; data: StoryItemMove }) =>
apiClient.moveStoryItem(storyId, itemId, data),
onSuccess: (_, variables) => {
queryClient.invalidateQueries({ queryKey: ['stories'] });
queryClient.invalidateQueries({ queryKey: ['stories', variables.storyId] });
@@ -138,15 +122,8 @@ export function useTrimStoryItem() {
const queryClient = useQueryClient();
return useMutation({
mutationFn: ({
storyId,
itemId,
data,
}: {
storyId: string;
itemId: string;
data: StoryItemTrim;
}) => apiClient.trimStoryItem(storyId, itemId, data),
mutationFn: ({ storyId, itemId, data }: { storyId: string; itemId: string; data: StoryItemTrim }) =>
apiClient.trimStoryItem(storyId, itemId, data),
onSuccess: (_, variables) => {
queryClient.invalidateQueries({ queryKey: ['stories'] });
queryClient.invalidateQueries({ queryKey: ['stories', variables.storyId] });
@@ -158,15 +135,8 @@ export function useSplitStoryItem() {
const queryClient = useQueryClient();
return useMutation({
mutationFn: ({
storyId,
itemId,
data,
}: {
storyId: string;
itemId: string;
data: StoryItemSplit;
}) => apiClient.splitStoryItem(storyId, itemId, data),
mutationFn: ({ storyId, itemId, data }: { storyId: string; itemId: string; data: StoryItemSplit }) =>
apiClient.splitStoryItem(storyId, itemId, data),
onSuccess: (_, variables) => {
queryClient.invalidateQueries({ queryKey: ['stories'] });
queryClient.invalidateQueries({ queryKey: ['stories', variables.storyId] });
@@ -187,26 +157,6 @@ export function useDuplicateStoryItem() {
});
}
export function useSetStoryItemVersion() {
const queryClient = useQueryClient();
return useMutation({
mutationFn: ({
storyId,
itemId,
data,
}: {
storyId: string;
itemId: string;
data: StoryItemVersionUpdate;
}) => apiClient.setStoryItemVersion(storyId, itemId, data),
onSuccess: (_, variables) => {
queryClient.invalidateQueries({ queryKey: ['stories'] });
queryClient.invalidateQueries({ queryKey: ['stories', variables.storyId] });
},
});
}
export function useExportStoryAudio() {
const platform = usePlatform();
@@ -215,10 +165,7 @@ export function useExportStoryAudio() {
const blob = await apiClient.exportStoryAudio(storyId);
// Create safe filename
const safeName = storyName
.substring(0, 50)
.replace(/[^a-z0-9]/gi, '-')
.toLowerCase();
const safeName = storyName.substring(0, 50).replace(/[^a-z0-9]/gi, '-').toLowerCase();
const filename = `${safeName || 'story'}.wav`;
await platform.filesystem.saveFile(filename, blob, [
+11 -23
View File
@@ -70,16 +70,6 @@ export function useStoryPlayback(items: StoryItemDetail[] | undefined) {
}
}, []);
// Resolve the audio buffer key and URL for an item.
// When a version_id is pinned, use that version's audio; otherwise use the generation default.
const getAudioKey = (item: StoryItemDetail) =>
item.version_id ? `v:${item.version_id}` : item.generation_id;
const getAudioUrlForItem = (item: StoryItemDetail) =>
item.version_id
? apiClient.getVersionAudioUrl(item.version_id)
: apiClient.getAudioUrl(item.generation_id);
// Preload audio files as AudioBuffers
useEffect(() => {
if (!items || items.length === 0) {
@@ -88,12 +78,12 @@ export function useStoryPlayback(items: StoryItemDetail[] | undefined) {
return;
}
const currentKeys = new Set(items.map(getAudioKey));
const currentIds = new Set(items.map((item) => item.generation_id));
const audioContext = getAudioContext();
// Remove buffers for items that no longer exist
for (const [id] of audioBuffersRef.current) {
if (!currentKeys.has(id)) {
if (!currentIds.has(id)) {
audioBuffersRef.current.delete(id);
}
}
@@ -101,25 +91,24 @@ export function useStoryPlayback(items: StoryItemDetail[] | undefined) {
// Preload audio for new items
const preloadPromises: Promise<void>[] = [];
for (const item of items) {
const key = getAudioKey(item);
if (!audioBuffersRef.current.has(key)) {
const audioUrl = getAudioUrlForItem(item);
console.log('[StoryPlayback] Preloading audio buffer:', key);
if (!audioBuffersRef.current.has(item.generation_id)) {
const audioUrl = apiClient.getAudioUrl(item.generation_id);
console.log('[StoryPlayback] Preloading audio buffer:', item.generation_id);
const preloadPromise = fetch(audioUrl)
.then((response) => response.arrayBuffer())
.then((arrayBuffer) => audioContext.decodeAudioData(arrayBuffer))
.then((audioBuffer) => {
audioBuffersRef.current.set(key, audioBuffer);
audioBuffersRef.current.set(item.generation_id, audioBuffer);
console.log(
'[StoryPlayback] Preloaded buffer:',
key,
item.generation_id,
'duration:',
audioBuffer.duration,
);
})
.catch((err) => {
console.error('[StoryPlayback] Failed to preload audio:', key, err);
console.error('[StoryPlayback] Failed to preload audio:', item.generation_id, err);
});
preloadPromises.push(preloadPromise);
@@ -227,16 +216,15 @@ export function useStoryPlayback(items: StoryItemDetail[] | undefined) {
// Schedule new sources for items that should be playing
for (const item of shouldBePlaying) {
if (!activeSourcesRef.current.has(item.id)) {
const bufferKey = getAudioKey(item);
const buffer = audioBuffersRef.current.get(bufferKey);
const buffer = audioBuffersRef.current.get(item.generation_id);
if (!buffer) {
console.warn('[StoryPlayback] Buffer not loaded for:', bufferKey);
console.warn('[StoryPlayback] Buffer not loaded for:', item.generation_id);
continue;
}
// Calculate when this item should start in AudioContext time
const itemStartContextTime = storyTimeToContextTime(item.start_time_ms);
// Calculate effective duration and trim offsets
const trimStartSec = (item.trim_start_ms || 0) / 1000;
const trimEndSec = (item.trim_end_ms || 0) / 1000;
+14
View File
@@ -0,0 +1,14 @@
import { useQuery } from '@tanstack/react-query';
import { apiClient } from '@/lib/api/client';
import { useServerStore } from '@/stores/serverStore';
export function useSystemFolders() {
const serverUrl = useServerStore((state) => state.serverUrl);
return useQuery({
queryKey: ['system', 'folders', serverUrl],
queryFn: () => apiClient.getSystemFolders(),
staleTime: 60000, // Cache for 1 minute - folder paths don't change often
retry: 1,
});
}
+18 -36
View File
@@ -22,11 +22,6 @@ export function formatAudioDuration(seconds: number): string {
* If the file has a recordedDuration property (from recording hooks),
* use that instead of trying to read metadata. This fixes issues on Windows
* where WebM files from MediaRecorder don't have proper duration metadata.
*
* For uploaded files we use AudioContext.decodeAudioData which fully decodes
* the audio and returns the exact duration. This is more reliable than
* HTMLMediaElement.duration which can return incorrect large values for VBR
* MP3 files that lack a proper XING/VBRI header.
*/
export async function getAudioDuration(
file: File & { recordedDuration?: number },
@@ -35,39 +30,26 @@ export async function getAudioDuration(
return file.recordedDuration;
}
// Use Web Audio API for accurate duration — avoids VBR MP3 metadata issues.
try {
const audioContext = new AudioContext();
try {
const arrayBuffer = await file.arrayBuffer();
const audioBuffer = await audioContext.decodeAudioData(arrayBuffer);
return audioBuffer.duration;
} finally {
await audioContext.close();
}
} catch {
// Fallback: read duration from the media element (less accurate but works for WAV).
return new Promise((resolve, reject) => {
const audio = new Audio();
const url = URL.createObjectURL(file);
return new Promise((resolve, reject) => {
const audio = new Audio();
const url = URL.createObjectURL(file);
audio.addEventListener('loadedmetadata', () => {
URL.revokeObjectURL(url);
if (Number.isFinite(audio.duration) && audio.duration > 0) {
resolve(audio.duration);
} else {
reject(new Error('Audio file has invalid duration metadata'));
}
});
audio.addEventListener('error', () => {
URL.revokeObjectURL(url);
reject(new Error('Failed to load audio file'));
});
audio.src = url;
audio.addEventListener('loadedmetadata', () => {
URL.revokeObjectURL(url);
if (Number.isFinite(audio.duration) && audio.duration > 0) {
resolve(audio.duration);
} else {
reject(new Error('Audio file has invalid duration metadata'));
}
});
}
audio.addEventListener('error', () => {
URL.revokeObjectURL(url);
reject(new Error('Failed to load audio file'));
});
audio.src = url;
});
}
/**
+1 -16
View File
@@ -21,25 +21,10 @@ export function formatDate(date: string | Date): string {
} else {
dateObj = date;
}
return formatDistance(dateObj, new Date(), { addSuffix: true }).replace(/^about /i, '');
}
const ENGINE_DISPLAY_NAMES: Record<string, string> = {
qwen: 'Qwen',
luxtts: 'LuxTTS',
chatterbox: 'Chatterbox',
chatterbox_turbo: 'Chatterbox Turbo',
};
export function formatEngineName(engine?: string, modelSize?: string): string {
const name = ENGINE_DISPLAY_NAMES[engine ?? 'qwen'] ?? engine ?? 'Qwen';
if (engine === 'qwen' && modelSize) {
return `${name} ${modelSize}`;
}
return name;
}
export function formatFileSize(bytes: number): string {
if (bytes === 0) return '0 Bytes';
const k = 1024;
+8 -4
View File
@@ -10,8 +10,13 @@ export interface FileFilter {
export interface PlatformFilesystem {
saveFile(filename: string, blob: Blob, filters?: FileFilter[]): Promise<void>;
openPath(path: string): Promise<void>;
pickDirectory(title: string): Promise<string | null>;
/**
* Open a folder in the native file explorer.
* On web, this is a no-op since browsers cannot open folders.
* @param path - The absolute path to the folder to open
* @returns true if the folder was opened, false if not supported
*/
openFolder(path: string): Promise<boolean>;
}
export interface UpdateStatus {
@@ -51,9 +56,8 @@ export interface PlatformAudio {
}
export interface PlatformLifecycle {
startServer(remote?: boolean, modelsDir?: string | null): Promise<string>;
startServer(remote?: boolean): Promise<string>;
stopServer(): Promise<void>;
restartServer(modelsDir?: string | null): Promise<string>;
setKeepServerRunning(keep: boolean): Promise<void>;
setupWindowCloseHandler(): Promise<void>;
onServerReady?: () => void;
-14
View File
@@ -1,7 +1,6 @@
import { createRootRoute, createRoute, createRouter, Outlet } from '@tanstack/react-router';
import { AppFrame } from '@/components/AppFrame/AppFrame';
import { AudioTab } from '@/components/AudioTab/AudioTab';
import { EffectsTab } from '@/components/EffectsTab/EffectsTab';
import { MainEditor } from '@/components/MainEditor/MainEditor';
import { ModelsTab } from '@/components/ModelsTab/ModelsTab';
import { ServerTab } from '@/components/ServerTab/ServerTab';
@@ -9,10 +8,8 @@ import { Sidebar } from '@/components/Sidebar';
import { StoriesTab } from '@/components/StoriesTab/StoriesTab';
import { Toaster } from '@/components/ui/toaster';
import { VoicesTab } from '@/components/VoicesTab/VoicesTab';
import { useGenerationProgress } from '@/lib/hooks/useGenerationProgress';
import { useModelDownloadToast } from '@/lib/hooks/useModelDownloadToast';
import { MODEL_DISPLAY_NAMES, useRestoreActiveTasks } from '@/lib/hooks/useRestoreActiveTasks';
// Simple platform check that works in both web and Tauri
const isMacOS = () => navigator.platform.toLowerCase().includes('mac');
@@ -21,9 +18,6 @@ function RootLayout() {
// Monitor active downloads/generations and show toasts for them
const activeDownloads = useRestoreActiveTasks();
// Subscribe to SSE for pending generations — handles completion, auto-play, and history refresh
useGenerationProgress();
return (
<AppFrame>
<div className="flex flex-1 min-h-0 overflow-hidden">
@@ -106,13 +100,6 @@ const audioRoute = createRoute({
component: AudioTab,
});
// Effects route
const effectsRoute = createRoute({
getParentRoute: () => rootRoute,
path: '/effects',
component: EffectsTab,
});
// Models route
const modelsRoute = createRoute({
getParentRoute: () => rootRoute,
@@ -133,7 +120,6 @@ const routeTree = rootRoute.addChildren([
storiesRoute,
voicesRoute,
audioRoute,
effectsRoute,
modelsRoute,
serverRoute,
]);
-26
View File
@@ -1,26 +0,0 @@
import { create } from 'zustand';
import type { EffectConfig } from '@/lib/api/types';
interface EffectsStore {
selectedPresetId: string | null;
setSelectedPresetId: (id: string | null) => void;
// Working chain for the detail panel (editing a preset or building a new one)
workingChain: EffectConfig[];
setWorkingChain: (chain: EffectConfig[]) => void;
// Track if editing an existing preset vs creating new
isCreatingNew: boolean;
setIsCreatingNew: (v: boolean) => void;
}
export const useEffectsStore = create<EffectsStore>((set) => ({
selectedPresetId: null,
setSelectedPresetId: (id) => set({ selectedPresetId: id, isCreatingNew: false }),
workingChain: [],
setWorkingChain: (chain) => set({ workingChain: chain }),
isCreatingNew: false,
setIsCreatingNew: (v) => set({ isCreatingNew: v, ...(v && { selectedPresetId: null }) }),
}));
+4 -47
View File
@@ -1,58 +1,15 @@
import { create } from 'zustand';
interface GenerationState {
/** IDs of generations currently in progress */
pendingGenerationIds: Set<string>;
/** Whether any generation is in progress (derived from pendingGenerationIds) */
isGenerating: boolean;
/** Map of generationId → storyId for deferred story additions */
pendingStoryAdds: Map<string, string>;
addPendingGeneration: (id: string) => void;
removePendingGeneration: (id: string) => void;
addPendingStoryAdd: (generationId: string, storyId: string) => void;
removePendingStoryAdd: (generationId: string) => string | undefined;
setActiveGenerationId: (id: string | null) => void;
activeGenerationId: string | null;
setIsGenerating: (generating: boolean) => void;
setActiveGenerationId: (id: string | null) => void;
}
export const useGenerationStore = create<GenerationState>((set, get) => ({
pendingGenerationIds: new Set(),
export const useGenerationStore = create<GenerationState>((set) => ({
isGenerating: false,
activeGenerationId: null,
pendingStoryAdds: new Map(),
addPendingGeneration: (id) =>
set((state) => {
const next = new Set(state.pendingGenerationIds);
next.add(id);
return { pendingGenerationIds: next, isGenerating: true };
}),
removePendingGeneration: (id) =>
set((state) => {
const next = new Set(state.pendingGenerationIds);
next.delete(id);
return { pendingGenerationIds: next, isGenerating: next.size > 0 };
}),
addPendingStoryAdd: (generationId, storyId) =>
set((state) => {
const next = new Map(state.pendingStoryAdds);
next.set(generationId, storyId);
return { pendingStoryAdds: next };
}),
removePendingStoryAdd: (generationId) => {
const storyId = get().pendingStoryAdds.get(generationId);
if (storyId) {
set((state) => {
const next = new Map(state.pendingStoryAdds);
next.delete(generationId);
return { pendingStoryAdds: next };
});
}
return storyId;
},
setIsGenerating: (generating) => set({ isGenerating: generating }),
setActiveGenerationId: (id) => set({ activeGenerationId: id }),
}));
-30
View File
@@ -13,21 +13,6 @@ interface ServerStore {
keepServerRunningOnClose: boolean;
setKeepServerRunningOnClose: (keepRunning: boolean) => void;
maxChunkChars: number;
setMaxChunkChars: (value: number) => void;
crossfadeMs: number;
setCrossfadeMs: (value: number) => void;
normalizeAudio: boolean;
setNormalizeAudio: (value: boolean) => void;
autoplayOnGenerate: boolean;
setAutoplayOnGenerate: (value: boolean) => void;
customModelsDir: string | null;
setCustomModelsDir: (dir: string | null) => void;
}
export const useServerStore = create<ServerStore>()(
@@ -44,21 +29,6 @@ export const useServerStore = create<ServerStore>()(
keepServerRunningOnClose: false,
setKeepServerRunningOnClose: (keepRunning) => set({ keepServerRunningOnClose: keepRunning }),
maxChunkChars: 800,
setMaxChunkChars: (value) => set({ maxChunkChars: value }),
crossfadeMs: 50,
setCrossfadeMs: (value) => set({ crossfadeMs: value }),
normalizeAudio: true,
setNormalizeAudio: (value) => set({ normalizeAudio: value }),
autoplayOnGenerate: true,
setAutoplayOnGenerate: (value) => set({ autoplayOnGenerate: value }),
customModelsDir: null,
setCustomModelsDir: (dir) => set({ customModelsDir: dir }),
}),
{
name: 'voicebox-server',
-7
View File
@@ -31,10 +31,6 @@ interface UIStore {
selectedProfileId: string | null;
setSelectedProfileId: (id: string | null) => void;
// Selected voice in Voices tab inspector
selectedVoiceId: string | null;
setSelectedVoiceId: (id: string | null) => void;
// Profile form draft (for persisting create voice modal state)
profileFormDraft: ProfileFormDraft | null;
setProfileFormDraft: (draft: ProfileFormDraft | null) => void;
@@ -59,9 +55,6 @@ export const useUIStore = create<UIStore>((set) => ({
selectedProfileId: null,
setSelectedProfileId: (id) => set({ selectedProfileId: id }),
selectedVoiceId: null,
setSelectedVoiceId: (id) => set({ selectedVoiceId: id }),
profileFormDraft: null,
setProfileFormDraft: (draft) => set({ profileFormDraft: draft }),
+420 -96
View File
@@ -1,135 +1,459 @@
# Voicebox Backend
# voicebox Backend
FastAPI server powering voice cloning, speech generation, and audio processing. Runs locally as a Tauri sidecar or standalone via `python -m backend.main`.
Production-quality FastAPI backend for Qwen3-TTS voice cloning.
## Running
## Features
```bash
# Via justfile (recommended)
just dev:server
# Standalone
python -m backend.main --host 127.0.0.1 --port 17493
# With custom data directory
python -m backend.main --data-dir /path/to/data
```
The server auto-initializes the SQLite database on first startup. Models are downloaded from HuggingFace on first use.
- ✅ **Voice Profile Management** - Create, update, delete voice profiles with multi-sample support
- ✅ **Voice Cloning** - Generate speech using voice profiles with caching
- ✅ **Generation History** - Full history tracking with search and filtering
- ✅ **Transcription** - Whisper-based audio transcription
- ✅ **Multi-Sample Profiles** - Combine multiple reference samples for better quality
- ✅ **Voice Prompt Caching** - Dual memory + disk caching for fast generation
- ✅ **Audio Validation** - Automatic validation of reference audio quality
- ✅ **Model Management** - Lazy loading and VRAM management
## Architecture
```
backend/
app.py # FastAPI app factory, CORS, lifecycle events
main.py # Entry point (imports app, runs uvicorn)
config.py # Data directory paths and configuration
models.py # Pydantic request/response schemas
server.py # Tauri sidecar launcher, parent-pid watchdog
routes/ # Thin HTTP handlers — validation, delegation, response formatting
services/ # Business logic, CRUD, orchestration
backends/ # TTS/STT engine implementations (MLX, PyTorch, etc.)
database/ # ORM models, session management, migrations, seed data
utils/ # Shared utilities (audio, effects, caching, progress tracking)
├── main.py # FastAPI app with all routes
├── models.py # Pydantic request/response models
├── platform_detect.py # Platform detection for backend selection
├── tts.py # TTS backend abstraction (delegates to MLX or PyTorch)
├── transcribe.py # STT backend abstraction (delegates to MLX or PyTorch)
├── backends/ # Backend implementations
│ ├── __init__.py # Backend factory and protocols
│ ├── mlx_backend.py # MLX backend (Apple Silicon)
│ └── pytorch_backend.py # PyTorch backend (Windows/Linux/Intel)
├── profiles.py # Voice profile CRUD
├── history.py # Generation history
├── studio.py # Audio editing (TODO)
├── database.py # SQLite ORM
└── utils/
├── audio.py # Audio processing utilities
├── cache.py # Voice prompt caching
└── validation.py # Input validation
```
### Request flow
### Backend Selection
```
HTTP request
-> routes/ (validate input, parse params)
-> services/ (business logic, database queries, orchestration)
-> backends/ (TTS/STT inference)
-> utils/ (audio processing, effects, caching)
Voicebox automatically selects the best backend based on platform:
- **Apple Silicon (M1/M2/M3)**: Uses MLX backend with native Metal acceleration (4-5x faster)
- **Windows/Linux/Intel Mac**: Uses PyTorch backend (CUDA GPU if available, CPU fallback)
The backend is detected at runtime via `platform_detect.py`. Both backends implement the same interface, so the API remains consistent across platforms.
## API Endpoints
### Health & Info
#### `GET /`
Root endpoint with version info.
#### `GET /health`
Health check with model status.
**Response:**
```json
{
"status": "healthy",
"model_loaded": true,
"gpu_available": true,
"gpu_type": "Metal (Apple Silicon via MLX)",
"backend_type": "mlx",
"vram_used_mb": null
}
```
Route handlers are intentionally thin. They validate input, delegate to a service function, and format the response. All business logic lives in `services/`.
**Backend Types:**
- `"mlx"` - MLX backend (Apple Silicon with Metal acceleration)
- `"pytorch"` - PyTorch backend (Windows/Linux/Intel Mac)
### Key modules
### Voice Profiles
**services/generation.py** -- Single `run_generation()` function that handles all three generation modes (generate, retry, regenerate). Manages model loading, voice prompt creation, chunked inference, normalization, effects, and version persistence.
**Note:** The database is automatically initialized when the server starts. No manual setup required.
**services/task_queue.py** -- Serial generation queue. Ensures only one GPU inference runs at a time. Background tasks are tracked to prevent garbage collection.
#### `POST /profiles`
Create a new voice profile.
**backends/__init__.py** -- Protocol definitions (`TTSBackend`, `STTBackend`), model config registry, and factory functions. Adding a new engine means implementing the protocol and registering a config entry.
**Request:**
```json
{
"name": "My Voice",
"description": "Optional description",
"language": "en"
}
```
**backends/base.py** -- Shared utilities used across all engine implementations: HuggingFace cache checks, device detection, voice prompt combination, progress tracking.
**Response:**
```json
{
"id": "uuid",
"name": "My Voice",
"description": "Optional description",
"language": "en",
"created_at": "2024-01-01T00:00:00Z",
"updated_at": "2024-01-01T00:00:00Z"
}
```
**database/** -- SQLAlchemy ORM models with a re-exporting `__init__.py` for backward compatibility. Migrations run automatically on startup.
#### `GET /profiles`
List all voice profiles.
### Backend selection
#### `GET /profiles/{profile_id}`
Get a specific profile.
The server detects the best inference backend at startup:
#### `PUT /profiles/{profile_id}`
Update a profile.
| Platform | Backend | Acceleration |
|----------|---------|-------------|
| macOS (Apple Silicon) | MLX | Metal / Neural Engine |
| Windows / Linux (NVIDIA) | PyTorch | CUDA |
| Linux (AMD) | PyTorch | ROCm |
| Intel Arc | PyTorch | IPEX / XPU |
| Windows (any GPU) | PyTorch | DirectML |
| Any | PyTorch | CPU fallback |
#### `DELETE /profiles/{profile_id}`
Delete a profile and all associated samples.
Detection is handled by `utils/platform_detect.py`. Both backends implement the same `TTSBackend` protocol, so the API layer is engine-agnostic.
#### `POST /profiles/{profile_id}/samples`
Add a sample to a profile.
## API
**Form Data:**
- `file`: Audio file (WAV, MP3, etc.)
- `reference_text`: Transcript of the audio
90 endpoints organized by domain. Full interactive documentation available at `http://localhost:17493/docs` when the server is running.
**Response:**
```json
{
"id": "sample-uuid",
"profile_id": "profile-uuid",
"audio_path": "/path/to/sample.wav",
"reference_text": "This is my voice"
}
```
| Domain | Prefix | Description |
|--------|--------|-------------|
| Health | `/`, `/health` | Server status, GPU info, filesystem checks |
| Profiles | `/profiles` | Voice profile CRUD, samples, avatars, import/export |
| Channels | `/channels` | Audio channel management and voice assignment |
| Generation | `/generate` | TTS generation, retry, regenerate, status SSE |
| History | `/history` | Generation history, search, favorites, export |
| Transcription | `/transcribe` | Whisper-based audio-to-text |
| Stories | `/stories` | Multi-track timeline editor, audio export |
| Effects | `/effects` | Effect presets, preview, version management |
| Audio | `/audio`, `/samples` | Audio file serving |
| Models | `/models` | Load, unload, download, migrate, status |
| Tasks | `/tasks`, `/cache` | Active task tracking, cache management |
| CUDA | `/backend/cuda-*` | CUDA binary download and management |
#### `GET /profiles/{profile_id}/samples`
List all samples for a profile.
### Quick examples
#### `DELETE /profiles/samples/{sample_id}`
Delete a specific sample.
### Generation
#### `POST /generate`
Generate speech from text using a voice profile.
**Request:**
```json
{
"profile_id": "uuid",
"text": "Hello, this is a test.",
"language": "en",
"seed": 42
}
```
**Response:**
```json
{
"id": "generation-uuid",
"profile_id": "profile-uuid",
"text": "Hello, this is a test.",
"language": "en",
"audio_path": "/path/to/audio.wav",
"duration": 2.5,
"seed": 42,
"created_at": "2024-01-01T00:00:00Z"
}
```
### History
#### `GET /history`
List generation history with optional filters.
**Query Parameters:**
- `profile_id` (optional): Filter by profile
- `search` (optional): Search in text content
- `limit` (default: 50): Results per page
- `offset` (default: 0): Pagination offset
#### `GET /history/{generation_id}`
Get a specific generation.
#### `DELETE /history/{generation_id}`
Delete a generation.
#### `GET /history/stats`
Get generation statistics.
**Response:**
```json
{
"total_generations": 100,
"total_duration_seconds": 250.5,
"generations_by_profile": {
"profile-uuid-1": 50,
"profile-uuid-2": 50
}
}
```
### Audio Files
#### `GET /audio/{generation_id}`
Download generated audio file.
Returns WAV file with appropriate headers.
### Transcription
#### `POST /transcribe`
Transcribe audio file to text.
**Form Data:**
- `file`: Audio file
- `language` (optional): Language hint (en or zh)
**Response:**
```json
{
"text": "Transcribed text here",
"duration": 5.5
}
```
### Model Management
#### `POST /models/load`
Manually load TTS model.
**Query Parameters:**
- `model_size`: Model size (1.7B or 0.6B)
#### `POST /models/unload`
Unload TTS model to free memory.
## Database Schema
### profiles
- `id`: UUID primary key
- `name`: Profile name (unique)
- `description`: Optional description
- `language`: Language code (en/zh)
- `created_at`: Creation timestamp
- `updated_at`: Last update timestamp
### profile_samples
- `id`: UUID primary key
- `profile_id`: Foreign key to profiles
- `audio_path`: Path to audio file
- `reference_text`: Transcript
### generations
- `id`: UUID primary key
- `profile_id`: Foreign key to profiles
- `text`: Generated text
- `language`: Language code
- `audio_path`: Path to audio file
- `duration`: Duration in seconds
- `seed`: Random seed (optional)
- `created_at`: Creation timestamp
### projects
- `id`: UUID primary key
- `name`: Project name
- `data`: JSON data
- `created_at`: Creation timestamp
- `updated_at`: Last update timestamp
## File Structure
```
data/
├── profiles/
│ └── {profile_id}/
│ ├── {sample_id}.wav
│ └── ...
├── generations/
│ └── {generation_id}.wav
├── cache/
│ └── {hash}.prompt
├── projects/
│ └── {project_id}.json
└── voicebox.db
```
## Setup
### 1. Install Dependencies
```bash
# Generate speech
curl -X POST http://localhost:17493/generate \
pip install -r requirements.txt
```
**Note:** On Apple Silicon, also install MLX dependencies for faster inference:
```bash
pip install -r requirements-mlx.txt
```
### 2. Download Models (Automatic)
The Qwen3-TTS models are automatically downloaded from HuggingFace Hub on first use, similar to how Whisper models work.
**No manual download required!** The models will be cached locally after the first download.
Available models:
- **1.7B** (recommended): `Qwen/Qwen3-TTS-12Hz-1.7B-Base` (~4GB)
- **0.6B** (faster): `Qwen/Qwen3-TTS-12Hz-0.6B-Base` (~2GB)
**Note:** The first generation will take longer as the model downloads. Subsequent generations will use the cached model.
#### Manual Download (Optional)
If you prefer to download models manually or have limited internet during runtime:
```bash
# Install huggingface-cli
pip install huggingface_hub
# Download 1.7B model
huggingface-cli download Qwen/Qwen3-TTS-12Hz-1.7B-Base
# Or use Python
python -c "from huggingface_hub import snapshot_download; snapshot_download('Qwen/Qwen3-TTS-12Hz-1.7B-Base')"
```
Models are cached in `~/.cache/huggingface/hub/` by default.
### 4. Run Server
```bash
# Development (local only)
python -m backend.main
# Production (allow remote access)
python -m backend.main --host 0.0.0.0 --port 8000
```
## Usage Examples
### Creating a Voice Profile
```bash
# 1. Create profile
curl -X POST http://localhost:8000/profiles \
-H "Content-Type: application/json" \
-d '{"text": "Hello world", "profile_id": "...", "language": "en"}'
-d '{"name": "My Voice", "language": "en"}'
# List profiles
curl http://localhost:17493/profiles
# Response: {"id": "abc-123", ...}
# Stream generation status (SSE)
curl http://localhost:17493/generate/{id}/status
# 2. Add sample
curl -X POST http://localhost:8000/profiles/abc-123/samples \
-F "[email protected]" \
-F "reference_text=This is my voice sample"
```
## Data directory
```
{data_dir}/
voicebox.db # SQLite database
profiles/{id}/ # Voice samples per profile
generations/ # Generated audio files
cache/ # Voice prompt cache (memory + disk)
backends/ # Downloaded CUDA binary (if applicable)
```
Default location is the OS-specific app data directory. Override with `--data-dir` or the `VOICEBOX_DATA_DIR` environment variable.
## Code quality
Linting and formatting are enforced by [ruff](https://docs.astral.sh/ruff/), configured in `pyproject.toml`. See `STYLE_GUIDE.md` for conventions.
### Generating Speech
```bash
just check-python # lint + format check
just fix-python # auto-fix lint issues + reformat
just test # run pytest
curl -X POST http://localhost:8000/generate \
-H "Content-Type: application/json" \
-d '{
"profile_id": "abc-123",
"text": "Hello, this is a test.",
"language": "en",
"seed": 42
}'
# Response: {"id": "gen-456", "audio_path": "/path/to/audio.wav", ...}
# Download audio
curl http://localhost:8000/audio/gen-456 -o output.wav
```
## Dependencies
### Transcribing Audio
Runtime dependencies are in `requirements.txt`. macOS-only MLX dependencies are in `requirements-mlx.txt`. Dev tools (ruff, pytest) are installed automatically by `just setup-python`.
```bash
curl -X POST http://localhost:8000/transcribe \
-F "[email protected]" \
-F "language=en"
# Response: {"text": "Transcribed text", "duration": 5.5}
```
## Advanced Features
### Multi-Sample Profiles
Add multiple samples to a profile for better quality:
```bash
# Add first sample
curl -X POST http://localhost:8000/profiles/abc-123/samples \
-F "[email protected]" \
-F "reference_text=First sample"
# Add second sample
curl -X POST http://localhost:8000/profiles/abc-123/samples \
-F "[email protected]" \
-F "reference_text=Second sample"
# Generation will automatically combine all samples
```
### Voice Prompt Caching
Voice prompts are automatically cached for faster generation:
- First generation: ~5-10 seconds (creates prompt)
- Subsequent generations: ~1-2 seconds (uses cached prompt)
Cache is stored in `data/cache/` and persists across server restarts.
### VRAM Management
Models are lazy-loaded and can be manually unloaded:
```bash
# Unload TTS model
curl -X POST http://localhost:8000/models/unload
# Load specific model size
curl -X POST "http://localhost:8000/models/load?model_size=0.6B"
```
## Error Handling
All endpoints return proper HTTP status codes:
- `200 OK`: Success
- `400 Bad Request`: Invalid input
- `404 Not Found`: Resource not found
- `500 Internal Server Error`: Server error
Error responses include details:
```json
{
"detail": "Profile not found"
}
```
## Performance Tips
1. **Use multi-sample profiles** - Better quality than single sample
2. **Let caching work** - Voice prompts are cached automatically
3. **Use 0.6B model on CPU** - Faster than 1.7B with acceptable quality
4. **Use 1.7B model on GPU** - Best quality, still fast
5. **Unload Whisper after transcription** - Frees VRAM for TTS
## TODO
- [ ] WebSocket support for generation progress
- [ ] Batch generation endpoint
- [ ] Audio effects (M3GAN, etc.)
- [ ] Voice design (text-to-voice)
- [ ] Audio studio timeline features
- [ ] Project management
- [ ] Authentication & rate limiting
- [ ] Export/import profiles
## License
See main project LICENSE.
-404
View File
@@ -1,404 +0,0 @@
# Python Style Guide
Target: **Python 3.12+** | Formatter/Linter: **Ruff** | Config: `backend/pyproject.toml`
This guide codifies the conventions used across the backend, and prescribes the target style for code written during the refactor (Phases 3-6). Existing code should be migrated incrementally -- don't reformat entire files in unrelated PRs.
---
## Formatting
Enforced by `ruff format` (Black-compatible).
- **Line length**: 120 characters.
- **Indent**: 4 spaces. No tabs.
- **Trailing commas**: Required on multi-line function signatures, arguments, collections.
- **Quotes**: Double quotes (`"`) for strings. Single quotes are acceptable in f-string expressions and dict keys inside f-strings where avoiding escapes improves readability.
Run: `ruff format backend/`
---
## Imports
Enforced by ruff's `isort` rules (rule set `I`).
**Grouping** -- three blocks separated by a blank line:
```python
import asyncio # 1. stdlib
from pathlib import Path
import numpy as np # 2. third-party
from fastapi import APIRouter, HTTPException
from sqlalchemy.orm import Session
from backend.config import get_data_dir # 3. local (absolute)
from .database import get_db # or relative
```
**Rules:**
- Within the `backend` package, use **relative imports** for sibling/child modules: `from .database import get_db`, `from ..utils.audio import load_audio`.
- Absolute imports are fine for top-level references from entry points (`main.py`, `server.py`).
- Never use wildcard imports (`from module import *`).
- One import per line for `from X import Y` when there are 4+ names; below that, comma-separated is fine.
- **Lazy imports** are acceptable for heavy dependencies (torch, transformers, mlx) inside functions to reduce startup time. Add a comment: `# lazy: heavy import`.
---
## Type Annotations
Python 3.12 means we use **built-in generics and union syntax natively**. No `from __future__ import annotations`, no `typing.List`/`typing.Dict`.
```python
# Yes
def process(items: list[str], config: dict[str, int] | None = None) -> tuple[int, str]: ...
# No
from typing import List, Dict, Optional, Tuple
def process(items: List[str], config: Optional[Dict[str, int]] = None) -> Tuple[int, str]: ...
```
**What to annotate:**
- All public function signatures (parameters + return type).
- Private functions: parameters at minimum; return type encouraged.
- Module-level variables: only when the type isn't obvious from the assignment.
- Route handlers: parameters are annotated via FastAPI's dependency injection. Add explicit `-> SomeResponse` return types when the route doesn't use `response_model`.
**Imports from `typing` that are still needed** (no built-in equivalent):
`Literal`, `TypeAlias`, `Protocol`, `runtime_checkable`, `Callable`, `Any`, `ClassVar`, `TypeVar`, `overload`, `TYPE_CHECKING`.
Use `collections.abc` for abstract types: `Sequence`, `Mapping`, `Iterable`, `Iterator`, `Generator`.
---
## Naming
| Thing | Convention | Example |
|-------|-----------|---------|
| Module | `snake_case` | `task_queue.py` |
| Class | `PascalCase` | `ProgressManager` |
| Function / method | `snake_case` | `create_profile` |
| Variable | `snake_case` | `sample_rate` |
| Constant | `UPPER_SNAKE_CASE` | `DEFAULT_SAMPLE_RATE` |
| Private | `_leading_underscore` | `_generation_queue` |
| Type alias | `PascalCase` | `EffectChain = list[dict[str, Any]]` |
**Specific conventions:**
- Database ORM models imported with `DB` prefix alias: `from .database import VoiceProfile as DBVoiceProfile`.
- Pydantic models use descriptive suffixes: `VoiceProfileCreate`, `VoiceProfileResponse`, `GenerationRequest`.
- Backend classes use engine-name prefix: `MLXTTSBackend`, `PyTorchSTTBackend`.
---
## Docstrings
**Google style**. Required on all public functions, classes, and modules.
```python
def combine_voice_prompts(
profile_dir: Path,
*,
target_sr: int = 24000,
) -> tuple[np.ndarray, int]:
"""Load and concatenate all voice prompt files for a profile.
Reads .wav/.mp3/.flac files from the profile directory, resamples to
the target sample rate, normalizes, and concatenates into a single array.
Args:
profile_dir: Path to the voice profile directory containing audio files.
target_sr: Target sample rate for the output. Defaults to 24000.
Returns:
Tuple of (concatenated audio array, sample rate).
Raises:
FileNotFoundError: If profile_dir does not exist.
ValueError: If no valid audio files are found.
"""
```
**Short form** is fine for simple functions:
```python
def get_db_path() -> Path:
"""Get the path to the SQLite database file."""
```
**When to skip**: Private helpers under ~5 lines where the name and signature make intent obvious.
**Module docstrings**: A single sentence at the top of every file describing its purpose.
```python
"""Voice profile CRUD operations."""
```
---
## Comments
Comments explain **why**, not **what**. If the code needs a comment to explain what it does, the code should be rewritten to be clearer. The exceptions are non-obvious performance choices, external constraints, and concurrency/race-condition reasoning -- those always deserve a comment.
### No section dividers
Do not use ASCII dividers to create visual sections in files:
```python
# No -- any of these:
# ============================================
# GENERATION ENDPOINTS
# ============================================
# ---------------------------------------------------------------------------
# Device detection
# ---------------------------------------------------------------------------
# --- Load model --------------------------------------------------
```
If a file needs section dividers to be navigable, the file is too long. Split it into modules. Within a function, if you need labeled sections to follow the logic, extract those sections into named functions.
### Inline comments
Inline comments (end-of-line) are fine when they add information the code can't express:
```python
# Yes -- explains a non-obvious constraint or gives context:
audio, sr = load_audio(path, sr=24000) # Qwen expects 24kHz mono
_generation_queue: asyncio.Queue = None # type: ignore # initialized at startup
"tauri://localhost", # Tauri webview (macOS)
# No -- restates the code:
# Check if profile name already exists
existing = db.query(DBVoiceProfile).filter_by(name=data.name).first()
# Delete from database
db.delete(sample)
# Update fields
profile.name = data.name
```
Delete comments that narrate what the next line of code obviously does. If the function name, variable name, or method call already communicates intent, the comment is noise.
### Block comments
Use block comments for **why** explanations -- constraints, workarounds, non-obvious decisions:
```python
# PyInstaller + multiprocessing: child processes re-execute the frozen binary
# with internal arguments. freeze_support() handles this and exits early.
multiprocessing.freeze_support()
# Mark any stale "generating" records as failed -- these are leftovers
# from a previous process that was killed mid-generation.
db.query(Generation).filter_by(status="generating").update({"status": "failed"})
```
Keep block comments tight. Two to three lines is normal. If you need a paragraph, it probably belongs in the docstring or a design doc.
### Linter/type-checker suppression
Always add a reason after `noqa` and `type: ignore`:
```python
import intel_extension_for_pytorch # noqa: F401 -- side-effect import enables XPU
_queue: asyncio.Queue = None # type: ignore[assignment] # initialized at startup
```
Bare `# noqa` or `# type: ignore` with no explanation are not allowed.
### TODO / FIXME
Use sparingly. Every `TODO` must include a brief description of what needs doing. Don't use them as a substitute for tracking work properly:
```python
# TODO: replace with async SQLAlchemy once CRUD modules are migrated (Phase 5)
result = await asyncio.to_thread(profiles.get_profile, profile_id, db)
```
Never commit `HACK`, `XXX`, or `FIXME` -- fix the problem or file an issue.
### Commented-out code
Delete it. That's what git is for. If you need to document that something was intentionally removed, a short tombstone comment is acceptable:
```python
# Removed config.json-only check -- too lenient, doesn't confirm weights exist.
```
---
## Error Handling
The refactor is standardizing on a **two-layer pattern**:
### 1. Domain layer -- raise plain exceptions
CRUD modules and services raise `ValueError`, `FileNotFoundError`, or (post-refactor) custom exceptions defined in `backend/errors.py`:
```python
# backend/errors.py (to be created in Phase 4)
class NotFoundError(Exception):
"""Raised when a requested resource does not exist."""
class ConflictError(Exception):
"""Raised on uniqueness constraint violations."""
```
```python
# In a service or CRUD module:
raise NotFoundError(f"Profile {profile_id} not found")
```
### 2. Route layer -- translate to HTTPException
Route handlers catch domain exceptions and convert:
```python
@router.post("/profiles")
async def create_profile(data: VoiceProfileCreate, db: Session = Depends(get_db)):
try:
return await profiles.create_profile(data, db)
except ConflictError as e:
raise HTTPException(status_code=409, detail=str(e))
```
**Background tasks** catch `Exception` broadly, log with `logger.exception()`, and update the task status to `"failed"`.
**Never**: silently swallow exceptions, use bare `except:`, or catch `BaseException`.
---
## Async
### Rules for the refactor
1. **Don't declare `async def` unless the function awaits something.** Several service modules still declare `async def` without awaiting -- these should be migrated to sync functions with `asyncio.to_thread()` at the route layer, or to real async SQLAlchemy.
2. **CPU-bound work** (audio processing, numpy operations) goes through `asyncio.to_thread()`:
```python
audio, sr = await asyncio.to_thread(load_audio, source_path)
```
3. **GPU-bound TTS inference** is serialized through the generation queue (`services/task_queue.py`). Never call a backend's `generate()` directly from a route handler.
4. **Fire-and-forget tasks**: use `asyncio.create_task()` and track the task reference to prevent garbage collection:
```python
task = asyncio.create_task(some_coro())
_background_tasks.add(task)
task.add_done_callback(_background_tasks.discard)
```
---
## Logging
Use the `logging` module. Not `print()`.
```python
import logging
logger = logging.getLogger(__name__)
logger.info("Loading model %s on %s", model_name, device)
logger.warning("Cache miss for %s, downloading", repo_id)
logger.exception("Generation %s failed") # logs traceback automatically
```
**Rules:**
- Use `%s`-style placeholders in log calls (not f-strings). This avoids formatting the string if the log level is filtered out.
- Use `logger.exception()` inside `except` blocks -- it captures the traceback.
- Logger name should be `__name__` (yields `backend.utils.audio`, etc.).
- Existing `print()` calls should be migrated to logging as files are touched during the refactor.
---
## Constants
- Define at **module level** in the file where they're primarily used.
- Use `UPPER_SNAKE_CASE`.
- Shared/cross-cutting constants (sample rates, file size limits, CORS origins) go in `backend/config.py` after Phase 6 consolidation.
- Magic numbers in function bodies should be extracted to named constants:
```python
# No
if len(audio) > 24000 * 60 * 10:
# Yes
MAX_AUDIO_DURATION_SAMPLES = SAMPLE_RATE * 60 * 10
if len(audio) > MAX_AUDIO_DURATION_SAMPLES:
```
---
## Function Signatures
- **Keyword-only arguments** (after `*`) for functions with 3+ parameters, especially when several share the same type:
```python
def is_model_cached(
hf_repo: str,
*,
weight_extensions: tuple[str, ...] = (".safetensors", ".bin"),
required_files: list[str] | None = None,
) -> bool:
```
- Parameters on **separate lines** when the signature exceeds ~100 characters or has 3+ params.
- **Trailing comma** after the last parameter in multi-line signatures.
- Default values inline with the parameter.
---
## String Formatting
- **f-strings** for runtime string construction.
- **`%s`-style** for `logging` calls (lazy evaluation).
- **`.format()`**: avoid; f-strings are preferred.
---
## Testing
Framework: **pytest** with `pytest-asyncio`.
- Test files: `test_<module>.py` in `backend/tests/`.
- Use `conftest.py` for shared fixtures (db sessions, test client, mock backends).
- Group related tests in classes: `class TestProfileCRUD:`.
- Use `@pytest.mark.asyncio` for async tests.
- Use `@pytest.mark.parametrize` to reduce repetition.
- Manual integration scripts stay in `tests/` but are clearly marked (filename prefix `manual_` or documented in `tests/README.md`).
---
## Project Layout
```
backend/
app.py # FastAPI app factory, CORS, lifecycle events
main.py # Entry point (imports app, runs uvicorn)
config.py # Data directory paths
models.py # Pydantic request/response schemas
server.py # Tauri sidecar launcher, parent-pid watchdog
routes/ # Thin HTTP handlers (validation, delegation, response formatting)
services/ # Business logic, CRUD, orchestration
backends/ # TTS/STT engine implementations
database/ # ORM models, session management, migrations, seeds
utils/ # Shared utilities (audio, effects, caching, progress)
tests/ # pytest suite
```
---
## Ruff Adoption
`pyproject.toml` configures ruff for linting and formatting. Run:
```bash
# Lint (check)
ruff check backend/
# Lint (auto-fix)
ruff check backend/ --fix
# Format
ruff format backend/
```
Introduce ruff fixes file-by-file as you touch them. Don't run `--fix` across the entire codebase in one shot -- that creates unreviewable diffs.
+1 -1
View File
@@ -1,3 +1,3 @@
# Backend package
__version__ = "0.2.3"
__version__ = "0.1.13"
-215
View File
@@ -1,215 +0,0 @@
"""FastAPI application factory, middleware, and lifecycle events."""
import asyncio
import logging
import os
import sys
from pathlib import Path
class ColoredFormatter(logging.Formatter):
"""Custom formatter to add colors matching uvicorn's style."""
COLORS = {
"DEBUG": "\033[36m", # Cyan
"INFO": "\033[32m", # Green
"WARNING": "\033[33m", # Yellow
"ERROR": "\033[31m", # Red
"CRITICAL": "\033[35m", # Magenta
}
RESET = "\033[0m"
def format(self, record):
log_color = self.COLORS.get(record.levelname, self.RESET)
record.levelname = f"{log_color}{record.levelname}{self.RESET}"
return super().format(record)
# Configure logging to match uvicorn's format with colors
handler = logging.StreamHandler(sys.stderr)
handler.setFormatter(ColoredFormatter("%(levelname)s: %(message)s"))
logging.basicConfig(
level=logging.INFO,
handlers=[handler],
)
logger = logging.getLogger(__name__)
# AMD GPU environment variables must be set before torch import
if not os.environ.get("HSA_OVERRIDE_GFX_VERSION"):
os.environ["HSA_OVERRIDE_GFX_VERSION"] = "10.3.0"
if not os.environ.get("MIOPEN_LOG_LEVEL"):
os.environ["MIOPEN_LOG_LEVEL"] = "4"
import torch
from fastapi import FastAPI
from fastapi.middleware.cors import CORSMiddleware
from urllib.parse import quote
from . import __version__, config, database
from .services import tts, transcribe
from .database import get_db
from .utils.platform_detect import get_backend_type
from .utils.progress import get_progress_manager
from .services.task_queue import create_background_task, init_queue
from .routes import register_routers
def safe_content_disposition(disposition_type: str, filename: str) -> str:
"""Build a Content-Disposition header safe for non-ASCII filenames.
Uses RFC 5987 ``filename*`` parameter so browsers can decode UTF-8
filenames while the ``filename`` fallback stays ASCII-only.
"""
ascii_name = "".join(c for c in filename if c.isascii() and (c.isalnum() or c in " -_.")).strip() or "download"
utf8_name = quote(filename, safe="")
return f"{disposition_type}; filename=\"{ascii_name}\"; filename*=UTF-8''{utf8_name}"
def create_app() -> FastAPI:
"""Create and configure the FastAPI application."""
application = FastAPI(
title="voicebox API",
description="Production-quality Qwen3-TTS voice cloning API",
version=__version__,
)
_configure_cors(application)
register_routers(application)
_register_lifecycle(application)
return application
def _configure_cors(application: FastAPI) -> None:
"""Set up CORS middleware with local-first defaults."""
default_origins = [
"http://localhost:5173", # Vite dev server
"http://127.0.0.1:5173",
"http://localhost:17493",
"http://127.0.0.1:17493",
"tauri://localhost", # Tauri webview (macOS)
"https://tauri.localhost", # Tauri webview (Windows/Linux)
"http://tauri.localhost", # Tauri webview (Windows, some builds)
]
env_origins = os.environ.get("VOICEBOX_CORS_ORIGINS", "")
all_origins = default_origins + [o.strip() for o in env_origins.split(",") if o.strip()]
application.add_middleware(
CORSMiddleware,
allow_origins=all_origins,
allow_credentials=True,
allow_methods=["*"],
allow_headers=["*"],
)
def _get_gpu_status() -> str:
"""Return a human-readable string describing GPU availability."""
backend_type = get_backend_type()
if torch.cuda.is_available():
device_name = torch.cuda.get_device_name(0)
is_rocm = hasattr(torch.version, "hip") and torch.version.hip is not None
if is_rocm:
return f"ROCm ({device_name})"
return f"CUDA ({device_name})"
elif hasattr(torch.backends, "mps") and torch.backends.mps.is_available():
return "MPS (Apple Silicon)"
elif backend_type == "mlx":
return "Metal (Apple Silicon via MLX)"
return "None (CPU only)"
def _register_lifecycle(application: FastAPI) -> None:
"""Attach startup and shutdown event handlers."""
@application.on_event("startup")
async def startup_event():
import platform
import sys
logger.info("Voicebox v%s starting up", __version__)
logger.info(
"Python %s on %s %s (%s)",
sys.version.split()[0],
platform.system(),
platform.release(),
platform.machine(),
)
database.init_db()
from .database.session import _db_path
logger.info("Database: %s", _db_path)
logger.info("Data directory: %s", config.get_data_dir())
init_queue()
# Mark stale "generating" records as failed -- leftovers from a killed process
from sqlalchemy import text as sa_text
db = next(get_db())
try:
result = db.execute(
sa_text(
"UPDATE generations SET status = 'failed', "
"error = 'Server was shut down during generation' "
"WHERE status IN ('generating', 'loading_model')"
)
)
if result.rowcount > 0:
logger.info("Marked %d stale generation(s) as failed", result.rowcount)
from .database import VoiceProfile as DBVoiceProfile, Generation as DBGeneration
profile_count = db.query(DBVoiceProfile).count()
generation_count = db.query(DBGeneration).count()
logger.info("Profiles: %d, Generations: %d", profile_count, generation_count)
db.commit()
except Exception as e:
db.rollback()
logger.warning("Could not clean up stale generations: %s", e)
finally:
db.close()
backend_type = get_backend_type()
logger.info("Backend: %s", backend_type.upper())
logger.info("GPU: %s", _get_gpu_status())
from .services.cuda import check_and_update_cuda_binary
create_background_task(check_and_update_cuda_binary())
try:
progress_manager = get_progress_manager()
progress_manager._set_main_loop(asyncio.get_running_loop())
except Exception as e:
logger.warning("Could not initialize progress manager event loop: %s", e)
try:
from huggingface_hub import constants as hf_constants
cache_dir = Path(hf_constants.HF_HUB_CACHE)
cache_dir.mkdir(parents=True, exist_ok=True)
logger.info("Model cache: %s", cache_dir)
except Exception as e:
logger.warning("Could not create HuggingFace cache directory: %s", e)
logger.info("Ready")
@application.on_event("shutdown")
async def shutdown_event():
logger.info("Voicebox server shutting down...")
try:
tts.unload_tts_model()
except Exception:
logger.exception("Failed to unload TTS model")
try:
transcribe.unload_whisper_model()
except Exception:
logger.exception("Failed to unload Whisper model")
app = create_app()
+47 -394
View File
@@ -1,66 +1,24 @@
"""
Backend abstraction layer for TTS and STT.
Provides a unified interface for MLX and PyTorch backends,
and a model config registry that eliminates per-engine dispatch maps.
Provides a unified interface for MLX and PyTorch backends.
"""
import threading
from dataclasses import dataclass, field
from typing import Protocol, Optional, Tuple, List
from typing_extensions import runtime_checkable
import numpy as np
from ..utils.platform_detect import get_backend_type
LANGUAGE_CODE_TO_NAME = {
"zh": "chinese",
"en": "english",
"ja": "japanese",
"ko": "korean",
"de": "german",
"fr": "french",
"ru": "russian",
"pt": "portuguese",
"es": "spanish",
"it": "italian",
}
WHISPER_HF_REPOS = {
"base": "openai/whisper-base",
"small": "openai/whisper-small",
"medium": "openai/whisper-medium",
"large": "openai/whisper-large-v3",
"turbo": "openai/whisper-large-v3-turbo",
}
@dataclass
class ModelConfig:
"""Declarative config for a downloadable model variant."""
model_name: str # e.g. "luxtts", "chatterbox-tts"
display_name: str # e.g. "LuxTTS (Fast, CPU-friendly)"
engine: str # e.g. "luxtts", "chatterbox"
hf_repo_id: str # e.g. "YatharthS/LuxTTS"
model_size: str = "default"
size_mb: int = 0
needs_trim: bool = False
supports_instruct: bool = False
languages: list[str] = field(default_factory=lambda: ["en"])
from ..platform_detect import get_backend_type
@runtime_checkable
class TTSBackend(Protocol):
"""Protocol for TTS backend implementations."""
# Each backend class should define MODEL_CONFIGS as a class variable:
# MODEL_CONFIGS: list[ModelConfig]
async def load_model(self, model_size: str) -> None:
"""Load TTS model."""
...
async def create_voice_prompt(
self,
audio_path: str,
@@ -69,12 +27,12 @@ class TTSBackend(Protocol):
) -> Tuple[dict, bool]:
"""
Create voice prompt from reference audio.
Returns:
Tuple of (voice_prompt_dict, was_cached)
"""
...
async def combine_voice_prompts(
self,
audio_paths: List[str],
@@ -82,12 +40,12 @@ class TTSBackend(Protocol):
) -> Tuple[np.ndarray, str]:
"""
Combine multiple voice prompts.
Returns:
Tuple of (combined_audio_array, combined_text)
"""
...
async def generate(
self,
text: str,
@@ -98,24 +56,24 @@ class TTSBackend(Protocol):
) -> Tuple[np.ndarray, int]:
"""
Generate audio from text.
Returns:
Tuple of (audio_array, sample_rate)
"""
...
def unload_model(self) -> None:
"""Unload model to free memory."""
...
def is_loaded(self) -> bool:
"""Check if model is loaded."""
...
def _get_model_path(self, model_size: str) -> str:
"""
Get model path for a given size.
Returns:
Model path or HuggingFace Hub ID
"""
@@ -125,11 +83,11 @@ class TTSBackend(Protocol):
@runtime_checkable
class STTBackend(Protocol):
"""Protocol for STT (Speech-to-Text) backend implementations."""
async def load_model(self, model_size: str) -> None:
"""Load STT model."""
...
async def transcribe(
self,
audio_path: str,
@@ -137,16 +95,16 @@ class STTBackend(Protocol):
) -> str:
"""
Transcribe audio to text.
Returns:
Transcribed text
"""
...
def unload_model(self) -> None:
"""Unload model to free memory."""
...
def is_loaded(self) -> bool:
"""Check if model is loaded."""
...
@@ -154,375 +112,70 @@ class STTBackend(Protocol):
# Global backend instances
_tts_backend: Optional[TTSBackend] = None
_tts_backends: dict[str, TTSBackend] = {}
_tts_backends_lock = threading.Lock()
_stt_backend: Optional[STTBackend] = None
# Supported TTS engines — keyed by engine name, value is the backend class import path.
# The factory function uses this for the if/elif chain; the model configs live on the backend classes.
TTS_ENGINES = {
"qwen": "Qwen TTS",
"luxtts": "LuxTTS",
"chatterbox": "Chatterbox TTS",
"chatterbox_turbo": "Chatterbox Turbo",
}
def _get_qwen_model_configs() -> list[ModelConfig]:
"""Return Qwen model configs with backend-aware HF repo IDs."""
backend_type = get_backend_type()
if backend_type == "mlx":
repo_1_7b = "mlx-community/Qwen3-TTS-12Hz-1.7B-Base-bf16"
repo_0_6b = "mlx-community/Qwen3-TTS-12Hz-1.7B-Base-bf16" # 0.6B not available in MLX, falls back
else:
repo_1_7b = "Qwen/Qwen3-TTS-12Hz-1.7B-Base"
repo_0_6b = "Qwen/Qwen3-TTS-12Hz-0.6B-Base"
return [
ModelConfig(
model_name="qwen-tts-1.7B",
display_name="Qwen TTS 1.7B",
engine="qwen",
hf_repo_id=repo_1_7b,
model_size="1.7B",
size_mb=3500,
supports_instruct=False, # Base model drops instruct silently
languages=["zh", "en", "ja", "ko", "de", "fr", "ru", "pt", "es", "it"],
),
ModelConfig(
model_name="qwen-tts-0.6B",
display_name="Qwen TTS 0.6B",
engine="qwen",
hf_repo_id=repo_0_6b,
model_size="0.6B",
size_mb=1200,
supports_instruct=False,
languages=["zh", "en", "ja", "ko", "de", "fr", "ru", "pt", "es", "it"],
),
]
def _get_non_qwen_tts_configs() -> list[ModelConfig]:
"""Return model configs for non-Qwen TTS engines.
These are static — no backend-type branching needed.
"""
return [
ModelConfig(
model_name="luxtts",
display_name="LuxTTS (Fast, CPU-friendly)",
engine="luxtts",
hf_repo_id="YatharthS/LuxTTS",
size_mb=300,
languages=["en"],
),
ModelConfig(
model_name="chatterbox-tts",
display_name="Chatterbox TTS (Multilingual)",
engine="chatterbox",
hf_repo_id="ResembleAI/chatterbox",
size_mb=3200,
needs_trim=True,
languages=[
"zh",
"en",
"ja",
"ko",
"de",
"fr",
"ru",
"pt",
"es",
"it",
"he",
"ar",
"da",
"el",
"fi",
"hi",
"ms",
"nl",
"no",
"pl",
"sv",
"sw",
"tr",
],
),
ModelConfig(
model_name="chatterbox-turbo",
display_name="Chatterbox Turbo (English, Tags)",
engine="chatterbox_turbo",
hf_repo_id="ResembleAI/chatterbox-turbo",
size_mb=1500,
needs_trim=True,
languages=["en"],
),
]
def _get_whisper_configs() -> list[ModelConfig]:
"""Return Whisper STT model configs."""
return [
ModelConfig(
model_name="whisper-base",
display_name="Whisper Base",
engine="whisper",
hf_repo_id="openai/whisper-base",
model_size="base",
),
ModelConfig(
model_name="whisper-small",
display_name="Whisper Small",
engine="whisper",
hf_repo_id="openai/whisper-small",
model_size="small",
),
ModelConfig(
model_name="whisper-medium",
display_name="Whisper Medium",
engine="whisper",
hf_repo_id="openai/whisper-medium",
model_size="medium",
),
ModelConfig(
model_name="whisper-large",
display_name="Whisper Large",
engine="whisper",
hf_repo_id="openai/whisper-large-v3",
model_size="large",
),
ModelConfig(
model_name="whisper-turbo",
display_name="Whisper Turbo",
engine="whisper",
hf_repo_id="openai/whisper-large-v3-turbo",
model_size="turbo",
),
]
def get_all_model_configs() -> list[ModelConfig]:
"""Return the full list of model configs (TTS + STT)."""
return _get_qwen_model_configs() + _get_non_qwen_tts_configs() + _get_whisper_configs()
def get_tts_model_configs() -> list[ModelConfig]:
"""Return only TTS model configs."""
return _get_qwen_model_configs() + _get_non_qwen_tts_configs()
# Lookup helpers — these replace the if/elif chains in main.py
def get_model_config(model_name: str) -> Optional[ModelConfig]:
"""Look up a model config by model_name."""
for cfg in get_all_model_configs():
if cfg.model_name == model_name:
return cfg
return None
def engine_needs_trim(engine: str) -> bool:
"""Whether this engine's output should be run through trim_tts_output."""
for cfg in get_tts_model_configs():
if cfg.engine == engine:
return cfg.needs_trim
return False
def engine_has_model_sizes(engine: str) -> bool:
"""Whether this engine supports multiple model sizes (only Qwen currently)."""
configs = [c for c in get_tts_model_configs() if c.engine == engine]
return len(configs) > 1
async def load_engine_model(engine: str, model_size: str = "default") -> None:
"""Load a model for the given engine, handling the Qwen model_size special case."""
backend = get_tts_backend_for_engine(engine)
if engine == "qwen":
await backend.load_model_async(model_size)
else:
await backend.load_model()
async def ensure_model_cached_or_raise(engine: str, model_size: str = "default") -> None:
"""Check if a model is cached, raise HTTPException if not. Used by streaming endpoint."""
from fastapi import HTTPException
backend = get_tts_backend_for_engine(engine)
cfg = None
for c in get_tts_model_configs():
if c.engine == engine and c.model_size == model_size:
cfg = c
break
if engine == "qwen":
if not backend._is_model_cached(model_size):
raise HTTPException(
status_code=400,
detail=f"Model {model_size} is not downloaded yet. Use /generate to trigger a download.",
)
else:
if not backend._is_model_cached():
display = cfg.display_name if cfg else engine
raise HTTPException(
status_code=400,
detail=f"{display} model is not downloaded yet. Use /generate to trigger a download.",
)
def unload_model_by_config(config: ModelConfig) -> bool:
"""Unload a model given its config. Returns True if it was loaded, False otherwise."""
from . import get_tts_backend_for_engine
from ..services import tts, transcribe
if config.engine == "whisper":
whisper_model = transcribe.get_whisper_model()
if whisper_model.is_loaded() and whisper_model.model_size == config.model_size:
transcribe.unload_whisper_model()
return True
return False
if config.engine == "qwen":
tts_model = tts.get_tts_model()
loaded_size = getattr(tts_model, "_current_model_size", None) or getattr(tts_model, "model_size", None)
if tts_model.is_loaded() and loaded_size == config.model_size:
tts.unload_tts_model()
return True
return False
# All other TTS engines
backend = get_tts_backend_for_engine(config.engine)
if backend.is_loaded():
backend.unload_model()
return True
return False
def check_model_loaded(config: ModelConfig) -> bool:
"""Check if a model is currently loaded."""
from . import get_tts_backend_for_engine
from ..services import tts, transcribe
try:
if config.engine == "whisper":
whisper_model = transcribe.get_whisper_model()
return whisper_model.is_loaded() and getattr(whisper_model, "model_size", None) == config.model_size
if config.engine == "qwen":
tts_model = tts.get_tts_model()
loaded_size = getattr(tts_model, "_current_model_size", None) or getattr(tts_model, "model_size", None)
return tts_model.is_loaded() and loaded_size == config.model_size
backend = get_tts_backend_for_engine(config.engine)
return backend.is_loaded()
except Exception:
return False
def get_model_load_func(config: ModelConfig):
"""Return a callable that loads/downloads the model."""
from . import get_tts_backend_for_engine
from ..services import tts, transcribe
if config.engine == "whisper":
return lambda: transcribe.get_whisper_model().load_model(config.model_size)
if config.engine == "qwen":
return lambda: tts.get_tts_model().load_model(config.model_size)
return lambda: get_tts_backend_for_engine(config.engine).load_model()
def get_tts_backend() -> TTSBackend:
"""
Get or create the default (Qwen) TTS backend instance based on platform.
Get or create TTS backend instance based on platform.
Returns:
TTS backend instance (MLX or PyTorch)
Raises:
ImportError: If required dependencies (mlx or torch) are not available
"""
return get_tts_backend_for_engine("qwen")
global _tts_backend
if _tts_backend is None:
backend_type = get_backend_type()
def get_tts_backend_for_engine(engine: str) -> TTSBackend:
"""
Get or create a TTS backend for the given engine.
Args:
engine: Engine name (e.g. "qwen", "luxtts", "chatterbox", "chatterbox_turbo")
Returns:
TTS backend instance
"""
global _tts_backends
# Fast path: check without lock
if engine in _tts_backends:
return _tts_backends[engine]
# Slow path: create with lock to avoid duplicate instantiation
with _tts_backends_lock:
# Double-check after acquiring lock
if engine in _tts_backends:
return _tts_backends[engine]
if engine == "qwen":
backend_type = get_backend_type()
if backend_type == "mlx":
if backend_type == "mlx":
try:
from .mlx_backend import MLXTTSBackend
backend = MLXTTSBackend()
else:
from .pytorch_backend import PyTorchTTSBackend
backend = PyTorchTTSBackend()
elif engine == "luxtts":
from .luxtts_backend import LuxTTSBackend
backend = LuxTTSBackend()
elif engine == "chatterbox":
from .chatterbox_backend import ChatterboxTTSBackend
backend = ChatterboxTTSBackend()
elif engine == "chatterbox_turbo":
from .chatterbox_turbo_backend import ChatterboxTurboTTSBackend
backend = ChatterboxTurboTTSBackend()
_tts_backend = MLXTTSBackend()
except ImportError as e:
raise ImportError(
f"MLX backend dependencies not available. "
f"Please install mlx and mlx_audio or download a provider. Error: {e}"
)
else:
raise ValueError(f"Unknown TTS engine: {engine}. Supported: {list(TTS_ENGINES.keys())}")
try:
from .pytorch_backend import PyTorchTTSBackend
_tts_backend = PyTorchTTSBackend()
except ImportError as e:
raise ImportError(
f"PyTorch backend dependencies not available. "
f"Please download a TTS provider (pytorch-cpu or pytorch-cuda) from the Downloads page. Error: {e}"
)
_tts_backends[engine] = backend
return backend
return _tts_backend
def get_stt_backend() -> STTBackend:
"""
Get or create STT backend instance based on platform.
Returns:
STT backend instance (MLX or PyTorch)
"""
global _stt_backend
if _stt_backend is None:
backend_type = get_backend_type()
if backend_type == "mlx":
from .mlx_backend import MLXSTTBackend
_stt_backend = MLXSTTBackend()
else:
from .pytorch_backend import PyTorchSTTBackend
_stt_backend = PyTorchSTTBackend()
return _stt_backend
def reset_backends():
"""Reset backend instances (useful for testing)."""
global _tts_backend, _tts_backends, _stt_backend
global _tts_backend, _stt_backend
_tts_backend = None
_tts_backends.clear()
_stt_backend = None
-258
View File
@@ -1,258 +0,0 @@
"""
Shared utilities for TTS/STT backend implementations.
Eliminates duplication of cache checking, device detection,
voice prompt combination, and model loading progress tracking.
"""
import logging
import platform
from contextlib import contextmanager
from pathlib import Path
from typing import Callable, List, Optional, Tuple
import numpy as np
from ..utils.audio import normalize_audio, load_audio
from ..utils.progress import get_progress_manager
from ..utils.hf_progress import HFProgressTracker, create_hf_progress_callback
from ..utils.tasks import get_task_manager
logger = logging.getLogger(__name__)
def is_model_cached(
hf_repo: str,
*,
weight_extensions: tuple[str, ...] = (".safetensors", ".bin"),
required_files: Optional[list[str]] = None,
) -> bool:
"""
Check if a HuggingFace model is fully cached locally.
Args:
hf_repo: HuggingFace repo ID (e.g. "Qwen/Qwen3-TTS-12Hz-1.7B-Base")
weight_extensions: File extensions that count as model weights.
required_files: If set, check that these specific filenames exist
in snapshots instead of checking by extension.
Returns:
True if model is fully cached, False if missing or incomplete.
"""
try:
from huggingface_hub import constants as hf_constants
repo_cache = Path(hf_constants.HF_HUB_CACHE) / ("models--" + hf_repo.replace("/", "--"))
if not repo_cache.exists():
return False
# Incomplete blobs mean a download is still in progress
blobs_dir = repo_cache / "blobs"
if blobs_dir.exists() and any(blobs_dir.glob("*.incomplete")):
logger.debug(f"Found .incomplete files for {hf_repo}")
return False
snapshots_dir = repo_cache / "snapshots"
if not snapshots_dir.exists():
return False
if required_files:
# Check that every required filename exists somewhere in snapshots
for fname in required_files:
if not any(snapshots_dir.rglob(fname)):
return False
return True
# Check that at least one weight file exists
for ext in weight_extensions:
if any(snapshots_dir.rglob(f"*{ext}")):
return True
logger.debug(f"No model weights found for {hf_repo}")
return False
except Exception as e:
logger.warning(f"Error checking cache for {hf_repo}: {e}")
return False
def get_torch_device(
*,
allow_xpu: bool = False,
allow_directml: bool = False,
allow_mps: bool = False,
force_cpu_on_mac: bool = False,
) -> str:
"""
Detect the best available torch device.
Args:
allow_xpu: Check for Intel XPU (IPEX) support.
allow_directml: Check for DirectML (Windows) support.
allow_mps: Allow MPS (Apple Silicon). If False, MPS falls back to CPU.
force_cpu_on_mac: Force CPU on macOS regardless of GPU availability.
"""
if force_cpu_on_mac and platform.system() == "Darwin":
return "cpu"
import torch
if torch.cuda.is_available():
return "cuda"
if allow_xpu:
try:
import intel_extension_for_pytorch # noqa: F401
if hasattr(torch, "xpu") and torch.xpu.is_available():
return "xpu"
except ImportError:
pass
if allow_directml:
try:
import torch_directml
if torch_directml.device_count() > 0:
return torch_directml.device(0)
except ImportError:
pass
if allow_mps:
if hasattr(torch.backends, "mps") and torch.backends.mps.is_available():
return "mps"
return "cpu"
async def combine_voice_prompts(
audio_paths: List[str],
reference_texts: List[str],
*,
sample_rate: Optional[int] = None,
) -> Tuple[np.ndarray, str]:
"""
Combine multiple reference audio samples into one.
Loads each audio file, normalizes, concatenates, and joins texts.
Args:
audio_paths: Paths to reference audio files.
reference_texts: Corresponding transcripts.
sample_rate: If set, resample audio to this rate during loading.
"""
combined_audio = []
for path in audio_paths:
kwargs = {"sample_rate": sample_rate} if sample_rate else {}
audio, _sr = load_audio(path, **kwargs)
audio = normalize_audio(audio)
combined_audio.append(audio)
mixed = np.concatenate(combined_audio)
mixed = normalize_audio(mixed)
combined_text = " ".join(reference_texts)
return mixed, combined_text
@contextmanager
def model_load_progress(
model_name: str,
is_cached: bool,
filter_non_downloads: Optional[bool] = None,
):
"""
Context manager for model loading with HF download progress tracking.
Handles the tqdm patching, progress_manager/task_manager lifecycle,
and error reporting that every backend duplicates.
Args:
model_name: Progress tracking key (e.g. "qwen-tts-1.7B", "whisper-base").
is_cached: Whether the model is already downloaded.
filter_non_downloads: Whether to filter non-download tqdm bars.
Defaults to `is_cached`.
Yields:
The tracker context (already entered). The caller loads the model
inside the `with` block. The tqdm patch is torn down on exit.
Usage:
with model_load_progress("qwen-tts-1.7B", is_cached) as ctx:
self.model = SomeModel.from_pretrained(...)
"""
if filter_non_downloads is None:
filter_non_downloads = is_cached
progress_manager = get_progress_manager()
task_manager = get_task_manager()
progress_callback = create_hf_progress_callback(model_name, progress_manager)
tracker = HFProgressTracker(progress_callback, filter_non_downloads=filter_non_downloads)
tracker_context = tracker.patch_download()
tracker_context.__enter__()
if not is_cached:
task_manager.start_download(model_name)
progress_manager.update_progress(
model_name=model_name,
current=0,
total=0,
filename="Connecting to HuggingFace...",
status="downloading",
)
try:
yield tracker_context
except Exception as e:
# Report error to both managers
progress_manager.mark_error(model_name, str(e))
task_manager.error_download(model_name, str(e))
raise
else:
# Only mark complete if we were tracking a download
if not is_cached:
progress_manager.mark_complete(model_name)
task_manager.complete_download(model_name)
finally:
tracker_context.__exit__(None, None, None)
def patch_chatterbox_f32(model) -> None:
"""
Patch float64 -> float32 dtype mismatches in upstream chatterbox.
librosa.load returns float64 numpy arrays. Multiple upstream code paths
convert these to torch tensors via torch.from_numpy() without casting,
then matmul against float32 model weights. This patches the two known
entry points:
1. S3Tokenizer.log_mel_spectrogram — audio tensor hits _mel_filters (f32)
2. VoiceEncoder.forward — float64 mel spectrograms hit LSTM weights (f32)
"""
import types
# Patch S3Tokenizer
_tokzr = model.s3gen.tokenizer
_orig_log_mel = _tokzr.log_mel_spectrogram.__func__
def _f32_log_mel(self_tokzr, audio, padding=0):
import torch as _torch
if _torch.is_tensor(audio):
audio = audio.float()
return _orig_log_mel(self_tokzr, audio, padding)
_tokzr.log_mel_spectrogram = types.MethodType(_f32_log_mel, _tokzr)
# Patch VoiceEncoder
_ve = model.ve
_orig_ve_forward = _ve.forward.__func__
def _f32_ve_forward(self_ve, mels):
return _orig_ve_forward(self_ve, mels.float())
_ve.forward = types.MethodType(_f32_ve_forward, _ve)
-230
View File
@@ -1,230 +0,0 @@
"""
Chatterbox TTS backend implementation.
Wraps ChatterboxMultilingualTTS from chatterbox-tts for zero-shot
voice cloning. Supports 23 languages including Hebrew. Forces CPU
on macOS due to known MPS tensor issues.
"""
import asyncio
import logging
import threading
from pathlib import Path
from typing import ClassVar, List, Optional, Tuple
import numpy as np
from . import TTSBackend
from .base import (
is_model_cached,
get_torch_device,
combine_voice_prompts as _combine_voice_prompts,
model_load_progress,
patch_chatterbox_f32,
)
logger = logging.getLogger(__name__)
CHATTERBOX_HF_REPO = "ResembleAI/chatterbox"
# Files that must be present for the multilingual model
_MTL_WEIGHT_FILES = [
"t3_mtl23ls_v2.safetensors",
"s3gen.pt",
"ve.pt",
]
class ChatterboxTTSBackend:
"""Chatterbox Multilingual TTS backend for voice cloning."""
# Class-level lock for torch.load monkey-patching
_load_lock: ClassVar[threading.Lock] = threading.Lock()
def __init__(self):
self.model = None
self.model_size = "default"
self._device = None
self._model_load_lock = asyncio.Lock()
def _get_device(self) -> str:
return get_torch_device(force_cpu_on_mac=True)
def is_loaded(self) -> bool:
return self.model is not None
def _get_model_path(self, model_size: str = "default") -> str:
return CHATTERBOX_HF_REPO
def _is_model_cached(self, model_size: str = "default") -> bool:
return is_model_cached(CHATTERBOX_HF_REPO, required_files=_MTL_WEIGHT_FILES)
async def load_model(self, model_size: str = "default") -> None:
"""Load the Chatterbox multilingual model."""
if self.model is not None:
return
async with self._model_load_lock:
if self.model is not None:
return
await asyncio.to_thread(self._load_model_sync)
def _load_model_sync(self):
"""Synchronous model loading."""
model_name = "chatterbox-tts"
is_cached = self._is_model_cached()
with model_load_progress(model_name, is_cached):
device = self._get_device()
self._device = device
logger.info(f"Loading Chatterbox Multilingual TTS on {device}...")
import torch
from chatterbox.mtl_tts import ChatterboxMultilingualTTS
if device == "cpu":
_orig_torch_load = torch.load
def _patched_load(*args, **kwargs):
kwargs.setdefault("map_location", "cpu")
return _orig_torch_load(*args, **kwargs)
with ChatterboxTTSBackend._load_lock:
torch.load = _patched_load
try:
model = ChatterboxMultilingualTTS.from_pretrained(device=device)
finally:
torch.load = _orig_torch_load
else:
model = ChatterboxMultilingualTTS.from_pretrained(device=device)
# Fix sdpa attention for output_attentions support
t3_tfmr = model.t3.tfmr
if hasattr(t3_tfmr, "config") and hasattr(t3_tfmr.config, "_attn_implementation"):
t3_tfmr.config._attn_implementation = "eager"
for layer in getattr(t3_tfmr, "layers", []):
if hasattr(layer, "self_attn"):
layer.self_attn._attn_implementation = "eager"
patch_chatterbox_f32(model)
self.model = model
logger.info("Chatterbox Multilingual TTS loaded successfully")
def unload_model(self) -> None:
"""Unload model to free memory."""
if self.model is not None:
device = self._device
del self.model
self.model = None
self._device = None
if device == "cuda":
import torch
torch.cuda.empty_cache()
logger.info("Chatterbox unloaded")
async def create_voice_prompt(
self,
audio_path: str,
reference_text: str,
use_cache: bool = True,
) -> Tuple[dict, bool]:
"""
Create voice prompt from reference audio.
Chatterbox processes reference audio at generation time, so the
prompt just stores the file path. The actual audio is loaded by
model.generate() via audio_prompt_path.
"""
voice_prompt = {
"ref_audio": str(audio_path),
"ref_text": reference_text,
}
return voice_prompt, False
async def combine_voice_prompts(
self,
audio_paths: List[str],
reference_texts: List[str],
) -> Tuple[np.ndarray, str]:
return await _combine_voice_prompts(audio_paths, reference_texts)
# Per-language generation defaults. Lower temp + higher cfg = clearer speech.
_LANG_DEFAULTS: ClassVar[dict] = {
"he": {
"exaggeration": 0.4,
"cfg_weight": 0.7,
"temperature": 0.65,
"repetition_penalty": 2.5,
},
}
_GLOBAL_DEFAULTS: ClassVar[dict] = {
"exaggeration": 0.5,
"cfg_weight": 0.5,
"temperature": 0.8,
"repetition_penalty": 2.0,
}
async def generate(
self,
text: str,
voice_prompt: dict,
language: str = "en",
seed: Optional[int] = None,
instruct: Optional[str] = None,
) -> Tuple[np.ndarray, int]:
"""
Generate audio using Chatterbox Multilingual TTS.
Args:
text: Text to synthesize
voice_prompt: Dict with ref_audio path
language: BCP-47 language code
seed: Random seed for reproducibility
instruct: Unused (protocol compatibility)
Returns:
Tuple of (audio_array, sample_rate)
"""
await self.load_model()
ref_audio = voice_prompt.get("ref_audio")
if ref_audio and not Path(ref_audio).exists():
logger.warning(f"Reference audio not found: {ref_audio}")
ref_audio = None
# Merge language-specific defaults with global defaults
lang_defaults = self._LANG_DEFAULTS.get(language, self._GLOBAL_DEFAULTS)
def _generate_sync():
import torch
if seed is not None:
torch.manual_seed(seed)
logger.info(f"[Chatterbox] Generating: lang={language}")
wav = self.model.generate(
text,
language_id=language,
audio_prompt_path=ref_audio,
exaggeration=lang_defaults["exaggeration"],
cfg_weight=lang_defaults["cfg_weight"],
temperature=lang_defaults["temperature"],
repetition_penalty=lang_defaults["repetition_penalty"],
)
# Convert tensor -> numpy
if isinstance(wav, torch.Tensor):
audio = wav.squeeze().cpu().numpy().astype(np.float32)
else:
audio = np.asarray(wav, dtype=np.float32)
sample_rate = (
getattr(self.model, "sr", None)
or getattr(self.model, "sample_rate", 24000)
)
return audio, sample_rate
return await asyncio.to_thread(_generate_sync)
@@ -1,210 +0,0 @@
"""
Chatterbox Turbo TTS backend implementation.
Wraps ChatterboxTurboTTS from chatterbox-tts for fast, English-only
voice cloning with paralinguistic tag support ([laugh], [cough], etc.).
Forces CPU on macOS due to known MPS tensor issues.
"""
import asyncio
import logging
import threading
from pathlib import Path
from typing import ClassVar, List, Optional, Tuple
import numpy as np
from . import TTSBackend
from .base import (
is_model_cached,
get_torch_device,
combine_voice_prompts as _combine_voice_prompts,
model_load_progress,
patch_chatterbox_f32,
)
logger = logging.getLogger(__name__)
CHATTERBOX_TURBO_HF_REPO = "ResembleAI/chatterbox-turbo"
# Files that must be present for the turbo model
_TURBO_WEIGHT_FILES = [
"t3_turbo_v1.safetensors",
"s3gen_meanflow.safetensors",
"ve.safetensors",
]
class ChatterboxTurboTTSBackend:
"""Chatterbox Turbo TTS backend — fast, English-only, with paralinguistic tags."""
# Class-level lock for torch.load monkey-patching
_load_lock: ClassVar[threading.Lock] = threading.Lock()
def __init__(self):
self.model = None
self.model_size = "default"
self._device = None
self._model_load_lock = asyncio.Lock()
def _get_device(self) -> str:
return get_torch_device(force_cpu_on_mac=True)
def is_loaded(self) -> bool:
return self.model is not None
def _get_model_path(self, model_size: str = "default") -> str:
return CHATTERBOX_TURBO_HF_REPO
def _is_model_cached(self, model_size: str = "default") -> bool:
return is_model_cached(CHATTERBOX_TURBO_HF_REPO, required_files=_TURBO_WEIGHT_FILES)
async def load_model(self, model_size: str = "default") -> None:
"""Load the Chatterbox Turbo model."""
if self.model is not None:
return
async with self._model_load_lock:
if self.model is not None:
return
await asyncio.to_thread(self._load_model_sync)
def _load_model_sync(self):
"""Synchronous model loading."""
model_name = "chatterbox-turbo"
is_cached = self._is_model_cached()
with model_load_progress(model_name, is_cached):
device = self._get_device()
self._device = device
logger.info(f"Loading Chatterbox Turbo TTS on {device}...")
import torch
from huggingface_hub import snapshot_download
from chatterbox.tts_turbo import ChatterboxTurboTTS
local_path = snapshot_download(
repo_id=CHATTERBOX_TURBO_HF_REPO,
token=None,
allow_patterns=["*.safetensors", "*.json", "*.txt", "*.pt", "*.model"],
)
if device == "cpu":
_orig_torch_load = torch.load
def _patched_load(*args, **kwargs):
kwargs.setdefault("map_location", "cpu")
return _orig_torch_load(*args, **kwargs)
with ChatterboxTurboTTSBackend._load_lock:
torch.load = _patched_load
try:
model = ChatterboxTurboTTS.from_local(local_path, device)
finally:
torch.load = _orig_torch_load
else:
model = ChatterboxTurboTTS.from_local(local_path, device)
patch_chatterbox_f32(model)
self.model = model
logger.info("Chatterbox Turbo TTS loaded successfully")
def unload_model(self) -> None:
"""Unload model to free memory."""
if self.model is not None:
device = self._device
del self.model
self.model = None
self._device = None
if device == "cuda":
import torch
torch.cuda.empty_cache()
logger.info("Chatterbox Turbo unloaded")
async def create_voice_prompt(
self,
audio_path: str,
reference_text: str,
use_cache: bool = True,
) -> Tuple[dict, bool]:
"""
Create voice prompt from reference audio.
Chatterbox Turbo processes reference audio at generation time, so the
prompt just stores the file path.
"""
voice_prompt = {
"ref_audio": str(audio_path),
"ref_text": reference_text,
}
return voice_prompt, False
async def combine_voice_prompts(
self,
audio_paths: List[str],
reference_texts: List[str],
) -> Tuple[np.ndarray, str]:
return await _combine_voice_prompts(audio_paths, reference_texts)
async def generate(
self,
text: str,
voice_prompt: dict,
language: str = "en",
seed: Optional[int] = None,
instruct: Optional[str] = None,
) -> Tuple[np.ndarray, int]:
"""
Generate audio using Chatterbox Turbo TTS.
Supports paralinguistic tags in text: [laugh], [cough], [chuckle], etc.
Args:
text: Text to synthesize (may include paralinguistic tags)
voice_prompt: Dict with ref_audio path
language: Ignored (Turbo is English-only)
seed: Random seed for reproducibility
instruct: Unused (protocol compatibility)
Returns:
Tuple of (audio_array, sample_rate)
"""
await self.load_model()
ref_audio = voice_prompt.get("ref_audio")
if ref_audio and not Path(ref_audio).exists():
logger.warning(f"Reference audio not found: {ref_audio}")
ref_audio = None
def _generate_sync():
import torch
if seed is not None:
torch.manual_seed(seed)
logger.info("[Chatterbox Turbo] Generating (English)")
wav = self.model.generate(
text,
audio_prompt_path=ref_audio,
temperature=0.8,
top_k=1000,
top_p=0.95,
repetition_penalty=1.2,
)
# Convert tensor -> numpy
if isinstance(wav, torch.Tensor):
audio = wav.squeeze().cpu().numpy().astype(np.float32)
else:
audio = np.asarray(wav, dtype=np.float32)
sample_rate = (
getattr(self.model, "sr", None)
or getattr(self.model, "sample_rate", 24000)
)
return audio, sample_rate
return await asyncio.to_thread(_generate_sync)
-178
View File
@@ -1,178 +0,0 @@
"""
LuxTTS backend implementation.
Wraps the LuxTTS (ZipVoice) model for zero-shot voice cloning.
~1GB VRAM, 48kHz output, 150x realtime on CPU.
"""
import asyncio
import logging
from typing import Optional, Tuple
import numpy as np
from . import TTSBackend
from .base import is_model_cached, get_torch_device, combine_voice_prompts as _combine_voice_prompts, model_load_progress
from ..utils.cache import get_cache_key, get_cached_voice_prompt, cache_voice_prompt
logger = logging.getLogger(__name__)
# HuggingFace repo for model weight detection
LUXTTS_HF_REPO = "YatharthS/LuxTTS"
class LuxTTSBackend:
"""LuxTTS backend for zero-shot voice cloning."""
def __init__(self):
self.model = None
self.model_size = "default" # LuxTTS has only one model size
self._device = None
def _get_device(self) -> str:
return get_torch_device(allow_mps=True)
def is_loaded(self) -> bool:
return self.model is not None
@property
def device(self) -> str:
if self._device is None:
self._device = self._get_device()
return self._device
def _get_model_path(self, model_size: str) -> str:
return LUXTTS_HF_REPO
def _is_model_cached(self, model_size: str = "default") -> bool:
return is_model_cached(
LUXTTS_HF_REPO,
weight_extensions=(".pt", ".safetensors", ".onnx", ".bin"),
)
async def load_model(self, model_size: str = "default") -> None:
"""Load the LuxTTS model."""
if self.model is not None:
return
await asyncio.to_thread(self._load_model_sync)
def _load_model_sync(self):
model_name = "luxtts"
is_cached = self._is_model_cached()
with model_load_progress(model_name, is_cached):
from zipvoice.luxvoice import LuxTTS
device = self.device
logger.info(f"Loading LuxTTS on {device}...")
if device == "cpu":
import os
threads = os.cpu_count() or 4
self.model = LuxTTS(
model_path=LUXTTS_HF_REPO, device="cpu", threads=min(threads, 8),
)
else:
self.model = LuxTTS(model_path=LUXTTS_HF_REPO, device=device)
logger.info("LuxTTS loaded successfully")
def unload_model(self) -> None:
"""Unload model to free memory."""
if self.model is not None:
del self.model
self.model = None
import torch
if torch.cuda.is_available():
torch.cuda.empty_cache()
logger.info("LuxTTS unloaded")
async def create_voice_prompt(
self,
audio_path: str,
reference_text: str,
use_cache: bool = True,
) -> Tuple[dict, bool]:
"""
Create voice prompt from reference audio.
LuxTTS uses its own encode_prompt() which runs Whisper ASR internally
to transcribe the reference. The reference_text parameter is not used
by LuxTTS itself, but we include it in the cache key for consistency.
"""
await self.load_model()
# Compute cache key once for both lookup and storage
cache_key = ("luxtts_" + get_cache_key(audio_path, reference_text)) if use_cache else None
if cache_key:
cached = get_cached_voice_prompt(cache_key)
if cached is not None and isinstance(cached, dict):
return cached, True
def _encode_sync():
return self.model.encode_prompt(
prompt_audio=str(audio_path),
duration=5,
rms=0.01,
)
encoded = await asyncio.to_thread(_encode_sync)
if cache_key:
cache_voice_prompt(cache_key, encoded)
return encoded, False
async def combine_voice_prompts(self, audio_paths, reference_texts):
return await _combine_voice_prompts(audio_paths, reference_texts, sample_rate=24000)
async def generate(
self,
text: str,
voice_prompt: dict,
language: str = "en",
seed: Optional[int] = None,
instruct: Optional[str] = None,
) -> Tuple[np.ndarray, int]:
"""
Generate audio from text using LuxTTS.
Args:
text: Text to synthesize
voice_prompt: Encoded prompt dict from encode_prompt()
language: Language code (LuxTTS is English-focused)
seed: Random seed for reproducibility
instruct: Not supported by LuxTTS (ignored)
Returns:
Tuple of (audio_array, sample_rate)
"""
await self.load_model()
def _generate_sync():
import torch
if seed is not None:
torch.manual_seed(seed)
if torch.cuda.is_available():
torch.cuda.manual_seed(seed)
wav = self.model.generate_speech(
text=text,
encode_dict=voice_prompt,
num_steps=4,
guidance_scale=3.0,
t_shift=0.5,
speed=1.0,
return_smooth=False, # 48kHz output
)
# LuxTTS returns a tensor (may be on GPU/MPS), move to CPU first
audio = wav.detach().cpu().numpy().squeeze()
return audio, 48000
return await asyncio.to_thread(_generate_sync)
+312 -121
View File
@@ -4,44 +4,36 @@ MLX backend implementation for TTS and STT using mlx-audio.
from typing import Optional, List, Tuple
import asyncio
import logging
import numpy as np
import os
from pathlib import Path
logger = logging.getLogger(__name__)
# PATCH: Import and apply offline patch BEFORE any huggingface_hub usage
# This prevents mlx_audio from making network requests when models are cached
from ..utils.hf_offline_patch import patch_huggingface_hub_offline, ensure_original_qwen_config_cached
patch_huggingface_hub_offline()
ensure_original_qwen_config_cached()
from . import TTSBackend, STTBackend, LANGUAGE_CODE_TO_NAME, WHISPER_HF_REPOS
from .base import is_model_cached, combine_voice_prompts as _combine_voice_prompts, model_load_progress
from . import TTSBackend, STTBackend
from ..utils.cache import get_cache_key, get_cached_voice_prompt, cache_voice_prompt
from ..utils.audio import normalize_audio, load_audio
from ..utils.progress import get_progress_manager
from ..utils.hf_progress import HFProgressTracker, create_hf_progress_callback
from ..utils.tasks import get_task_manager
class MLXTTSBackend:
"""MLX-based TTS backend using mlx-audio."""
def __init__(self, model_size: str = "1.7B"):
self.model = None
self.model_size = model_size
self._current_model_size = None
def is_loaded(self) -> bool:
"""Check if model is loaded."""
return self.model is not None
def _get_model_path(self, model_size: str) -> str:
"""
Get the MLX model path.
Args:
model_size: Model size (1.7B or 0.6B)
Returns:
HuggingFace Hub model ID for MLX
"""
@@ -51,90 +43,167 @@ class MLXTTSBackend:
# 0.6B not yet converted to MLX format
"0.6B": "mlx-community/Qwen3-TTS-12Hz-1.7B-Base-bf16", # Fallback to 1.7B
}
if model_size not in mlx_model_map:
raise ValueError(f"Unknown model size: {model_size}")
hf_model_id = mlx_model_map[model_size]
logger.info("Will download MLX model from HuggingFace Hub: %s", hf_model_id)
print(f"Will download MLX model from HuggingFace Hub: {hf_model_id}")
return hf_model_id
def _is_model_cached(self, model_size: str) -> bool:
return is_model_cached(
self._get_model_path(model_size),
weight_extensions=(".safetensors", ".bin", ".npz"),
)
"""
Check if the model is already cached locally AND fully downloaded.
Args:
model_size: Model size to check
Returns:
True if model is fully cached, False if missing or incomplete
"""
try:
from huggingface_hub import constants as hf_constants
model_path = self._get_model_path(model_size)
repo_cache = Path(hf_constants.HF_HUB_CACHE) / ("models--" + model_path.replace("/", "--"))
if not repo_cache.exists():
return False
# Check for .incomplete files - if any exist, download is still in progress
blobs_dir = repo_cache / "blobs"
if blobs_dir.exists() and any(blobs_dir.glob("*.incomplete")):
print(f"[_is_model_cached] Found .incomplete files for {model_size}, treating as not cached")
return False
# Check that actual model weight files exist in snapshots
snapshots_dir = repo_cache / "snapshots"
if snapshots_dir.exists():
has_weights = (
any(snapshots_dir.rglob("*.safetensors")) or
any(snapshots_dir.rglob("*.bin")) or
any(snapshots_dir.rglob("*.npz"))
)
if not has_weights:
print(f"[_is_model_cached] No model weights found for {model_size}, treating as not cached")
return False
return True
except Exception as e:
print(f"[_is_model_cached] Error checking cache for {model_size}: {e}")
return False
async def load_model_async(self, model_size: Optional[str] = None):
"""
Lazy load the MLX TTS model.
Args:
model_size: Model size to load (1.7B or 0.6B)
"""
if model_size is None:
model_size = self.model_size
# If already loaded with correct size, return
if self.model is not None and self._current_model_size == model_size:
return
# Unload existing model if different size requested
if self.model is not None and self._current_model_size != model_size:
self.unload_model()
# Run blocking load in thread pool
await asyncio.to_thread(self._load_model_sync, model_size)
# Alias for compatibility
load_model = load_model_async
def _load_model_sync(self, model_size: str):
"""Synchronous model loading."""
model_path = self._get_model_path(model_size)
model_name = f"qwen-tts-{model_size}"
is_cached = self._is_model_cached(model_size)
# Force offline mode when cached to avoid network requests
original_hf_hub_offline = os.environ.get("HF_HUB_OFFLINE")
if is_cached:
os.environ["HF_HUB_OFFLINE"] = "1"
logger.info("[PATCH] Model %s is cached, forcing HF_HUB_OFFLINE=1 to avoid network requests", model_size)
try:
with model_load_progress(model_name, is_cached):
from mlx_audio.tts import load
logger.info("Loading MLX TTS model %s...", model_size)
try:
self.model = load(model_path)
except Exception as load_error:
if is_cached and "offline" in str(load_error).lower():
logger.warning("[PATCH] Offline load failed, trying with network: %s", load_error)
os.environ.pop("HF_HUB_OFFLINE", None)
self.model = load(model_path)
else:
raise
finally:
if original_hf_hub_offline is not None:
os.environ["HF_HUB_OFFLINE"] = original_hf_hub_offline
else:
os.environ.pop("HF_HUB_OFFLINE", None)
self._current_model_size = model_size
self.model_size = model_size
logger.info("MLX TTS model %s loaded successfully", model_size)
# Get model path BEFORE importing mlx_audio
model_path = self._get_model_path(model_size)
# Set up progress tracking
progress_manager = get_progress_manager()
task_manager = get_task_manager()
model_name = f"qwen-tts-{model_size}"
# Check if model is already cached
is_cached = self._is_model_cached(model_size)
# Set up progress callback
# If cached: filter out non-download progress
# If not cached: report all progress (we're actually downloading)
progress_callback = create_hf_progress_callback(model_name, progress_manager)
tracker = HFProgressTracker(progress_callback, filter_non_downloads=is_cached)
print(f"Loading MLX TTS model {model_size}...")
# Only track download progress if model is NOT cached
if not is_cached:
# Start tracking download task
task_manager.start_download(model_name)
# Initialize progress state so SSE endpoint has initial data to send
# This provides immediate feedback while HuggingFace fetches metadata
progress_manager.update_progress(
model_name=model_name,
current=0,
total=0, # Will be updated once actual total is known
filename="Connecting to HuggingFace...",
status="downloading",
)
# IMPORTANT: Patch tqdm BEFORE importing mlx_audio
# Otherwise mlx_audio caches reference to original tqdm
tracker_context = tracker.patch_download()
tracker_context.__enter__()
# Import mlx_audio AFTER patching tqdm
from mlx_audio.tts import load
# Load MLX model (downloads automatically)
try:
self.model = load(model_path)
finally:
# Exit the patch context
tracker_context.__exit__(None, None, None)
# Only mark download as complete if we were tracking it
if not is_cached:
progress_manager.mark_complete(model_name)
task_manager.complete_download(model_name)
self._current_model_size = model_size
self.model_size = model_size
print(f"MLX TTS model {model_size} loaded successfully")
except ImportError as e:
print(f"Error: mlx_audio package not found. Install with: pip install mlx-audio")
progress_manager = get_progress_manager()
task_manager = get_task_manager()
model_name = f"qwen-tts-{model_size}"
progress_manager.mark_error(model_name, str(e))
task_manager.error_download(model_name, str(e))
raise
except Exception as e:
print(f"Error loading MLX TTS model: {e}")
progress_manager = get_progress_manager()
task_manager = get_task_manager()
model_name = f"qwen-tts-{model_size}"
progress_manager.mark_error(model_name, str(e))
task_manager.error_download(model_name, str(e))
raise
def unload_model(self):
"""Unload the model to free memory."""
if self.model is not None:
del self.model
self.model = None
self._current_model_size = None
logger.info("MLX TTS model unloaded")
print("MLX TTS model unloaded")
async def create_voice_prompt(
self,
audio_path: str,
@@ -143,20 +212,20 @@ class MLXTTSBackend:
) -> Tuple[dict, bool]:
"""
Create voice prompt from reference audio.
MLX backend stores voice prompt as a dict with audio path and text.
The actual voice prompt processing happens during generation.
Args:
audio_path: Path to reference audio file
reference_text: Transcript of reference audio
use_cache: Whether to use cached prompt if available
Returns:
Tuple of (voice_prompt_dict, was_cached)
"""
await self.load_model_async(None)
# Check cache if enabled
if use_cache:
cache_key = get_cache_key(audio_path, reference_text)
@@ -170,25 +239,53 @@ class MLXTTSBackend:
return cached_prompt, True
else:
# Cached file no longer exists, invalidate cache
logger.warning("Cached audio file not found: %s, regenerating prompt", cached_audio_path)
print(f"Cached audio file not found: {cached_audio_path}, regenerating prompt")
# MLX voice prompt format - store audio path and text
# The model will process this during generation
voice_prompt_items = {
"ref_audio": str(audio_path),
"ref_text": reference_text,
}
# Cache if enabled
if use_cache:
cache_key = get_cache_key(audio_path, reference_text)
cache_voice_prompt(cache_key, voice_prompt_items)
return voice_prompt_items, False
async def combine_voice_prompts(self, audio_paths, reference_texts):
return await _combine_voice_prompts(audio_paths, reference_texts)
async def combine_voice_prompts(
self,
audio_paths: List[str],
reference_texts: List[str],
) -> Tuple[np.ndarray, str]:
"""
Combine multiple reference samples for better quality.
Args:
audio_paths: List of audio file paths
reference_texts: List of reference texts
Returns:
Tuple of (combined_audio, combined_text)
"""
combined_audio = []
for audio_path in audio_paths:
audio, sr = load_audio(audio_path)
audio = normalize_audio(audio)
combined_audio.append(audio)
# Concatenate audio
mixed = np.concatenate(combined_audio)
mixed = normalize_audio(mixed)
# Combine texts
combined_text = " ".join(reference_texts)
return mixed, combined_text
async def generate(
self,
text: str,
@@ -212,33 +309,31 @@ class MLXTTSBackend:
"""
await self.load_model_async(None)
logger.info("Generating audio for text: %s", text)
print(f"Generating audio for text: {text}")
def _generate_sync():
"""Run synchronous generation in thread pool."""
# MLX generate() returns a generator yielding GenerationResult objects
audio_chunks = []
sample_rate = 24000
lang = LANGUAGE_CODE_TO_NAME.get(language, "auto")
# Set seed if provided (MLX uses numpy random)
if seed is not None:
import mlx.core as mx
np.random.seed(seed)
mx.random.seed(seed)
# Extract voice prompt info
ref_audio = voice_prompt.get("ref_audio") or voice_prompt.get("ref_audio_path")
ref_text = voice_prompt.get("ref_text", "")
# Validate that the audio file exists
if ref_audio and not Path(ref_audio).exists():
logger.warning("Audio file not found: %s", ref_audio)
logger.warning("This may be due to a cached voice prompt referencing a deleted temp file.")
logger.warning("Regenerating without voice prompt.")
print(f"Warning: Audio file not found: {ref_audio}")
print("This may be due to a cached voice prompt referencing a deleted temp file.")
print("Regenerating without voice prompt.")
ref_audio = None
# Check if model supports voice cloning via generate method
# MLX API may support ref_audio parameter directly
try:
@@ -246,37 +341,36 @@ class MLXTTSBackend:
if ref_audio:
# Check if generate accepts ref_audio parameter
import inspect
sig = inspect.signature(self.model.generate)
if "ref_audio" in sig.parameters:
# Generate with voice cloning
for result in self.model.generate(text, ref_audio=ref_audio, ref_text=ref_text, lang_code=lang):
for result in self.model.generate(text, ref_audio=ref_audio, ref_text=ref_text):
audio_chunks.append(np.array(result.audio))
sample_rate = result.sample_rate
else:
# Fallback: generate without voice cloning
for result in self.model.generate(text, lang_code=lang):
for result in self.model.generate(text):
audio_chunks.append(np.array(result.audio))
sample_rate = result.sample_rate
else:
# No voice prompt, generate normally
for result in self.model.generate(text, lang_code=lang):
for result in self.model.generate(text):
audio_chunks.append(np.array(result.audio))
sample_rate = result.sample_rate
except Exception as e:
# If voice cloning fails, try without it
logger.warning("Voice cloning failed, generating without voice prompt: %s", e)
for result in self.model.generate(text, lang_code=lang):
print(f"Warning: Voice cloning failed, generating without voice prompt: {e}")
for result in self.model.generate(text):
audio_chunks.append(np.array(result.audio))
sample_rate = result.sample_rate
# Concatenate all chunks
if audio_chunks:
audio = np.concatenate([np.asarray(chunk, dtype=np.float32) for chunk in audio_chunks])
else:
# Fallback: empty audio
audio = np.array([], dtype=np.float32)
return audio, sample_rate
# Run blocking inference in thread pool
@@ -287,60 +381,157 @@ class MLXTTSBackend:
class MLXSTTBackend:
"""MLX-based STT backend using mlx-audio Whisper."""
def __init__(self, model_size: str = "base"):
self.model = None
self.model_size = model_size
def is_loaded(self) -> bool:
"""Check if model is loaded."""
return self.model is not None
def _is_model_cached(self, model_size: str) -> bool:
hf_repo = WHISPER_HF_REPOS.get(model_size, f"openai/whisper-{model_size}")
return is_model_cached(hf_repo, weight_extensions=(".safetensors", ".bin", ".npz"))
"""
Check if the Whisper model is already cached locally AND fully downloaded.
Args:
model_size: Model size to check
Returns:
True if model is fully cached, False if missing or incomplete
"""
try:
from huggingface_hub import constants as hf_constants
model_name = f"openai/whisper-{model_size}"
repo_cache = Path(hf_constants.HF_HUB_CACHE) / ("models--" + model_name.replace("/", "--"))
if not repo_cache.exists():
return False
# Check for .incomplete files - if any exist, download is still in progress
blobs_dir = repo_cache / "blobs"
if blobs_dir.exists() and any(blobs_dir.glob("*.incomplete")):
print(f"[_is_model_cached] Found .incomplete files for whisper-{model_size}, treating as not cached")
return False
# Check that actual model weight files exist in snapshots
snapshots_dir = repo_cache / "snapshots"
if snapshots_dir.exists():
has_weights = (
any(snapshots_dir.rglob("*.safetensors")) or
any(snapshots_dir.rglob("*.bin")) or
any(snapshots_dir.rglob("*.npz"))
)
if not has_weights:
print(f"[_is_model_cached] No model weights found for whisper-{model_size}, treating as not cached")
return False
return True
except Exception as e:
print(f"[_is_model_cached] Error checking cache for whisper-{model_size}: {e}")
return False
async def load_model_async(self, model_size: Optional[str] = None):
"""
Lazy load the MLX Whisper model.
Args:
model_size: Model size (tiny, base, small, medium, large)
"""
if model_size is None:
model_size = self.model_size
if self.model is not None and self.model_size == model_size:
return
# Run blocking load in thread pool
await asyncio.to_thread(self._load_model_sync, model_size)
# Alias for compatibility
load_model = load_model_async
def _load_model_sync(self, model_size: str):
"""Synchronous model loading."""
progress_model_name = f"whisper-{model_size}"
is_cached = self._is_model_cached(model_size)
try:
progress_manager = get_progress_manager()
task_manager = get_task_manager()
progress_model_name = f"whisper-{model_size}"
with model_load_progress(progress_model_name, is_cached):
# Check if model is already cached
is_cached = self._is_model_cached(model_size)
# Set up progress callback and tracker
# If cached: filter out non-download progress
# If not cached: report all progress (we're actually downloading)
progress_callback = create_hf_progress_callback(progress_model_name, progress_manager)
tracker = HFProgressTracker(progress_callback, filter_non_downloads=is_cached)
# Patch tqdm BEFORE importing mlx_audio
tracker_context = tracker.patch_download()
tracker_context.__enter__()
# Import mlx_audio
from mlx_audio.stt import load
model_name = WHISPER_HF_REPOS.get(model_size, f"openai/whisper-{model_size}")
logger.info("Loading MLX Whisper model %s...", model_size)
self.model = load(model_name)
# MLX Whisper uses the standard OpenAI models
model_name = f"openai/whisper-{model_size}"
self.model_size = model_size
logger.info("MLX Whisper model %s loaded successfully", model_size)
print(f"Loading MLX Whisper model {model_size}...")
# Only track download progress if model is NOT cached
if not is_cached:
# Start tracking download task
task_manager.start_download(progress_model_name)
# Initialize progress state so SSE endpoint has initial data to send
progress_manager.update_progress(
model_name=progress_model_name,
current=0,
total=0,
filename="Connecting to HuggingFace...",
status="downloading",
)
# Load the model (tqdm is patched, but filters out non-download progress)
try:
self.model = load(model_name)
finally:
# Exit the patch context
tracker_context.__exit__(None, None, None)
# Only mark download as complete if we were tracking it
if not is_cached:
progress_manager.mark_complete(progress_model_name)
task_manager.complete_download(progress_model_name)
self.model_size = model_size
print(f"MLX Whisper model {model_size} loaded successfully")
except ImportError as e:
print(f"Error: mlx_audio package not found. Install with: pip install mlx-audio")
progress_manager = get_progress_manager()
task_manager = get_task_manager()
progress_model_name = f"whisper-{model_size}"
progress_manager.mark_error(progress_model_name, str(e))
task_manager.error_download(progress_model_name, str(e))
raise
except Exception as e:
print(f"Error loading MLX Whisper model: {e}")
progress_manager = get_progress_manager()
task_manager = get_task_manager()
progress_model_name = f"whisper-{model_size}"
progress_manager.mark_error(progress_model_name, str(e))
task_manager.error_download(progress_model_name, str(e))
raise
def unload_model(self):
"""Unload the model to free memory."""
if self.model is not None:
del self.model
self.model = None
logger.info("MLX Whisper model unloaded")
print("MLX Whisper model unloaded")
async def transcribe(
self,
audio_path: str,
+313 -98
View File
@@ -4,47 +4,47 @@ PyTorch backend implementation for TTS and STT.
from typing import Optional, List, Tuple
import asyncio
import logging
import torch
import numpy as np
from pathlib import Path
logger = logging.getLogger(__name__)
from . import TTSBackend, STTBackend, LANGUAGE_CODE_TO_NAME, WHISPER_HF_REPOS
from .base import (
is_model_cached,
get_torch_device,
combine_voice_prompts as _combine_voice_prompts,
model_load_progress,
)
from . import TTSBackend, STTBackend
from ..utils.cache import get_cache_key, get_cached_voice_prompt, cache_voice_prompt
from ..utils.audio import load_audio
from ..utils.audio import normalize_audio, load_audio
from ..utils.progress import get_progress_manager
from ..utils.hf_progress import HFProgressTracker, create_hf_progress_callback
from ..utils.tasks import get_task_manager
class PyTorchTTSBackend:
"""PyTorch-based TTS backend using Qwen3-TTS."""
def __init__(self, model_size: str = "1.7B"):
self.model = None
self.model_size = model_size
self.device = self._get_device()
self._current_model_size = None
def _get_device(self) -> str:
"""Get the best available device."""
return get_torch_device(allow_xpu=True, allow_directml=True)
if torch.cuda.is_available():
return "cuda"
elif hasattr(torch.backends, 'mps') and torch.backends.mps.is_available():
# MPS can have issues, use CPU for stability
return "cpu"
return "cpu"
def is_loaded(self) -> bool:
"""Check if model is loaded."""
return self.model is not None
def _get_model_path(self, model_size: str) -> str:
"""
Get the HuggingFace Hub model ID.
Args:
model_size: Model size (1.7B or 0.6B)
Returns:
HuggingFace Hub model ID
"""
@@ -52,79 +52,169 @@ class PyTorchTTSBackend:
"1.7B": "Qwen/Qwen3-TTS-12Hz-1.7B-Base",
"0.6B": "Qwen/Qwen3-TTS-12Hz-0.6B-Base",
}
if model_size not in hf_model_map:
raise ValueError(f"Unknown model size: {model_size}")
return hf_model_map[model_size]
def _is_model_cached(self, model_size: str) -> bool:
return is_model_cached(self._get_model_path(model_size))
"""
Check if the model is already cached locally AND fully downloaded.
Args:
model_size: Model size to check
Returns:
True if model is fully cached, False if missing or incomplete
"""
try:
from huggingface_hub import constants as hf_constants
model_path = self._get_model_path(model_size)
repo_cache = Path(hf_constants.HF_HUB_CACHE) / ("models--" + model_path.replace("/", "--"))
if not repo_cache.exists():
return False
# Check for .incomplete files - if any exist, download is still in progress
blobs_dir = repo_cache / "blobs"
if blobs_dir.exists() and any(blobs_dir.glob("*.incomplete")):
print(f"[_is_model_cached] Found .incomplete files for {model_size}, treating as not cached")
return False
# Check that actual model weight files exist in snapshots
snapshots_dir = repo_cache / "snapshots"
if snapshots_dir.exists():
has_weights = (
any(snapshots_dir.rglob("*.safetensors")) or
any(snapshots_dir.rglob("*.bin"))
)
if not has_weights:
print(f"[_is_model_cached] No model weights found for {model_size}, treating as not cached")
return False
return True
except Exception as e:
print(f"[_is_model_cached] Error checking cache for {model_size}: {e}")
return False
async def load_model_async(self, model_size: Optional[str] = None):
"""
Lazy load the TTS model with automatic downloading from HuggingFace Hub.
Args:
model_size: Model size to load (1.7B or 0.6B)
"""
if model_size is None:
model_size = self.model_size
# If already loaded with correct size, return
if self.model is not None and self._current_model_size == model_size:
return
# Unload existing model if different size requested
if self.model is not None and self._current_model_size != model_size:
self.unload_model()
# Run blocking load in thread pool
await asyncio.to_thread(self._load_model_sync, model_size)
# Alias for compatibility
load_model = load_model_async
def _load_model_sync(self, model_size: str):
"""Synchronous model loading."""
model_name = f"qwen-tts-{model_size}"
is_cached = self._is_model_cached(model_size)
try:
progress_manager = get_progress_manager()
task_manager = get_task_manager()
model_name = f"qwen-tts-{model_size}"
with model_load_progress(model_name, is_cached):
# Check if model is already cached
is_cached = self._is_model_cached(model_size)
# Set up progress callback and tracker
# If cached: filter out non-download progress (like "Segment 1/1" during generation)
# If not cached: report all progress (we're actually downloading)
progress_callback = create_hf_progress_callback(model_name, progress_manager)
tracker = HFProgressTracker(progress_callback, filter_non_downloads=is_cached)
# Patch tqdm BEFORE importing qwen_tts
tracker_context = tracker.patch_download()
tracker_context.__enter__()
# Import qwen_tts
from qwen_tts import Qwen3TTSModel
# Get model path (local or HuggingFace Hub ID)
model_path = self._get_model_path(model_size)
logger.info("Loading TTS model %s on %s...", model_size, self.device)
if self.device == "cpu":
self.model = Qwen3TTSModel.from_pretrained(
model_path,
torch_dtype=torch.float32,
low_cpu_mem_usage=False,
print(f"Loading TTS model {model_size} on {self.device}...")
# Only track download progress if model is NOT cached
if not is_cached:
# Start tracking download task
task_manager.start_download(model_name)
# Initialize progress state so SSE endpoint has initial data to send
progress_manager.update_progress(
model_name=model_name,
current=0,
total=0, # Will be updated once actual total is known
filename="Connecting to HuggingFace...",
status="downloading",
)
else:
# Load the model (tqdm is patched, but filters out non-download progress)
try:
self.model = Qwen3TTSModel.from_pretrained(
model_path,
device_map=self.device,
torch_dtype=torch.bfloat16,
torch_dtype=torch.float32 if self.device == "cpu" else torch.bfloat16,
)
self._current_model_size = model_size
self.model_size = model_size
logger.info("TTS model %s loaded successfully", model_size)
finally:
# Exit the patch context
tracker_context.__exit__(None, None, None)
# Only mark download as complete if we were tracking it
if not is_cached:
progress_manager.mark_complete(model_name)
task_manager.complete_download(model_name)
self._current_model_size = model_size
self.model_size = model_size
print(f"TTS model {model_size} loaded successfully")
except ImportError as e:
print(f"Error: qwen_tts package not found. Install with: pip install git+https://github.com/QwenLM/Qwen3-TTS.git")
progress_manager = get_progress_manager()
task_manager = get_task_manager()
model_name = f"qwen-tts-{model_size}"
progress_manager.mark_error(model_name, str(e))
task_manager.error_download(model_name, str(e))
raise
except Exception as e:
print(f"Error loading TTS model: {e}")
print(f"Tip: The model will be automatically downloaded from HuggingFace Hub on first use.")
progress_manager = get_progress_manager()
task_manager = get_task_manager()
model_name = f"qwen-tts-{model_size}"
progress_manager.mark_error(model_name, str(e))
task_manager.error_download(model_name, str(e))
raise
def unload_model(self):
"""Unload the model to free memory."""
if self.model is not None:
del self.model
self.model = None
self._current_model_size = None
if torch.cuda.is_available():
torch.cuda.empty_cache()
logger.info("TTS model unloaded")
print("TTS model unloaded")
async def create_voice_prompt(
self,
audio_path: str,
@@ -133,17 +223,17 @@ class PyTorchTTSBackend:
) -> Tuple[dict, bool]:
"""
Create voice prompt from reference audio.
Args:
audio_path: Path to reference audio file
reference_text: Transcript of reference audio
use_cache: Whether to use cached prompt if available
Returns:
Tuple of (voice_prompt_dict, was_cached)
"""
await self.load_model_async(None)
# Check cache if enabled
if use_cache:
cache_key = get_cache_key(audio_path, reference_text)
@@ -159,7 +249,7 @@ class PyTorchTTSBackend:
# Legacy cache format - convert to dict
# This shouldn't happen in practice, but handle it
return {"prompt": cached_prompt}, True
def _create_prompt_sync():
"""Run synchronous voice prompt creation in thread pool."""
return self.model.create_voice_clone_prompt(
@@ -167,24 +257,48 @@ class PyTorchTTSBackend:
ref_text=reference_text,
x_vector_only_mode=False,
)
# Run blocking operation in thread pool
voice_prompt_items = await asyncio.to_thread(_create_prompt_sync)
# Cache if enabled
if use_cache:
cache_key = get_cache_key(audio_path, reference_text)
cache_voice_prompt(cache_key, voice_prompt_items)
return voice_prompt_items, False
async def combine_voice_prompts(
self,
audio_paths: List[str],
reference_texts: List[str],
) -> Tuple[np.ndarray, str]:
return await _combine_voice_prompts(audio_paths, reference_texts)
"""
Combine multiple reference samples for better quality.
Args:
audio_paths: List of audio file paths
reference_texts: List of reference texts
Returns:
Tuple of (combined_audio, combined_text)
"""
combined_audio = []
for audio_path in audio_paths:
audio, sr = load_audio(audio_path)
audio = normalize_audio(audio)
combined_audio.append(audio)
# Concatenate audio
mixed = np.concatenate(combined_audio)
mixed = normalize_audio(mixed)
# Combine texts
combined_text = " ".join(reference_texts)
return mixed, combined_text
async def generate(
self,
text: str,
@@ -221,7 +335,6 @@ class PyTorchTTSBackend:
wavs, sample_rate = self.model.generate_voice_clone(
text=text,
voice_clone_prompt=voice_prompt,
language=LANGUAGE_CODE_TO_NAME.get(language, "auto"),
instruct=instruct,
)
return wavs[0], sample_rate
@@ -234,25 +347,66 @@ class PyTorchTTSBackend:
class PyTorchSTTBackend:
"""PyTorch-based STT backend using Whisper."""
def __init__(self, model_size: str = "base"):
self.model = None
self.processor = None
self.model_size = model_size
self.device = self._get_device()
def _get_device(self) -> str:
"""Get the best available device."""
return get_torch_device(allow_xpu=True, allow_directml=True)
if torch.cuda.is_available():
return "cuda"
elif hasattr(torch.backends, 'mps') and torch.backends.mps.is_available():
# MPS support for Whisper
return "cpu" # Use CPU for stability
return "cpu"
def is_loaded(self) -> bool:
"""Check if model is loaded."""
return self.model is not None
def _is_model_cached(self, model_size: str) -> bool:
hf_repo = WHISPER_HF_REPOS.get(model_size, f"openai/whisper-{model_size}")
return is_model_cached(hf_repo)
"""
Check if the Whisper model is already cached locally AND fully downloaded.
Args:
model_size: Model size to check
Returns:
True if model is fully cached, False if missing or incomplete
"""
try:
from huggingface_hub import constants as hf_constants
model_name = f"openai/whisper-{model_size}"
repo_cache = Path(hf_constants.HF_HUB_CACHE) / ("models--" + model_name.replace("/", "--"))
if not repo_cache.exists():
return False
# Check for .incomplete files - if any exist, download is still in progress
blobs_dir = repo_cache / "blobs"
if blobs_dir.exists() and any(blobs_dir.glob("*.incomplete")):
print(f"[_is_model_cached] Found .incomplete files for whisper-{model_size}, treating as not cached")
return False
# Check that actual model weight files exist in snapshots
snapshots_dir = repo_cache / "snapshots"
if snapshots_dir.exists():
has_weights = (
any(snapshots_dir.rglob("*.safetensors")) or
any(snapshots_dir.rglob("*.bin"))
)
if not has_weights:
print(f"[_is_model_cached] No model weights found for whisper-{model_size}, treating as not cached")
return False
return True
except Exception as e:
print(f"[_is_model_cached] Error checking cache for whisper-{model_size}: {e}")
return False
async def load_model_async(self, model_size: Optional[str] = None):
"""
Lazy load the Whisper model.
@@ -260,35 +414,95 @@ class PyTorchSTTBackend:
Args:
model_size: Model size (tiny, base, small, medium, large)
"""
print(f"[DEBUG] load_model_async called with size: {model_size}")
if model_size is None:
model_size = self.model_size
print(f"[DEBUG] Model already loaded? {self.model is not None}, current size: {self.model_size}, requested: {model_size}")
if self.model is not None and self.model_size == model_size:
print(f"[DEBUG] Early return - model already loaded")
return
print(f"[DEBUG] Calling asyncio.to_thread for _load_model_sync")
# Run blocking load in thread pool
await asyncio.to_thread(self._load_model_sync, model_size)
print(f"[DEBUG] asyncio.to_thread completed")
# Alias for compatibility
load_model = load_model_async
def _load_model_sync(self, model_size: str):
"""Synchronous model loading."""
progress_model_name = f"whisper-{model_size}"
is_cached = self._is_model_cached(model_size)
print(f"[DEBUG] _load_model_sync called for Whisper {model_size}")
try:
progress_manager = get_progress_manager()
task_manager = get_task_manager()
progress_model_name = f"whisper-{model_size}"
with model_load_progress(progress_model_name, is_cached):
# Check if model is already cached
is_cached = self._is_model_cached(model_size)
# Set up progress callback and tracker
# If cached: filter out non-download progress
# If not cached: report all progress (we're actually downloading)
progress_callback = create_hf_progress_callback(progress_model_name, progress_manager)
tracker = HFProgressTracker(progress_callback, filter_non_downloads=is_cached)
# Patch tqdm BEFORE importing transformers
print("[DEBUG] Starting tqdm patch BEFORE transformers import")
tracker_context = tracker.patch_download()
tracker_context.__enter__()
print("[DEBUG] tqdm patched, now importing transformers")
# Import transformers
from transformers import WhisperProcessor, WhisperForConditionalGeneration
model_name = WHISPER_HF_REPOS.get(model_size, f"openai/whisper-{model_size}")
logger.info("Loading Whisper model %s on %s...", model_size, self.device)
model_name = f"openai/whisper-{model_size}"
print(f"[DEBUG] Model name: {model_name}")
self.processor = WhisperProcessor.from_pretrained(model_name)
self.model = WhisperForConditionalGeneration.from_pretrained(model_name)
print(f"Loading Whisper model {model_size} on {self.device}...")
self.model.to(self.device)
self.model_size = model_size
logger.info("Whisper model %s loaded successfully", model_size)
# Only track download progress if model is NOT cached
if not is_cached:
# Start tracking download task
task_manager.start_download(progress_model_name)
# Initialize progress state so SSE endpoint has initial data to send
progress_manager.update_progress(
model_name=progress_model_name,
current=0,
total=0, # Will be updated once actual total is known
filename="Connecting to HuggingFace...",
status="downloading",
)
# Load models (tqdm is patched, but filters out non-download progress)
try:
self.processor = WhisperProcessor.from_pretrained(model_name)
self.model = WhisperForConditionalGeneration.from_pretrained(model_name)
finally:
# Exit the patch context
tracker_context.__exit__(None, None, None)
# Only mark download as complete if we were tracking it
if not is_cached:
progress_manager.mark_complete(progress_model_name)
task_manager.complete_download(progress_model_name)
self.model.to(self.device)
self.model_size = model_size
print(f"Whisper model {model_size} loaded successfully")
except Exception as e:
print(f"Error loading Whisper model: {e}")
progress_manager = get_progress_manager()
task_manager = get_task_manager()
progress_model_name = f"whisper-{model_size}"
progress_manager.mark_error(progress_model_name, str(e))
task_manager.error_download(progress_model_name, str(e))
raise
def unload_model(self):
"""Unload the model to free memory."""
if self.model is not None:
@@ -296,12 +510,12 @@ class PyTorchSTTBackend:
del self.processor
self.model = None
self.processor = None
if torch.cuda.is_available():
torch.cuda.empty_cache()
logger.info("Whisper model unloaded")
print("Whisper model unloaded")
async def transcribe(
self,
audio_path: str,
@@ -309,21 +523,21 @@ class PyTorchSTTBackend:
) -> str:
"""
Transcribe audio to text.
Args:
audio_path: Path to audio file
language: Optional language hint (en or zh)
Returns:
Transcribed text
"""
await self.load_model_async(None)
def _transcribe_sync():
"""Run synchronous transcription in thread pool."""
# Load audio
audio, sr = load_audio(audio_path, sample_rate=16000)
# Process audio
inputs = self.processor(
audio,
@@ -331,30 +545,31 @@ class PyTorchSTTBackend:
return_tensors="pt",
)
inputs = inputs.to(self.device)
# Generate transcription
# If language is provided, force it; otherwise let Whisper auto-detect
generate_kwargs = {}
# Set language if provided
forced_decoder_ids = None
if language:
# Support all languages from frontend: en, zh, ja, ko, de, fr, ru, pt, es, it
# Whisper supports these and many more
forced_decoder_ids = self.processor.get_decoder_prompt_ids(
language=language,
task="transcribe",
)
generate_kwargs["forced_decoder_ids"] = forced_decoder_ids
# Generate transcription
with torch.no_grad():
predicted_ids = self.model.generate(
inputs["input_features"],
**generate_kwargs,
forced_decoder_ids=forced_decoder_ids,
)
# Decode
transcription = self.processor.batch_decode(
predicted_ids,
skip_special_tokens=True,
)[0]
return transcription.strip()
# Run blocking transcription in thread pool
return await asyncio.to_thread(_transcribe_sync)
+102 -310
View File
@@ -1,343 +1,135 @@
"""
PyInstaller build script for creating standalone Python server binary.
Usage:
python build_binary.py # Build default (CPU) server binary
python build_binary.py --cuda # Build CUDA-enabled server binary
"""
import PyInstaller.__main__
import argparse
import logging
import os
import platform
import sys
from pathlib import Path
logger = logging.getLogger(__name__)
def is_apple_silicon():
"""Check if running on Apple Silicon."""
return platform.system() == "Darwin" and platform.machine() == "arm64"
def build_server(cuda=False):
"""Build Python server as standalone binary.
Args:
cuda: If True, build with CUDA support and name the binary
voicebox-server-cuda instead of voicebox-server.
"""
def build_server():
"""Build Python server as standalone binary."""
backend_dir = Path(__file__).parent
binary_name = "voicebox-server-cuda" if cuda else "voicebox-server"
# PyInstaller arguments
args = [
"server.py", # Use server.py as entry point instead of main.py
"--onefile",
"--name",
binary_name,
'server.py', # Use server.py as entry point instead of main.py
'--onefile',
'--name', 'voicebox-server',
]
# Hide console window on Windows only. On macOS/Linux the sidecar needs
# stdout/stderr for Tauri to capture logs.
if platform.system() == "Windows":
args.append("--noconsole")
# Add local qwen_tts path if specified (for editable installs)
qwen_tts_path = os.getenv("QWEN_TTS_PATH")
qwen_tts_path = os.getenv('QWEN_TTS_PATH')
if qwen_tts_path and Path(qwen_tts_path).exists():
args.extend(["--paths", str(qwen_tts_path)])
logger.info("Using local qwen_tts source from: %s", qwen_tts_path)
args.extend(['--paths', str(qwen_tts_path)])
print(f"Using local qwen_tts source from: {qwen_tts_path}")
# Add common hidden imports
args.extend(
[
"--hidden-import",
"backend",
"--hidden-import",
"backend.main",
"--hidden-import",
"backend.config",
"--hidden-import",
"backend.database",
"--hidden-import",
"backend.models",
"--hidden-import",
"backend.services.profiles",
"--hidden-import",
"backend.services.history",
"--hidden-import",
"backend.services.tts",
"--hidden-import",
"backend.services.transcribe",
"--hidden-import",
"backend.utils.platform_detect",
"--hidden-import",
"backend.backends",
"--hidden-import",
"backend.backends.pytorch_backend",
"--hidden-import",
"backend.utils.audio",
"--hidden-import",
"backend.utils.cache",
"--hidden-import",
"backend.utils.progress",
"--hidden-import",
"backend.utils.hf_progress",
"--hidden-import",
"backend.services.cuda",
"--hidden-import",
"backend.services.effects",
"--hidden-import",
"backend.utils.effects",
"--hidden-import",
"backend.services.versions",
"--hidden-import",
"pedalboard",
"--hidden-import",
"chatterbox",
"--hidden-import",
"chatterbox.tts_turbo",
"--hidden-import",
"chatterbox.mtl_tts",
"--hidden-import",
"backend.backends.chatterbox_backend",
"--hidden-import",
"backend.backends.chatterbox_turbo_backend",
"--hidden-import",
"backend.backends.luxtts_backend",
"--hidden-import",
"zipvoice",
"--hidden-import",
"zipvoice.luxvoice",
"--collect-all",
"zipvoice",
"--collect-all",
"linacodec",
"--hidden-import",
"torch",
"--hidden-import",
"transformers",
"--hidden-import",
"fastapi",
"--hidden-import",
"uvicorn",
"--hidden-import",
"sqlalchemy",
"--hidden-import",
"librosa",
"--hidden-import",
"soundfile",
"--hidden-import",
"qwen_tts",
"--hidden-import",
"qwen_tts.inference",
"--hidden-import",
"qwen_tts.inference.qwen3_tts_model",
"--hidden-import",
"qwen_tts.inference.qwen3_tts_tokenizer",
"--hidden-import",
"qwen_tts.core",
"--hidden-import",
"qwen_tts.cli",
"--copy-metadata",
"qwen-tts",
"--copy-metadata",
"requests",
"--copy-metadata",
"transformers",
"--copy-metadata",
"huggingface-hub",
"--copy-metadata",
"tokenizers",
"--copy-metadata",
"safetensors",
"--copy-metadata",
"tqdm",
"--hidden-import",
"requests",
"--collect-submodules",
"qwen_tts",
"--collect-data",
"qwen_tts",
# Fix for pkg_resources and jaraco namespace packages
"--hidden-import",
"pkg_resources.extern",
"--collect-submodules",
"jaraco",
# inflect uses typeguard @typechecked which calls inspect.getsource()
# at import time — needs .py source files, not just .pyc bytecode
"--collect-all",
"inflect",
# perth ships pretrained watermark model files (hparams.yaml, .pth.tar)
# in perth/perth_net/pretrained/ — needed by chatterbox at runtime
"--collect-all",
"perth",
# piper_phonemize ships espeak-ng-data/ (phoneme tables, language dicts)
# needed by LuxTTS for text-to-phoneme conversion
"--collect-all",
"piper_phonemize",
]
)
# Add common hidden imports (always included)
args.extend([
'--hidden-import', 'backend',
'--hidden-import', 'backend.main',
'--hidden-import', 'backend.config',
'--hidden-import', 'backend.database',
'--hidden-import', 'backend.models',
'--hidden-import', 'backend.profiles',
'--hidden-import', 'backend.history',
'--hidden-import', 'backend.tts',
'--hidden-import', 'backend.transcribe',
'--hidden-import', 'backend.platform_detect',
'--hidden-import', 'backend.providers',
'--hidden-import', 'backend.providers.base',
'--hidden-import', 'backend.providers.bundled',
'--hidden-import', 'backend.providers.types',
'--hidden-import', 'backend.utils.audio',
'--hidden-import', 'backend.utils.cache',
'--hidden-import', 'backend.utils.progress',
'--hidden-import', 'backend.utils.hf_progress',
'--hidden-import', 'backend.utils.validation',
'--hidden-import', 'numpy',
'--hidden-import', 'numpy.core',
'--hidden-import', 'numpy.core._multiarray_umath',
'--hidden-import', 'scipy',
'--hidden-import', 'scipy.signal',
'--hidden-import', 'fastapi',
'--hidden-import', 'uvicorn',
'--hidden-import', 'sqlalchemy',
'--hidden-import', 'librosa',
'--hidden-import', 'soundfile',
# Fix for pkg_resources and jaraco namespace packages
'--hidden-import', 'pkg_resources.extern',
'--collect-submodules', 'jaraco',
# Asyncio and threading support for PyInstaller
'--hidden-import', 'asyncio',
'--hidden-import', 'asyncio.subprocess',
'--hidden-import', 'concurrent.futures',
'--hidden-import', 'concurrent.futures.thread',
])
# Add CUDA-specific hidden imports
if cuda:
logger.info("Building with CUDA support")
args.extend(
[
"--hidden-import",
"torch.cuda",
"--hidden-import",
"torch.backends.cudnn",
]
)
# Platform-specific TTS backend handling
system = platform.system()
if is_apple_silicon():
print("Building for Apple Silicon - including MLX dependencies (bundled)")
args.extend([
'--hidden-import', 'backend.backends',
'--hidden-import', 'backend.backends.mlx_backend',
'--hidden-import', 'mlx',
'--hidden-import', 'mlx.core',
'--hidden-import', 'mlx.nn',
'--hidden-import', 'mlx_audio',
'--hidden-import', 'mlx_audio.tts',
'--hidden-import', 'mlx_audio.stt',
'--collect-submodules', 'mlx',
'--collect-submodules', 'mlx_audio',
# Collect MLX data files including Metal shader libraries (.metallib)
'--collect-data', 'mlx',
'--collect-data', 'mlx_audio',
])
elif system == "Windows" or (system == "Darwin" and not is_apple_silicon()):
# Windows and Intel macOS: Bundle PyTorch CPU provider
print(f"Building for {system} - including PyTorch CPU provider (bundled)")
args.extend([
'--hidden-import', 'backend.backends',
'--hidden-import', 'backend.backends.pytorch_backend',
'--hidden-import', 'torch',
'--hidden-import', 'transformers',
'--hidden-import', 'qwen_tts',
'--hidden-import', 'qwen_tts.inference',
'--hidden-import', 'qwen_tts.inference.qwen3_tts_model',
'--hidden-import', 'qwen_tts.inference.qwen3_tts_tokenizer',
'--hidden-import', 'qwen_tts.core',
'--hidden-import', 'qwen_tts.cli',
'--copy-metadata', 'qwen-tts',
'--collect-submodules', 'qwen_tts',
'--collect-data', 'qwen_tts',
])
else:
# Exclude NVIDIA CUDA packages from CPU-only builds to keep binary small.
# When building from a venv with CUDA torch installed, PyInstaller would
# bundle ~3GB of NVIDIA shared libraries. We exclude both the Python
# modules and the binary DLLs.
nvidia_packages = [
"nvidia",
"nvidia.cublas",
"nvidia.cuda_cupti",
"nvidia.cuda_nvrtc",
"nvidia.cuda_runtime",
"nvidia.cudnn",
"nvidia.cufft",
"nvidia.curand",
"nvidia.cusolver",
"nvidia.cusparse",
"nvidia.nccl",
"nvidia.nvjitlink",
"nvidia.nvtx",
]
for pkg in nvidia_packages:
args.extend(["--exclude-module", pkg])
# Linux: No bundled provider - users download providers separately
print("Building for Linux - no bundled provider (users download separately)")
args.extend([
'--hidden-import', 'backend.backends',
'--hidden-import', 'backend.backends.pytorch_backend',
])
# Add MLX-specific imports if building on Apple Silicon (never for CUDA builds)
if is_apple_silicon() and not cuda:
logger.info("Building for Apple Silicon - including MLX dependencies")
args.extend(
[
"--hidden-import",
"backend.backends.mlx_backend",
"--hidden-import",
"mlx",
"--hidden-import",
"mlx.core",
"--hidden-import",
"mlx.nn",
"--hidden-import",
"mlx_audio",
"--hidden-import",
"mlx_audio.tts",
"--hidden-import",
"mlx_audio.stt",
"--collect-submodules",
"mlx",
"--collect-submodules",
"mlx_audio",
# Use --collect-all so PyInstaller bundles both data files AND
# native shared libraries (.dylib, .metallib) for MLX.
# Previously only --collect-data was used, which caused MLX to
# raise OSError at runtime inside the bundled binary because
# the Metal shader libraries were missing.
"--collect-all",
"mlx",
"--collect-all",
"mlx_audio",
]
)
elif not cuda:
logger.info("Building for non-Apple Silicon platform - PyTorch only")
dist_dir = str(backend_dir / "dist")
build_dir = str(backend_dir / "build")
args.extend(
[
"--distpath",
dist_dir,
"--workpath",
build_dir,
"--noconfirm",
"--clean",
]
)
args.extend([
'--noconfirm',
'--clean',
])
# Change to backend directory
os.chdir(backend_dir)
# For CPU builds on Windows, ensure we're using CPU-only torch.
# If CUDA torch is installed (local dev), swap to CPU torch before building,
# then restore CUDA torch after. This prevents PyInstaller from bundling
# ~3GB of CUDA DLLs into the CPU binary.
restore_cuda = False
if not cuda and platform.system() == "Windows":
import subprocess
result = subprocess.run(
[sys.executable, "-c", "import torch; print(torch.version.cuda or '')"], capture_output=True, text=True
)
has_cuda_torch = bool(result.stdout.strip())
if has_cuda_torch:
logger.info("CUDA torch detected — installing CPU torch for CPU build...")
subprocess.run(
[
sys.executable,
"-m",
"pip",
"install",
"torch",
"torchvision",
"torchaudio",
"--index-url",
"https://download.pytorch.org/whl/cpu",
"--force-reinstall",
"-q",
],
check=True,
)
restore_cuda = True
# Run PyInstaller
try:
PyInstaller.__main__.run(args)
finally:
# Restore CUDA torch if we swapped it out (even on build failure)
if restore_cuda:
logger.info("Restoring CUDA torch...")
import subprocess
subprocess.run(
[
sys.executable,
"-m",
"pip",
"install",
"torch",
"torchvision",
"torchaudio",
"--index-url",
"https://download.pytorch.org/whl/cu126",
"--force-reinstall",
"-q",
],
check=True,
)
logger.info("Binary built in %s", backend_dir / "dist" / binary_name)
PyInstaller.__main__.run(args)
print(f"Binary built in {backend_dir / 'dist' / 'voicebox-server'}")
if __name__ == "__main__":
parser = argparse.ArgumentParser(description="Build voicebox-server binary")
parser.add_argument(
"--cuda",
action="store_true",
help="Build CUDA-enabled binary (voicebox-server-cuda)",
)
cli_args = parser.parse_args()
build_server(cuda=cli_args.cuda)
if __name__ == '__main__':
build_server()
@@ -7,14 +7,14 @@ from datetime import datetime
import uuid
from sqlalchemy.orm import Session
from ..models import (
from .models import (
AudioChannelCreate,
AudioChannelUpdate,
AudioChannelResponse,
ChannelVoiceAssignment,
ProfileChannelAssignment,
)
from ..database import (
from .database import (
AudioChannel as DBAudioChannel,
ChannelDeviceMapping as DBChannelDeviceMapping,
ProfileChannelMapping as DBProfileChannelMapping,

Some files were not shown because too many files have changed in this diff Show More