Compare commits

..
Author SHA1 Message Date
Jamie Pine 1622e08372 format 2026-02-03 06:45:35 -08:00
Jamie Pine dab3344fb5 Enhance StoryList component with dynamic bottom padding
- Introduced useMemo to calculate bottom padding based on the visibility of the StoryTrackEditor and FloatingGenerateBox.
- Updated imports to include useStory hook for fetching the current story data.
- Adjusted the StoryList layout to accommodate the new padding logic, improving UI consistency.
2026-02-03 06:36:09 -08:00
Jamie Pine 66cdb307e9 remove redundant file 2026-02-03 06:33:24 -08:00
Jamie Pine 5d66970fe9 Refactor FloatingGenerateBox and HistoryTable components for improved UI consistency
- Removed unnecessary class from the FloatingGenerateBox button for cleaner styling.
- Updated HistoryTable to remove redundant class from the delete button.
- Reorganized imports in dropdown-menu for better readability.
- Enhanced useAutoUpdater hook with improved dependency management and updated icon usage.
- Deleted obsolete useAutoUpdater.ts file to streamline codebase.
2026-02-03 06:25:27 -08:00
Jamie Pine 637e0b4585 Enhance ProviderSettings component for platform-specific functionality
- Added platform detection for macOS and Windows to improve user experience.
- Updated UI to conditionally disable options and provide clearer guidance based on installed providers.
- Refactored button states and labels for PyTorch CUDA and CPU to reflect availability and download status accurately.
2026-02-03 06:07:19 -08:00
Jamie Pine d21954358d Merge branch 'main' into external-provider-binaries 2026-02-03 05:44:44 -08:00
Jamie Pine 3f633f4a02 Update ProviderSettings component for improved clarity and user guidance
- Enhanced text descriptions for TTS providers, clarifying the functionality of PyTorch CUDA and Apple MLX.
- Removed outdated references to availability and added notes regarding model version differences.
- Improved UI structure by ensuring consistent labeling and disabling options based on system compatibility.
2026-02-03 05:40:37 -08:00
Jamie Pine 43241b85cf Update Dockerfile to install Python 3.12 venv and bootstrap pip
- Replaced python3-pip with python3.12-venv in the installation process.
- Added commands to ensure pip is upgraded and installed after setting Python 3.12 as the default.
- Streamlined the installation of PyTorch by removing redundant pip upgrade command.
2026-02-03 04:11:27 -08:00
Jamie Pine 793e392e56 Add DataFolders component to ServerSettings for managing system folder paths
- Introduced a new DataFolders component to display and manage paths for application data, models, and providers.
- Implemented a FolderRow component for individual folder display, including loading states and open folder functionality.
- Added API client methods and hooks to fetch system folder paths from the backend.
- Updated ServerTab to include the new DataFolders component, enhancing server settings management.
2026-02-03 00:32:18 -08:00
Jamie Pine 73d9b5d70e Refactor ProviderSettings component for improved readability and maintainability
- Removed unnecessary line breaks in the rendering of radio buttons and labels for a cleaner code structure.
- Added missing imports for numpy and scipy in build_binary.py to ensure proper functionality.
- Updated binary Assets.car file to reflect recent changes.
2026-02-02 20:08:07 -08:00
Jamie PineandGitHub 1040625a88 Merge pull request #44 from selop/feature/delivery-instructions
Enhances floating generate box UX
2026-02-02 17:58:19 -08:00
Sergej Lopatkin 6f4503b521 Enhances floating generate box UX
- Adds tooltips on hover for buttons of the generate box
- Replaces the message square icon with a sliders icon for the instruction mode toggle.
- Adds a tooltip to the instruction mode toggle button.
- Updates the placeholder text for the input field.
2026-02-02 22:19:54 +01:00
Sergej LopatkinandGitHub f5b6edc2e7 Merge pull request #1 from jamiepine/main
update fork
2026-02-02 22:19:30 +01:00
Jamie PineandGitHub 8197f0724c Merge pull request #40 from Spyabo/fix/audio-export-path-resolution
Fix: audio export path resolution
2026-02-02 06:54:39 -08:00
Reese Wright d40f7d2676 refactor: improve path resolution readability 2026-02-02 14:54:05 +00:00
Reese Wright 99fbcca7f4 update CHANGELOG for audio export fix 2026-02-02 14:34:39 +00:00
Reese Wright 04f9880c9a fix audio export path resolution 2026-02-02 14:26:34 +00:00
Jamie Pine af7e9814db Remove 'Active' badge for inactive PyTorch providers in ProviderSettings component 2026-02-02 05:23:15 -08:00
Jamie Pine 409ec2dbb1 Enhance ProviderSettings component with loading state and improved UI interactions
- Introduced a loading state to indicate when a provider is starting, enhancing user experience.
- Disabled radio buttons and action buttons during the loading state to prevent user interaction.
- Updated UI elements to reflect the loading state, including a spinner and appropriate cursor styles.
2026-02-02 05:16:17 -08:00
Jamie Pine 3ffbdeed89 Update Dockerfiles to prevent interactive prompts and include timezone data
- Added environment variables to Dockerfiles to set non-interactive mode and configure the timezone to UTC.
- Included installation of `tzdata` in both Dockerfiles to support timezone configuration during the build process.
2026-02-02 05:12:11 -08:00
Jamie Pine 61dabe7382 Enhance provider packaging and CUDA download functionality
- Added support for packaging provider archives in the release workflow, creating platform-specific zip and tar.gz files for distribution.
- Updated the `.gitignore` to exclude `.spec` files.
- Introduced a new `CudaDownloadSection` component to manage CUDA downloads, including progress tracking and error handling.
- Refactored provider download logic to handle archive extraction and cleanup after download.
- Improved subprocess output handling in the provider manager for better logging and error reporting.
2026-02-02 04:06:21 -08:00
Jamie Pine 732d35ca89 Update README and documentation for Linux support and Docker usage
- Revised README to include links for downloading the latest releases for macOS, Windows, and Linux.
- Added detailed instructions for running Voicebox with Docker, including Docker Compose usage.
- Updated installation documentation to reflect Linux availability and provide specific download options for AppImage and Deb packages.
- Enhanced clarity in Docker documentation regarding accessing the web UI.
2026-02-02 03:34:20 -08:00
Jamie Pine 53b1e8868c Enhance provider logging and error handling
- Added functionality to create log files for provider output, improving debugging on Windows.
- Updated error handling to read from log files instead of using subprocess output directly.
- Enhanced logging messages to include log file locations for easier troubleshooting.
2026-02-02 02:19:11 -08:00
Jamie Pine f090759d8f Add Docker support and update dependencies
- Introduced Docker support with CPU-only and GPU-enabled configurations via Dockerfiles and docker-compose files.
- Added a .dockerignore file to exclude unnecessary files from Docker images.
- Updated bun.lock and package.json to include new dependencies for icon handling.
- Enhanced README with Docker usage instructions and deployment options.
- Refactored components to utilize new icon libraries for improved UI consistency.
2026-02-02 02:19:05 -08:00
Jamie Pine 4c4b3e5463 Update dependencies and enhance project configuration
- Removed `lucide-react` version 0.454.0 and downgraded to version 0.316.0 in `bun.lock`.
- Added `@tailwindcss/vite` and `tailwindcss` as development dependencies in `package.json`.
- Updated Vite configuration to include Tailwind CSS plugin.
- Set the HTML document to use a dark theme by adding the `class="dark"` attribute to the `<html>` tag.
2026-02-02 00:36:24 -08:00
Jamie Pine e4f3647f9a Add CPU-only PyTorch installation for Linux in release workflow
- Included a conditional installation of CPU-only PyTorch packages for Ubuntu 22.04 to reduce unnecessary CUDA dependencies.
- Updated the release workflow to ensure compatibility with CPU-focused builds on Linux.
2026-02-02 00:17:57 -08:00
Jamie Pine 9b07a8480d Update Windows configuration in release workflow to use PyTorch backend
- Changed the backend setting for Windows from "none" to "pytorch" to ensure compatibility with bundled PyTorch CPU providers.
- Updated comments for clarity regarding the Windows setup.
2026-02-01 23:25:03 -08:00
Jamie Pine be30a0ac6b Update provider documentation and enhance build configurations
- Clarified the bundling of PyTorch CPU providers for Windows and macOS Intel builds in documentation.
- Improved handling of platform-specific dependencies in the build process, including asyncio support for PyInstaller.
- Updated backend logic to gracefully handle missing dependencies and provide clearer error messages.
- Enhanced progress management to ensure compatibility with PyInstaller's async handling.
- Removed unnecessary exclusions from the build scripts for PyTorch providers to streamline the build process.
2026-02-01 23:21:27 -08:00
Jamie Pine 595747c3d0 Refactor provider health status and update build configurations
- Updated the `ProviderSettings` component to log the current active provider.
- Changed the provider health status to use specific names for MLX and PyTorch backends.
- Removed unnecessary exclusions from the build scripts for both PyTorch CPU and CUDA providers.
- Ensured consistency in the `.spec` files for PyTorch providers by aligning exclusion lists.
2026-02-01 02:29:40 -08:00
Jamie Pine 580179eba3 Refactor TTS provider management and enhance documentation
- Renamed `bundled-mlx` to `apple-mlx` for clarity in provider types.
- Updated the ProviderSettings component to reflect the new provider naming.
- Improved logging for provider startup and error handling in the backend.
- Added scripts for building and installing PyTorch CPU and CUDA providers locally.
- Enhanced the documentation to include details on TTS provider architecture and development setup.
2026-02-01 01:23:20 -08:00
Jamie Pine b9c858295d Update Voicebox description as an alternative to ElevenLabs, rather than Ollama 2026-02-01 00:45:47 -08:00
Jamie Pine 3b14f81741 Enhance release workflow and update provider settings
- Added macOS support for PyTorch CPU providers in the release workflow.
- Updated the ProviderSettings component to handle macOS-specific conditions and improve UI interactions.
- Refactored the radio group component styles for better accessibility and visual consistency.
- Improved provider management logic to ensure proper handling of available providers across different platforms.
2026-02-01 00:00:47 -08:00
Jamie Pine 6dd5bb2311 hugeicons 2026-01-31 22:18:01 -08:00
Jamie Pine dcbdf3e89b Bump version: 0.1.12 → 0.1.13 2026-01-31 20:18:32 -08:00
Jamie Pine 942064912a Update TTS provider methods and dependencies
- Renamed `load_model` to `load_model_async` in TTS provider classes for clarity and consistency.
- Added compatibility alias for `load_model` to maintain existing functionality.
- Enhanced `get_model_status` to handle both synchronous and asynchronous check functions.
- Updated version numbers in `bun.lock` and `Cargo.lock` to 0.1.12, reflecting recent changes.
2026-01-31 20:18:27 -08:00
Jamie Pine 610f64c762 fix linux compile 2026-01-31 07:44:28 -08:00
Jamie Pine a52ff7d950 Add Linux audio capture module with unsupported functionality
- Introduced a new `linux.rs` module for audio capture, indicating that audio capture is not supported on Linux at this time.
- Updated `mod.rs` to include the Linux module conditionally based on the target OS.
2026-01-31 05:10:27 -08:00
Jamie Pine ab10c26ce4 Update release workflow to include libasound2-dev dependency for Ubuntu 22.04 2026-01-31 04:11:30 -08:00
Jamie Pine ce4269ffa5 Refactor build scripts and update release workflow
- Commented out the PyTorch CPU configuration in the release workflow for Ubuntu 22.04.
- Updated TTS provider documentation to clarify options for Windows and Linux users.
- Enhanced build scripts for both CPU and CUDA providers by excluding large unused modules to reduce binary size.
2026-01-31 03:45:41 -08:00
Jamie Pine ec0fb60197 Enhance release workflow and add radio group component
- Updated the release workflow to include a new configuration for the Ubuntu 22.04 platform without TTS bundled.
- Added the @radix-ui/react-radio-group dependency to package.json.
- Implemented a new RadioGroup component for better UI handling of radio inputs.
2026-01-31 03:40:27 -08:00
Jamie Pine ec9402c568 Update qwen-tts version in requirements for CPU and CUDA providers from 0.1.0 to 0.0.5 2026-01-31 03:11:34 -08:00
Jamie Pine d89521559a Update LocalProvider to manage model size dynamically and clean up build scripts
- Introduced a new attribute `_current_model_size` in `LocalProvider` to store the current model size, allowing for dynamic configuration during generation.
- Updated the `generate` method to use the current model size instead of a hardcoded value.
- Modified the `load_model` method to track the requested model size.
- Removed platform-specific extension handling from the build scripts for both CPU and CUDA providers to streamline the build process.
2026-01-31 03:09:49 -08:00
Jamie Pine 80689ad8ce Implement TTS provider management system and update release workflow
- Added support for TTS providers in the backend, including endpoints for listing, starting, stopping, and downloading providers.
- Enhanced the release workflow to build and upload TTS provider binaries for both Windows and Linux platforms.
- Updated the architecture documentation to reflect the new provider system and its benefits for modularity and user experience.
- Introduced a new `ProviderSettings` component in the frontend for managing provider configurations.
2026-01-31 03:05:50 -08:00
Jamie Pine 220333b3bb corrections 2026-01-31 02:15:45 -08:00
Jamie Pine e194e95512 corrections 2026-01-31 02:14:37 -08:00
Jamie Pine e796412c2c corrections 2026-01-31 02:13:42 -08:00
Jamie Pine cb541521d2 Update TTS Provider Architecture status to v0.1.13 2026-01-31 02:11:41 -08:00
Jamie Pine 2bc243f93e Add TTS Provider Architecture plan
Solves GitHub 2GB limit + frequent update UX issues by splitting app into:
- Main app (~150MB): UI + backend logic + Whisper
- TTS Providers (plugins): Separate downloadable binaries
  - pytorch-cpu (~300MB)
  - pytorch-cuda (~2.4GB)
  - mlx (~800MB, macOS)
  - remote (connect to external server)
  - openai (API wrapper)

Benefits:
- Main app under GitHub 2GB limit
- Updates don't require re-downloading providers
- User choice of compute backend
- External provider support for teams/cloud
- Future-proof extensibility
2026-01-31 02:09:42 -08:00
Jamie Pine 0209008d73 disable cuda for 0.1.12 2026-01-31 01:46:14 -08:00
Jamie Pine 9bde534860 Bump version: 0.1.11 → 0.1.12 2026-01-30 21:23:07 -08:00
Jamie PineandGitHub 97eb570b28 Merge pull request #25 from jamiepine/fix-dl-notification-when-generating-from-already-cached-model
Fix dl notification when generating from already cached model
2026-01-30 21:20:25 -08:00
Jamie PineandGitHub 7d0557a099 Merge pull request #27 from jamiepine/model-dl-fix
Enhance model caching checks and progress tracking for downloads
2026-01-30 21:19:52 -08:00
Jamie Pine 60a03c56a9 Enhance model caching checks and progress tracking for downloads
- Updated caching methods in MLX, PyTorch, and backend to ensure models are fully downloaded before being marked as cached.
- Improved progress tracking to filter out non-download progress and provide accurate feedback during model downloads.
- Enhanced HFProgressTracker to skip non-byte progress bars and ensure meaningful progress reporting.
- Refactored progress initialization to provide immediate feedback while fetching metadata from HuggingFace.
- Added error handling and logging for better debugging during cache checks and download processes.
2026-01-30 21:17:01 -08:00
Jamie Pine d3393fb940 Refactor model download progress tracking and enhance SSE handling
- Rearranged imports for consistency in useModelDownloadToast hook.
- Improved logging in useModelDownloadToast for better debugging during download events.
- Updated progress calculation to handle cases where progress exceeds 100%.
- Enhanced toast notifications to reflect download completion and error states.
- Introduced throttling in ProgressManager to optimize SSE updates and prevent overwhelming clients.
- Added new test scripts for monitoring SSE events during model downloads, ensuring accurate progress reporting.
2026-01-30 20:18:53 -08:00
Jamie Pine 07c0aba883 Refactor model download handling and improve progress tracking
- Rearranged imports for consistency across components.
- Enhanced the ModelManagement component to include detailed logging for download actions and errors.
- Updated the ModelProgress component to connect to SSE only when actively downloading, preventing connection exhaustion.
- Added a downloading state to the model status to indicate ongoing downloads.
- Improved toast notifications for model downloads with completion and error callbacks.
- Refactored the useModelDownloadToast hook to support new callbacks for download completion and error handling.
- Updated backend model status to reflect downloading state during active downloads.
2026-01-30 19:53:20 -08:00
Jamie Pine 77418a52ae Update release workflow and model references
- Added a step to install PyTorch with CUDA for Windows in the release workflow.
- Updated model references in backend/main.py to use openai/whisper models instead of mlx-community for the MLX backend.
2026-01-30 18:10:17 -08:00
Jamie Pine 46f6806e14 Update versions and implement auto-update feature
- Bumped version numbers for @voicebox/app, @voicebox/landing, @voicebox/tauri, and @voicebox/web to 0.1.11.
- Added a new `useAutoUpdater` hook to check for app updates on startup and notify users with toast messages.
- Enhanced `UpdateStatus` component to handle version retrieval errors more gracefully.
- Updated dependencies in `package.json` for Tauri plugins to support new update functionalities.
2026-01-30 18:02:28 -08:00
Jamie PineandGitHub 20851ccc2b Merge pull request #24 from jamiepine/fix-multi-sample
Fix multi sample
2026-01-30 17:07:53 -08:00
Jamie Pine e5f4606a6c Update CircleButton component to include default button type
- Added a default `type` prop set to 'button' in the CircleButton component to ensure proper button behavior.
- Enhanced the component's flexibility by allowing the type to be overridden through props.
2026-01-30 17:07:35 -08:00
Jamie Pine 146ef5aaeb Add delete confirmation dialogs in HistoryTable and SampleList components
- Implemented delete confirmation dialogs for both HistoryTable and SampleList components to enhance user experience and prevent accidental deletions.
- Added state management for handling the selected item to be deleted and the visibility of the delete dialog.
- Refactored delete handling functions to utilize the new dialog confirmation flow, improving code clarity and maintainability.
2026-01-30 17:06:12 -08:00
Jamie Pine 971604d14f Implement profile cache management in audio processing
- Added `clear_profile_cache` function to manage cache files for specific profiles.
- Integrated cache clearing in `add_profile_sample`, `delete_profile`, and `delete_profile_sample` functions to ensure stale audio caches are invalidated after modifications.
- Enhanced `clear_voice_prompt_cache` to also delete combined audio files, improving overall cache management.
2026-01-30 16:50:24 -08:00
Jamie Pine 0b17073345 Add test suite for Voicebox backend
- Introduced a new directory for manual test scripts aimed at debugging and validating backend functionality.
- Added README.md detailing the purpose and usage of various test scripts, including tests for TTS generation, model downloads, and progress tracking.
- Included an __init__.py file to define the test suite structure and provide context for the tests.
2026-01-30 16:48:14 -08:00
Jamie Pine 17106b1e40 Add progress tracking and caching checks for model downloads
- Introduced methods to check if models are cached locally in MLX and PyTorch backends.
- Enhanced progress tracking during model loading to filter out non-download progress when models are cached.
- Updated HFProgressTracker to conditionally report progress based on download status.
- Added test scripts for monitoring SSE events during model downloads and verifying progress tracking functionality.
- Improved overall error handling and logging for better debugging during model download processes.
2026-01-30 16:47:54 -08:00
Jamie Pine 953e6ec7d8 Refactor import order and fix typo in SampleList component
- Rearranged import statements for consistency and clarity.
- Corrected the spelling of "interchangeable" in the note about sample quality.
2026-01-30 16:16:17 -08:00
Jamie Pine d3c65fc6c2 Enhance HistoryTable Component with Infinite Scroll and Cache Management
- Updated HistoryTable to implement infinite scrolling for loading history items dynamically.
- Introduced state management for accumulated history and total item count.
- Added Intersection Observer for triggering additional data fetches when scrolling.
- Implemented cache clearing functionality in the backend to manage voice prompt caches effectively.
- Improved loading indicators and user feedback for data fetching states.
- Refactored code for better readability and maintainability.
2026-01-30 16:16:05 -08:00
Jamie PineandGitHub 7fcca09f24 Merge pull request #23 from jamiepine/audio-export-entitlement-fix
Audio export entitlement fix
2026-01-30 15:08:50 -08:00
Jamie Pine b6e772c6ac formatting 2026-01-30 15:08:34 -08:00
Jamie Pine a6b070201b Refactor Tauri Integration to Use Platform Context
- Replaced direct Tauri API calls with a unified platform context across multiple components, enhancing code maintainability and readability.
- Removed the deprecated tauri.ts file, consolidating platform-related logic into the new PlatformContext.
- Updated components such as App, AudioPlayer, and ServerSettings to utilize the new platform context for lifecycle management and server interactions.
- Improved platform detection and handling for audio playback and system audio capture functionalities.
- Ensured consistent error handling and user feedback across the application when interacting with platform-specific features.
2026-01-30 15:04:38 -08:00
Jamie Pine 30352e2419 formatting 2026-01-30 14:39:41 -08:00
Jamie Pine bfa38b36b7 Update file filters for export generation and profile export
- Modified the file extension filters in useHistory.ts and useProfiles.ts to only allow 'zip' files, removing 'voicebox.zip' for a more streamlined export process.
- Added user-selected read-write permission in Entitlements.plist to enhance file handling capabilities.
2026-01-30 14:39:29 -08:00
Jamie Pine 1b66a528d1 Enhance README and UI Components for Performance and Features
- Updated README.md to highlight MLX backend performance improvements on Mac with Metal acceleration.
- Refined ProfileCard and ProfileForm components by optimizing imports and improving error handling for avatar uploads.
- Adjusted landing page content to better describe features, including a new multi-voice narrative editor and performance optimizations for different platforms.
- Bumped version to 0.1.11 in Cargo.lock to reflect recent changes.
2026-01-30 02:53:15 -08:00
Jamie Pine bef4092e6e Bump version: 0.1.10 → 0.1.11 2026-01-30 02:28:08 -08:00
Jamie Pine 9654f7b642 Refactor MLX and PyTorch Backend Model Loading
- Updated hidden imports in build_binary.py to replace 'mlx_audio.asr' with 'mlx_audio.stt'.
- Enhanced model loading logic in MLX and PyTorch backends to ensure proper progress tracking during model downloads.
- Improved error handling and context management for progress tracking in both backends.
- Bumped version to 0.1.10 in Cargo.lock to reflect recent changes.
2026-01-30 02:26:50 -08:00
Jamie Pine eba1244add Bump version: 0.1.9 → 0.1.10 2026-01-29 23:12:16 -08:00
Jamie Pine 94487f32a5 Enhance MLX and PyTorch Backend Integration
- Added support for MLX backend on Apple Silicon, enabling optimized performance for TTS and STT tasks.
- Implemented platform detection to dynamically select between MLX and PyTorch based on the runtime environment.
- Updated build process to include MLX-specific dependencies and configurations for macOS.
- Refactored backend code to improve model loading and inference logic, accommodating backend-specific requirements.
- Enhanced documentation to clarify backend selection and performance benefits for different platforms.
- Streamlined installation instructions and troubleshooting guidance for MLX-related issues.
2026-01-29 23:11:48 -08:00
Jamie Pine 081f45e680 ADDED MLX FOR SUPER FAST GENERATIONS ON APPLE SILICON
- Added support for MLX backend on Apple Silicon, enabling optimized performance for TTS and STT tasks.
- Updated release workflow to include MLX-specific dependencies and configurations for macOS platforms.
- Refactored backend code to dynamically select between MLX and PyTorch based on the runtime environment.
- Enhanced model loading and inference logic to accommodate backend-specific requirements, including updated model IDs and hidden imports.
- Improved health check and model status reporting to reflect the active backend type.
- Streamlined caching mechanisms to support both backend types, ensuring compatibility and performance.
2026-01-29 21:50:46 -08:00
Jamie Pine 86768288ce Enhance MLX Audio Documentation and Testing Framework
- Updated MLX_AUDIO.md to reflect validated status and included detailed validation results, model mapping, and API usage examples.
- Added a demo script (demo.py) for testing audio generation speed and functionality.
- Introduced a test script (test_tts.py) to validate MLX audio model loading and generation, ensuring robust testing for future developments.
- Created a .gitignore file in the mlx-test directory to exclude unnecessary files from version control.
2026-01-29 21:28:22 -08:00
Jamie Pine 0fd063442a Remove risks and mitigations section from MLX_AUDIO.md and update open questions with responses for clarity. This streamlines the documentation and provides clearer guidance on future considerations. 2026-01-29 21:08:28 -08:00
Jamie Pine a0c2493e98 Add MLX Audio Integration for Apple Silicon Support
- Introduced a new backend for MLX audio to enable GPU acceleration on macOS Apple Silicon, improving performance and user experience.
- Implemented platform detection to switch between MLX and PyTorch backends based on the runtime environment.
- Added new streaming capabilities for TTS and STT, enhancing real-time audio generation.
- Updated API endpoints and frontend components to support new features while maintaining backward compatibility.
- Created documentation for backend integration and performance comparisons.
2026-01-29 20:57:52 -08:00
Jamie Pine b39f48cc81 Refactor useGenerationForm to streamline model download handling
- Removed unnecessary isDownloading variable and related logic.
- Consolidated model download state reset to the finally block for improved clarity and reliability.
- Enhanced error handling by ensuring model download state is reset in case of failure.
2026-01-29 20:17:55 -08:00
Jamie Pine 4ff775bc98 Update packageManager version in package.json to [email protected] 2026-01-29 19:53:38 -08:00
Jamie Pine 6351aa75e9 Refactor StoryList component for improved readability and organization
- Reorganized import statements for clarity and consistency.
- Adjusted formatting of state declarations for better readability.
- Streamlined JSX structure for improved visual hierarchy.
- Updated dialog descriptions for consistency in presentation.
- Made minor adjustments to spacing and layout for enhanced UI consistency.
2026-01-29 19:46:27 -08:00
Jamie Pine 43873a883b Update StoryList component styles for improved UI consistency
- Changed border radius of the "No stories yet" message to rounded-2xl for a softer appearance.
- Updated story item borders to rounded-2xl to enhance visual cohesion across the component.
2026-01-29 19:34:28 -08:00
Jamie Pine 60012b81c0 Refactor ProfileForm and SampleList components for improved UI and functionality
- Updated button styles in ProfileForm for better visual consistency and user experience.
- Replaced Pencil icon with Edit in SampleList for clearer action representation.
- Introduced CircleButton component for action buttons in SampleList, enhancing UI responsiveness and clarity.
- Improved layout and hover effects for action buttons in SampleList to streamline user interactions.
2026-01-29 19:32:05 -08:00
Jamie Pine 89f3127c37 Implement avatar upload and management for voice profiles
- Added functionality to upload, delete, and retrieve avatar images for voice profiles.
- Introduced new API endpoints for avatar management, including upload and delete operations.
- Enhanced profile forms and components to support avatar image handling, including previews and error handling.
- Updated database schema to include avatar_path for profiles and added necessary migrations.
- Implemented image validation and processing utilities to ensure proper avatar uploads.
2026-01-29 19:28:42 -08:00
Jamie Pine ef3c3a7f8c Refactor ProfileForm for improved readability and maintainability
- Reorganized import statements for clarity.
- Enhanced conditional checks for restoring saved files with improved formatting.
- Streamlined draft saving logic by consolidating variable declarations.
- Updated UI components for better structure and readability in the form layout.
2026-01-29 18:56:22 -08:00
Jamie Pine 7b5e73cfa8 Add .npmrc for bun usage and update dependencies
- Created a new .npmrc file to enforce bun usage.
- Bumped version numbers for multiple packages to 0.1.9 in bun.lock.
- Added react-sound-visualizer dependency to enhance audio visualization features.
- Introduced convert:assets script in package.json for asset optimization.
- Updated CONTRIBUTING.md with instructions for converting assets to web formats.
- Added documentation files for API endpoints and developer guidelines in the docs directory.
2026-01-29 18:56:10 -08:00
Jamie Pine 462f104494 Enhance ProgressManager for thread safety and event loop integration
- Added thread-safe mechanisms to the ProgressManager for handling model download progress updates.
- Introduced a main event loop setter to ensure safe operations from background threads.
- Improved listener notification to handle updates in a thread-safe manner.
- Updated methods to ensure thread safety when accessing progress data.
2026-01-29 16:23:46 -08:00
Jamie Pine fadb57164e Update README to reflect API endpoint changes and enhance profile creation example
- Updated API endpoints from `/api/...` to `/...` for consistency.
- Modified the speech generation example to include a language parameter.
- Revised the profile creation example to use JSON format instead of form data.
2026-01-29 16:14:19 -08:00
Jamie Pine e870d65136 Add badges to README for downloads, releases, stars, and license 2026-01-29 16:09:10 -08:00
Jamie PineandGitHub 3df40278cc Merge pull request #5 from Snowy7/fix/dev-mode-sidecar
Fix dev mode sidecar and cross-platform HuggingFace cache paths
2026-01-29 16:05:00 -08:00
Jamie PineandGitHub deeef5a474 Merge pull request #12 from tomasmach/feat/makefile
feat: add Makefile for streamlined development workflow
2026-01-29 16:04:48 -08:00
Jamie Pine 236e464525 Bump version: 0.1.8 → 0.1.9 2026-01-29 15:58:31 -08:00
Jamie Pine cf3cf3f002 Enhance model download handling in useGenerationForm and ProgressManager
- Introduced a flag to track download status in useGenerationForm, ensuring proper UI updates during model downloads.
- Updated ProgressManager to only send initial progress updates if the model is actively downloading or extracting, preventing outdated status messages from being sent.
- Improved error handling and logging for better visibility into model download processes.
2026-01-29 15:57:53 -08:00
tomasmach 9d98e1e768 fix: improve Makefile robustness and update CONTRIBUTING docs
- Add exit 1 to test-backend when pytest not installed
- Add exit 1 to test-frontend when no test script configured
- Add venv dependency to db-init target
- Document Makefile usage in CONTRIBUTING.md
2026-01-30 00:51:41 +01:00
Jamie Pine 3be8980f48 Refactor ProfileForm to support draft state management and improve file handling
- Introduced functionality to save and restore form state as a draft when creating a new voice profile.
- Added helper functions for converting files to and from base64 format to facilitate file handling.
- Updated the API types to use a more flexible LanguageCode type for language parameters.
- Enhanced the UI store to manage profile form drafts, improving user experience during profile creation.
2026-01-29 15:45:14 -08:00
tomasmach 76bc070f5b docs: update CHANGELOG with Makefile feature 2026-01-30 00:44:10 +01:00
tomasmach 01838f4773 docs: add Makefile reference and setup instructions to README 2026-01-30 00:35:20 +01:00
tomasmach 39e4f9d08c fix: correct backend server port to match frontend expectations (17493) 2026-01-30 00:33:50 +01:00
Jamie Pine 341d71470c Implement auto-scroll feature in StoryTrackEditor to keep playhead centered during playback
- Added a useEffect hook to automatically scroll the timeline when the playhead moves past the halfway point of the visible area, enhancing user experience during playback.
2026-01-29 15:32:34 -08:00
Jamie Pine bb6cea24ba Refactor StoryTrackEditor to account for time ruler height during drag operations
- Introduced a constant for TIME_RULER_HEIGHT to improve code readability.
- Updated drag position calculations to subtract the time ruler height, ensuring accurate positioning of clips relative to the tracks area.
2026-01-29 15:31:23 -08:00
Jamie Pine fa7ac88abc Reset playback timing anchors in story store for fresh initialization by playback hook 2026-01-29 15:27:27 -08:00
Jamie Pine c68ddc45b1 Enhance contribution guidelines and improve FloatingGenerateBox component
- Updated CONTRIBUTING.md to include instructions for building with a local Qwen3-TTS development version, facilitating easier testing and development.
- Refactored FloatingGenerateBox component to streamline the rendering of text and instruct fields, improving code readability and maintainability.
- Added functionality to handle auto-resizing of text areas based on content changes, enhancing user experience.
- Improved event handling for keyboard interactions in StoryTrackEditor, allowing for play/pause functionality with the spacebar.
- Introduced a MiniSamplePlayer component in SampleList for better audio playback control, including play, pause, and seek features.
- Implemented sample update functionality in the backend, allowing users to edit reference text for audio samples, with appropriate error handling and user feedback.
2026-01-29 15:25:40 -08:00
tomasmach f89dc66d0c feat: add Python version fallback (3.12 > 3.13 > python3) and compatibility warning 2026-01-30 00:10:30 +01:00
tomasmach cba7d7bc23 feat: add Makefile for streamlined development workflow 2026-01-30 00:05:36 +01:00
Jamie PineandGitHub 3c89b068f3 Merge pull request #6 from jamiepine/windows-server-shutdown
Windows server shutdown
2026-01-29 03:12:40 -08:00
Jamie Pine 229841e05e Add GPU type information to health check response
- Updated the health check endpoint to include the type of GPU available (CUDA or MPS).
- Modified the HealthResponse model to accommodate the new gpu_type field, enhancing the response with detailed GPU information.
- This change improves the clarity of system capabilities for users and developers.
2026-01-29 03:12:11 -08:00
Jamie Pine 123e8215e4 Merge branch 'main' into windows-server-shutdown 2026-01-29 03:00:08 -08:00
Jamie Pine 2a3afec2ca Implement graceful shutdown for the server and enhance process management on Windows
- Added a new `/shutdown` endpoint to allow graceful server shutdown via HTTP.
- Implemented process tree management functions to handle child processes during shutdown on Windows.
- Updated the `stop_server` function to attempt graceful shutdown before forcefully terminating processes.
- Enhanced error handling and logging for shutdown operations.
2026-01-29 02:58:21 -08:00
Jamie Pine 99ddd5a0b4 Add asynchronous model download handling for TTS and Whisper models
- Implemented background tasks for downloading TTS and Whisper models to prevent blocking HTTP responses.
- Enhanced error handling during model downloads, providing users with real-time feedback on download status.
- Updated HTTP responses to indicate when models are being downloaded, improving user experience during model initialization.
2026-01-29 02:55:17 -08:00
Jamie Pine 8d730621bc Refactor model download handling to use background tasks
- Moved model download logic into a separate asynchronous function to allow non-blocking HTTP responses.
- Improved error handling by tracking download status and reporting errors without interrupting the main request flow.
- The frontend is now expected to poll the progress endpoint for download status updates.
2026-01-29 02:42:02 -08:00
Jamie Pine e23118f610 Bump version: 0.1.7 → 0.1.8 2026-01-29 02:21:01 -08:00
Jamie Pine d4bfdc0d68 Update version handling in backend and improve HuggingFace cache management
- Added __version__ variable in backend/__init__.py to centralize versioning.
- Updated main.py to use __version__ for API versioning in the FastAPI app.
- Enhanced cache directory handling by utilizing HuggingFace's constants for improved compatibility across platforms.
2026-01-29 02:20:33 -08:00
Jamie Pine 116c108906 Update screenshot asset in landing page for consistency with current design 2026-01-29 00:06:06 -08:00
Jamie Pine 2d23c8e06a Swap screenshot assets in landing page for improved visual representation
- Replaced app screenshot paths to ensure correct images are displayed.
- Adjusted alt text for screenshots to accurately reflect their content.
2026-01-29 00:06:00 -08:00
Jamie Pine 3973a59ba3 Revise README to clarify Voicebox features and benefits
- Changed section title from "Why Voicebox?" to "What is Voicebox?" for better clarity.
- Expanded description to emphasize local-first voice cloning capabilities and professional tools.
- Highlighted privacy, model flexibility, and native performance as key advantages over cloud services.
2026-01-28 23:59:47 -08:00
Jamie Pine d9aa75253a Enhance README with new features and multi-track editor details
- Added multi-sample support for higher quality cloning.
- Introduced a new Stories Editor section with features for multi-track composition, inline audio editing, auto-playback, and voice mixing.
- Updated recording section to include system audio capture for macOS and Windows.
2026-01-28 23:55:44 -08:00
Jamie Pine b22bf36565 Update README and landing page with new screenshots; bump version to 0.1.7
- Replaced existing screenshot paths in README and landing page with new assets.
- Added additional screenshots to the landing page for enhanced visual representation.
- Updated version in Cargo.lock from 0.1.6 to 0.1.7.
2026-01-28 23:51:43 -08:00
Jamie Pine 33f4ed9b44 Bump version: 0.1.6 → 0.1.7 2026-01-28 22:28:18 -08:00
Jamie Pine cc37e04221 Refactor HistoryTable and SampleList components for improved code consistency
- Cleaned up formatting in HistoryTable for better readability.
- Adjusted import statements in SampleList to maintain consistent structure.
2026-01-28 22:28:01 -08:00
Jamie Pine 2b4fbe5173 Refactor AudioPlayer and related components to support conditional auto-play functionality
- Updated AudioPlayer to auto-play only if the shouldAutoPlay flag is set, enhancing user control over playback.
- Refactored HistoryTable, SampleList, and useGenerationForm to utilize setAudioWithAutoPlay for consistent audio loading and playback behavior.
- Improved user experience by ensuring audio is only played when explicitly intended, reducing unexpected playback.
2026-01-28 22:27:37 -08:00
Snowy 423d69b7cc Use HuggingFace's built-in cache detection for cross-platform support
Replace hardcoded ~/.cache/huggingface/hub paths with
huggingface_hub.constants.HF_HUB_CACHE which correctly handles
OS-specific cache locations (Windows uses AppData, etc.)
2026-01-29 09:25:41 +03:00
Jamie Pine ea943876dc formatting 2026-01-28 22:23:27 -08:00
Jamie Pine b55d8cc567 Implement auto-activation of stories in StoryTrackEditor and improve playback state management
- Added useEffect to automatically activate the story when the editor is shown, ensuring the playhead is visible.
- Introduced setActiveStory function in storyStore to manage story activation without playback.
- Updated playback state checks to reflect the current playing status accurately.
- Enhanced UI to always display the playhead for better user experience during playback.
2026-01-28 22:22:47 -08:00
Jamie Pine 036d90dc8e Enhance story item management with trimming, splitting, and duplication features
- Updated StoryTrackEditor and StoryContent components to support trimming and splitting of story items.
- Introduced new API endpoints for trimming, splitting, and duplicating story items, enhancing item management capabilities.
- Refactored related hooks and state management to accommodate new functionalities.
- Improved data models to include trim start and end times for better audio playback control.
- Enhanced UI interactions for selecting and managing story items within the track editor.
2026-01-28 22:16:53 -08:00
Snowy c513451277 Fix dev mode to work without pre-built server binary
Previously, running `bun run dev` would fail because Tauri requires
the sidecar binary to exist at compile time, even in development mode.
This forced developers to build the full PyInstaller binary before
they could start development.

This change introduces a streamlined dev workflow:

1. Add `scripts/setup-dev-sidecar.js` - Creates minimal placeholder
   binaries that satisfy Tauri's compile-time check. Works cross-platform
   (Windows PE stub, Unix shell script).

2. Update Rust code to gracefully handle dev mode - When the sidecar
   fails to start, it checks if a manually-started server is already
   running on the expected port and connects to it instead.

3. Update npm scripts - `bun run dev` now auto-runs the setup script,
   and `dev:server` uses the correct port (17493).

4. Update CONTRIBUTING.md with clearer dev workflow documentation.

New development workflow:
  Terminal 1: bun run dev:server
  Terminal 2: bun run dev

The bundled binary is only required for production builds.
2026-01-29 09:11:41 +03:00
Jamie PineandGitHub 27ae6dfbab Merge pull request #3 from jamiepine/stories
Stories
2026-01-28 21:18:19 -08:00
238 changed files with 28972 additions and 2553 deletions
+4 -4
View File
@@ -1,5 +1,5 @@
[bumpversion]
current_version = 0.1.6
current_version = 0.1.13
commit = True
tag = True
tag_name = v{new_version}
@@ -34,6 +34,6 @@ replace = "version": "{new_version}"
search = "version": "{current_version}"
replace = "version": "{new_version}"
[bumpversion:file:backend/main.py]
search = "version": "{current_version}"
replace = "version": "{new_version}"
[bumpversion:file:backend/__init__.py]
search = __version__ = "{current_version}"
replace = __version__ = "{new_version}"
+55
View File
@@ -0,0 +1,55 @@
# Dependencies
node_modules/
__pycache__/
*.pyc
*.pyo
*.pyd
.Python
env/
venv/
# Build outputs
build/
*.egg-info/
.eggs/
target/
# Keep web/dist for the Docker image
!web/dist
# Development
.git/
.github/
.vscode/
.idea/
*.swp
*.swo
*~
# Data and logs
data/
*.log
*.sqlite
*.db
# OS
.DS_Store
Thumbs.db
# Documentation
docs/
landing/
mlx-test/
# Test files
*.test.ts
*.test.tsx
*.spec.ts
*.spec.tsx
# Keep these out
.env
.env.local
*.pem
*.key
credentials.json
+260 -21
View File
@@ -4,9 +4,153 @@ on:
workflow_dispatch:
push:
tags:
- 'v*'
- "v*"
env:
PROVIDER_VERSION: "1.0.0"
jobs:
# ============================================
# Build TTS Providers (uploaded to R2, not GitHub)
# ============================================
build-providers:
runs-on: ${{ matrix.platform }}
strategy:
fail-fast: false
matrix:
include:
# PyTorch CPU provider (Windows)
- platform: "windows-latest"
provider: "pytorch-cpu"
python-version: "3.12"
# PyTorch CUDA provider (Windows) - large binary, uploaded to R2
- platform: "windows-latest"
provider: "pytorch-cuda"
python-version: "3.12"
# PyTorch CPU provider (Linux)
- platform: "ubuntu-22.04"
provider: "pytorch-cpu"
python-version: "3.12"
# PyTorch CUDA provider (Linux) - large binary, uploaded to R2
- platform: "ubuntu-22.04"
provider: "pytorch-cuda"
python-version: "3.12"
# PyTorch CPU provider (macOS Apple Silicon)
- platform: "macos-latest"
provider: "pytorch-cpu"
python-version: "3.12"
# PyTorch CPU provider (macOS Intel)
- platform: "macos-15-intel"
provider: "pytorch-cpu"
python-version: "3.12"
steps:
- uses: actions/checkout@v4
- name: Install dependencies (ubuntu only)
if: matrix.platform == 'ubuntu-22.04'
run: |
sudo apt-get update
sudo apt-get install -y llvm-dev
- name: Setup Python
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
cache: "pip"
- name: Install CPU-only torch (Linux)
if: matrix.provider == 'pytorch-cpu' && matrix.platform == 'ubuntu-22.04'
run: |
python -m pip install --upgrade pip
pip install pyinstaller
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cpu
pip install -r providers/pytorch-cpu/requirements.txt
pip install -r backend/requirements.txt
- name: Install Python dependencies (CPU - non-Linux)
if: matrix.provider == 'pytorch-cpu' && matrix.platform != 'ubuntu-22.04'
run: |
python -m pip install --upgrade pip
pip install pyinstaller
pip install -r providers/pytorch-cpu/requirements.txt
pip install -r backend/requirements.txt
- name: Install Python dependencies (CUDA)
if: matrix.provider == 'pytorch-cuda'
run: |
python -m pip install --upgrade pip
pip install pyinstaller
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121
pip install -r providers/pytorch-cuda/requirements.txt
pip install -r backend/requirements.txt
- name: Build provider binary
shell: bash
run: |
cd providers/${{ matrix.provider }}
python build.py
- name: Package provider for distribution
shell: bash
run: |
cd providers/${{ matrix.provider }}/dist
# Add platform suffix for archive name
if [ "${{ matrix.platform }}" == "windows-latest" ]; then
ARCHIVE_NAME="tts-provider-${{ matrix.provider }}-windows.zip"
# On Windows, zip the directory
powershell Compress-Archive -Path "tts-provider-${{ matrix.provider }}/*" -DestinationPath "$ARCHIVE_NAME"
elif [ "${{ matrix.platform }}" == "macos-latest" ]; then
ARCHIVE_NAME="tts-provider-${{ matrix.provider }}-macos-arm64.tar.gz"
tar -czf "$ARCHIVE_NAME" tts-provider-${{ matrix.provider }}/
elif [ "${{ matrix.platform }}" == "macos-15-intel" ]; then
ARCHIVE_NAME="tts-provider-${{ matrix.provider }}-macos-x64.tar.gz"
tar -czf "$ARCHIVE_NAME" tts-provider-${{ matrix.provider }}/
else
ARCHIVE_NAME="tts-provider-${{ matrix.provider }}-linux.tar.gz"
tar -czf "$ARCHIVE_NAME" tts-provider-${{ matrix.provider }}/
fi
echo "Created archive: $ARCHIVE_NAME"
ls -lh "$ARCHIVE_NAME"
- name: Upload provider to R2
shell: bash
env:
R2_ACCESS_KEY_ID: ${{ secrets.R2_ACCESS_KEY_ID }}
R2_SECRET_ACCESS_KEY: ${{ secrets.R2_SECRET_ACCESS_KEY }}
R2_ENDPOINT: ${{ secrets.R2_ENDPOINT }}
run: |
# Install AWS CLI (compatible with R2)
pip install awscli
# Configure AWS CLI for R2
aws configure set aws_access_key_id $R2_ACCESS_KEY_ID
aws configure set aws_secret_access_key $R2_SECRET_ACCESS_KEY
aws configure set region auto
# Determine archive name based on platform
if [ "${{ matrix.platform }}" == "windows-latest" ]; then
ARCHIVE_NAME="tts-provider-${{ matrix.provider }}-windows.zip"
elif [ "${{ matrix.platform }}" == "macos-latest" ]; then
ARCHIVE_NAME="tts-provider-${{ matrix.provider }}-macos-arm64.tar.gz"
elif [ "${{ matrix.platform }}" == "macos-15-intel" ]; then
ARCHIVE_NAME="tts-provider-${{ matrix.provider }}-macos-x64.tar.gz"
else
ARCHIVE_NAME="tts-provider-${{ matrix.provider }}-linux.tar.gz"
fi
# Upload to R2 (bucket: voicebox)
aws s3 cp "providers/${{ matrix.provider }}/dist/$ARCHIVE_NAME" \
"s3://voicebox/providers/v${{ env.PROVIDER_VERSION }}/$ARCHIVE_NAME" \
--endpoint-url "$R2_ENDPOINT"
echo "Uploaded $ARCHIVE_NAME to R2"
# ============================================
# Build Main App (without bundled TTS on Win/Linux)
# ============================================
release:
permissions:
contents: write
@@ -14,18 +158,26 @@ jobs:
fail-fast: false
matrix:
include:
- platform: 'macos-latest'
args: '--target aarch64-apple-darwin'
python-version: '3.12'
- platform: 'macos-15-intel'
args: '--target x86_64-apple-darwin'
python-version: '3.12'
# - platform: 'ubuntu-22.04'
# args: ''
# python-version: '3.12'
- platform: 'windows-latest'
args: ''
python-version: '3.12'
# macOS Apple Silicon - MLX bundled (works out of the box)
- platform: "macos-latest"
args: "--target aarch64-apple-darwin"
python-version: "3.12"
backend: "mlx"
# macOS Intel - PyTorch bundled (smaller user base, keep simple)
- platform: "macos-15-intel"
args: "--target x86_64-apple-darwin"
python-version: "3.12"
backend: "pytorch"
# Linux - No TTS bundled, providers downloaded separately
- platform: "ubuntu-22.04"
args: ""
python-version: "3.12"
backend: "none"
# Windows - PyTorch CPU bundled (works out of the box)
- platform: "windows-latest"
args: ""
python-version: "3.12"
backend: "pytorch"
runs-on: ${{ matrix.platform }}
@@ -36,7 +188,7 @@ jobs:
if: matrix.platform == 'ubuntu-22.04'
run: |
sudo apt-get update
sudo apt-get install -y libwebkit2gtk-4.1-dev libappindicator3-dev librsvg2-dev patchelf llvm-dev
sudo apt-get install -y libwebkit2gtk-4.1-dev libappindicator3-dev librsvg2-dev patchelf llvm-dev libasound2-dev
- name: Install LLVM (macOS)
if: matrix.platform == 'macos-latest' || matrix.platform == 'macos-15-intel'
@@ -49,14 +201,29 @@ jobs:
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
cache: 'pip'
cache: "pip"
- name: Install Python dependencies
- name: Install Python dependencies (with TTS)
if: matrix.backend != 'none'
run: |
python -m pip install --upgrade pip
pip install pyinstaller
pip install -r backend/requirements.txt
- name: Install Python dependencies (without TTS)
if: matrix.backend == 'none'
run: |
python -m pip install --upgrade pip
pip install pyinstaller
# Install base requirements without PyTorch/Qwen-TTS
pip install fastapi uvicorn sqlalchemy librosa soundfile numpy httpx
pip install huggingface_hub # For Whisper downloads
- name: Install MLX dependencies (Apple Silicon only)
if: matrix.backend == 'mlx'
run: |
pip install -r backend/requirements-mlx.txt
- name: Build Python server (Linux/macOS)
if: matrix.platform != 'windows-latest'
run: |
@@ -91,7 +258,7 @@ jobs:
- name: Rust cache
uses: swatinem/rust-cache@v2
with:
workspaces: './tauri/src-tauri -> target'
workspaces: "./tauri/src-tauri -> target"
- name: Install dependencies
run: bun install
@@ -127,18 +294,90 @@ jobs:
with:
projectPath: tauri
tagName: v__VERSION__
releaseName: 'voicebox v__VERSION__'
releaseName: "voicebox v__VERSION__"
releaseBody: |
## What's Changed
See the assets below to download and install this version.
### Installation
- **macOS**: Download the `.dmg` file
- **Windows**: Download the `.msi` installer
- **Linux**: Download the `.AppImage` or `.deb` package
- **macOS (Apple Silicon)**: Download the `aarch64.dmg` file - uses MLX for fast native inference (works out of the box)
- **macOS (Intel)**: Download the `x64.dmg` file - uses PyTorch
- **Windows**: Download the `.msi` installer - requires downloading a TTS provider on first use
- **Linux**: Download the `.AppImage` or `.deb` package - requires downloading a TTS provider on first use
### TTS Providers
Windows and Linux users will be prompted to download a TTS provider on first launch:
- **Windows**: PyTorch CPU (~300MB) or PyTorch CUDA (~2.4GB for NVIDIA GPUs)
- **Linux**: PyTorch CUDA (~2.4GB) - requires NVIDIA GPU
The app includes automatic updates - future updates will be installed automatically.
releaseDraft: true
prerelease: false
args: ${{ matrix.args }}
includeUpdaterJson: true
# ============================================
# Build and Push Docker Images
# ============================================
docker:
runs-on: ubuntu-latest
permissions:
contents: read
packages: write
steps:
- uses: actions/checkout@v4
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Login to GitHub Container Registry
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Setup Bun
uses: oven-sh/setup-bun@v2
- name: Install dependencies and build web UI
run: |
bun install
cd web
bun run build
- name: Extract version from tag
id: version
run: |
if [[ $GITHUB_REF == refs/tags/v* ]]; then
VERSION=${GITHUB_REF#refs/tags/v}
else
VERSION="dev"
fi
echo "version=$VERSION" >> $GITHUB_OUTPUT
- name: Build and push CPU image
uses: docker/build-push-action@v5
with:
context: .
file: ./Dockerfile
platforms: linux/amd64,linux/arm64
push: true
tags: |
ghcr.io/jamiepine/voicebox:latest
ghcr.io/jamiepine/voicebox:${{ steps.version.outputs.version }}
cache-from: type=gha
cache-to: type=gha,mode=max
- name: Build and push CUDA image
uses: docker/build-push-action@v5
with:
context: .
file: ./Dockerfile.cuda
platforms: linux/amd64
push: true
tags: |
ghcr.io/jamiepine/voicebox:latest-cuda
ghcr.io/jamiepine/voicebox:${{ steps.version.outputs.version }}-cuda
cache-from: type=gha
cache-to: type=gha,mode=max
+1
View File
@@ -15,6 +15,7 @@ dist/
build/
*.egg-info/
*.egg
*.spec
target/
*.app
*.dmg
+2
View File
@@ -0,0 +1,2 @@
# Force bun usage
engine-strict=true
+17
View File
@@ -53,6 +53,23 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased]
### Fixed
- Audio export failing when Tauri save dialog returns object instead of string path
### Added
- **Makefile** - Comprehensive development workflow automation with commands for setup, development, building, testing, and code quality checks
- Includes Python version detection and compatibility warnings
- Self-documenting help system with `make help`
- Colored output for better readability
- Supports parallel development server execution
### Changed
- **README** - Added Makefile reference and updated Quick Start with Makefile-based setup instructions alongside manual setup
---
## [Unreleased - Planned]
### Planned
- Real-time streaming synthesis
- Conversation mode with multiple speakers
+214 -25
View File
@@ -14,16 +14,19 @@ Thank you for your interest in contributing to Voicebox! This document provides
### Prerequisites
- **[Bun](https://bun.sh)** - Fast JavaScript runtime and package manager
```bash
curl -fsSL https://bun.sh/install | bash
```
- **[Python 3.11+](https://python.org)** - For backend development
```bash
python --version # Should be 3.11 or higher
```
- **[Rust](https://rustup.rs)** - For Tauri desktop app (installed automatically by Tauri CLI)
```bash
rustc --version # Check if installed
```
@@ -32,111 +35,293 @@ Thank you for your interest in contributing to Voicebox! This document provides
### Development Setup
**Using the Makefile (recommended for macOS/Linux):** Run `make setup` to install all dependencies, then `make dev` to start development servers. See `make help` for all available commands.
**Manual setup (required for Windows):**
1. **Fork and clone the repository**
```bash
git clone https://github.com/YOUR_USERNAME/voicebox.git
cd voicebox
```
2. **Install JavaScript dependencies**
```bash
bun install
```
This installs dependencies for:
- `app/` - Shared React frontend
- `tauri/` - Tauri desktop wrapper
- `web/` - Web deployment wrapper
3. **Set up Python backend**
```bash
cd backend
# Create virtual environment
python -m venv venv
# Activate virtual environment
source venv/bin/activate # On macOS/Linux
# or
venv\Scripts\activate # On Windows
# Install Python dependencies
pip install -r requirements.txt
# Install MLX dependencies (Apple Silicon only - for faster inference)
# On Apple Silicon, this enables native Metal acceleration
if [[ $(uname -m) == "arm64" ]]; then
pip install -r requirements-mlx.txt
fi
# Install Qwen3-TTS (required for voice synthesis)
pip install git+https://github.com/QwenLM/Qwen3-TTS.git
```
4. **Initialize database**
```bash
cd backend
python -c "from database import init_db; init_db()"
```
This creates the SQLite database at `data/voicebox.db`.
4. **Start development servers**
Development requires two terminals: one for the Python backend, one for the Tauri app.
**Terminal 1: Backend server** (start this first)
5. **Start development servers**
**Terminal 1: Backend server**
```bash
cd backend
source venv/bin/activate # Activate venv if not already active
bun run dev:server
# Or manually: uvicorn main:app --reload --port 8000
# Or manually: uvicorn main:app --reload --port 17493
```
Backend will be available at `http://localhost:8000`
Backend will be available at `http://localhost:17493`
**Terminal 2: Desktop app**
```bash
bun run dev
```
This will:
- Create a placeholder sidecar binary (for Tauri compilation)
- Start Vite dev server on port 5173
- Launch Tauri window pointing to localhost:5173
- Connect to the Python server you started in Terminal 1
- Enable hot reload
> **Note:** In dev mode, the app connects to your manually-started Python server.
> The bundled server binary is only used in production builds.
**Optional: Web app**
```bash
bun run dev:web
```
Web app will be available at `http://localhost:5174`
### Model Downloads
Models are automatically downloaded from HuggingFace Hub on first use:
- **Whisper** (transcription): Auto-downloads on first transcription
- **Qwen3-TTS** (voice cloning): Auto-downloads on first generation (~2-4GB)
First-time usage will be slower due to model downloads, but subsequent runs will use cached models.
### TTS Provider Development
Voicebox uses a modular provider system to support different inference backends. Understanding this architecture is important when working on TTS features.
#### Provider Types
**Bundled Providers** — Included with the app binary:
- `apple-mlx` — Bundled with macOS Apple Silicon builds (`.dmg` for aarch64)
- Uses MLX for native Metal acceleration
- Configured in `.github/workflows/release.yml` with `backend: "mlx"`
**Hybrid Provider:**
- `pytorch-cpu` — Can be bundled OR downloaded depending on platform
- **Bundled** with Windows and macOS Intel builds
- macOS Intel: `.dmg` for x64 with `backend: "pytorch"`
- Windows: `.exe` installer with PyTorch CPU included
- **Downloaded** on first use for Linux builds (~300MB)
- Falls back to bundled version if external binary not found
**External-Only Providers:**
- `pytorch-cuda` — NVIDIA GPU-accelerated provider (~2.4GB)
- Windows/Linux only (no NVIDIA GPUs on macOS)
- Downloaded on demand, not bundled
- Optional for users with CUDA-capable GPUs
#### Provider Architecture
```
backend/providers/
├── __init__.py # ProviderManager - lifecycle management
├── base.py # TTSProvider protocol
├── bundled.py # BundledProvider - wraps built-in backends
├── local.py # LocalProvider - wraps external subprocess
├── installer.py # Download and install external providers
└── types.py # Provider type definitions
providers/
├── pytorch-cpu/ # External PyTorch CPU provider
│ ├── main.py # FastAPI server
│ ├── build.py # PyInstaller build script
│ └── build_and_install.py # Build and install locally
└── pytorch-cuda/ # External PyTorch CUDA provider
├── main.py
├── build.py
└── build_and_install.py
```
**How it works:**
1. **Bundled providers** run in-process within the main backend
2. **External providers** run as separate subprocess servers
3. **LocalProvider** communicates with external providers via HTTP
4. **ProviderManager** handles starting/stopping and health checks
#### Building Providers Locally
When developing provider features, you'll need to build and test external providers:
**Build a single provider:**
```bash
cd providers/pytorch-cpu
python build_and_install.py
```
**Build all providers:**
```bash
bun run build:providers
```
This script:
- Builds the provider binary with PyInstaller
- Detects your platform (Windows/macOS/Linux)
- Copies to the correct location:
- macOS: `~/Library/Application Support/voicebox/providers/`
- Windows: `%APPDATA%\voicebox\providers\`
- Linux: `~/.local/share/voicebox/providers/`
- Sets executable permissions on Unix
**Testing provider changes:**
1. Make changes to `providers/pytorch-cpu/main.py`
2. Run `bun run build:providers`
3. Restart the Voicebox app
4. Select the provider in Settings → TTS Provider
#### Provider Binary Distribution
For production releases, provider binaries are:
1. Built by GitHub Actions for all platforms
2. Uploaded to Cloudflare R2 at `downloads.voicebox.sh/providers/v{VERSION}/`
3. Downloaded on-demand by users based on their platform and GPU
See `.github/workflows/release.yml` for the build matrix.
### Building
**Build Python server binary:**
**Build everything (recommended):**
```bash
bun run build
```
This automatically:
1. Builds the Python server binary (`./scripts/build-server.sh`)
2. Builds the Tauri desktop app (`cd tauri && bun run tauri build`)
Creates platform-specific installers (`.dmg`, `.msi`, `.AppImage`) in `tauri/src-tauri/target/release/bundle/`.
**Note:** The build process detects your platform and includes the appropriate backend (MLX for Apple Silicon, PyTorch for others).
**Build server binary only:**
```bash
bun run build:server
# or
./scripts/build-server.sh
```
Creates platform-specific binary in `tauri/src-tauri/binaries/`
**Build Tauri desktop app:**
**Build provider binaries (for development):**
```bash
cd tauri
bun run tauri build
bun run build:providers
```
Creates platform-specific installers (`.dmg`, `.msi`, `.AppImage`)
Builds all external provider binaries and installs them to the system provider directory. See [TTS Provider Development](#tts-provider-development) for details.
**Building with local Qwen3-TTS development version:**
If you're actively developing or modifying the Qwen3-TTS library, set the `QWEN_TTS_PATH` environment variable to point to your local clone:
```bash
export QWEN_TTS_PATH=~/path/to/your/Qwen3-TTS
bun run build:server
```
This makes PyInstaller use your local qwen-tts version instead of the pip-installed package. Useful when testing changes to the TTS library before they're published to PyPI or when using an editable install (`pip install -e`).
**Build web app:**
```bash
cd web
bun run build
```
Output in `web/dist/`
### Generate OpenAPI Client
After starting the backend server:
```bash
./scripts/generate-api.sh
```
This downloads the OpenAPI schema and generates the TypeScript client in `app/src/lib/api/`
### Convert Assets to Web Formats
To optimize images and videos for the web, run:
```bash
bun run convert:assets
```
This script:
- Converts PNG → WebP (better compression, same quality)
- Converts MOV → WebM (VP9 codec, smaller file size)
- Processes files in `landing/public/` and `docs/public/`
- **Deletes original files** after successful conversion
**Requirements:** Install `webp` and `ffmpeg`:
```bash
brew install webp ffmpeg
```
> **Note:** Run this before committing new images or videos to keep the repository size small.
## Development Workflow
### 1. Create a Branch
@@ -177,6 +362,7 @@ git push origin feature/your-feature-name
```
Then create a pull request on GitHub with:
- Clear description of changes
- Screenshots (for UI changes)
- Reference to related issues
@@ -322,21 +508,23 @@ Currently, testing is primarily manual. When adding tests:
Releases are managed by maintainers:
1. **Bump version using bumpversion:**
```bash
# Install bumpversion (if not already installed)
pip install bumpversion
# Bump patch version (0.1.0 -> 0.1.1)
bumpversion patch
# Or bump minor version (0.1.0 -> 0.2.0)
bumpversion minor
# Or bump major version (0.1.0 -> 1.0.0)
bumpversion major
```
This automatically:
- Updates version numbers in all files (`tauri.conf.json`, `Cargo.toml`, all `package.json` files, `backend/main.py`)
- Creates a git commit with the version bump
- Creates a git tag (e.g., `v0.1.1`, `v0.2.0`)
@@ -344,6 +532,7 @@ Releases are managed by maintainers:
2. **Update CHANGELOG.md** with release notes
3. **Push commits and tags:**
```bash
git push
git push --tags
+43
View File
@@ -0,0 +1,43 @@
# Base Dockerfile for Voicebox (CPU-only)
# For GPU support, use Dockerfile.cuda
FROM python:3.12-slim
# Prevent interactive prompts during build
ENV DEBIAN_FRONTEND=noninteractive
ENV TZ=UTC
WORKDIR /app
# Install system dependencies
RUN apt-get update && apt-get install -y \
ffmpeg \
curl \
tzdata \
&& rm -rf /var/lib/apt/lists/*
# Copy backend
COPY backend/ /app/backend/
COPY providers/ /app/providers/
# Copy pre-built web UI
COPY web/dist/ /app/web/dist/
# Install Python dependencies (without PyTorch - will be downloaded via provider system)
RUN python -m pip install --upgrade pip && \
pip install --no-cache-dir \
fastapi uvicorn[standard] pydantic sqlalchemy alembic \
librosa soundfile numpy python-multipart Pillow \
huggingface_hub transformers accelerate
# Create data directory for profiles/generations
RUN mkdir -p /app/data
# Health check
HEALTHCHECK --interval=30s --timeout=10s --retries=3 --start-period=40s \
CMD curl -f http://localhost:8000/health || exit 1
EXPOSE 8000
# Run server with web UI
CMD ["uvicorn", "backend.main:app", "--host", "0.0.0.0", "--port", "8000"]
+58
View File
@@ -0,0 +1,58 @@
# Dockerfile for Voicebox with NVIDIA GPU support (CUDA)
FROM nvidia/cuda:12.1.1-runtime-ubuntu22.04
# Prevent interactive prompts during build
ENV DEBIAN_FRONTEND=noninteractive
ENV TZ=UTC
WORKDIR /app
# Install Python 3.12
RUN apt-get update && apt-get install -y \
software-properties-common \
&& add-apt-repository ppa:deadsnakes/ppa \
&& apt-get update && apt-get install -y \
python3.12 \
python3.12-dev \
python3.12-venv \
ffmpeg \
curl \
tzdata \
&& rm -rf /var/lib/apt/lists/*
# Set Python 3.12 as default and bootstrap pip
RUN update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.12 1 && \
update-alternatives --install /usr/bin/python python /usr/bin/python3.12 1 && \
python3.12 -m ensurepip --upgrade && \
python3.12 -m pip install --upgrade pip
# Copy backend
COPY backend/ /app/backend/
COPY providers/ /app/providers/
# Copy pre-built web UI
COPY web/dist/ /app/web/dist/
# Install PyTorch with CUDA support first
RUN pip install --no-cache-dir \
torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121
# Install remaining dependencies
RUN pip install --no-cache-dir \
fastapi uvicorn[standard] pydantic sqlalchemy alembic \
transformers accelerate huggingface_hub \
librosa soundfile numpy python-multipart Pillow \
qwen-tts
# Create data directory
RUN mkdir -p /app/data
# Health check
HEALTHCHECK --interval=30s --timeout=10s --retries=3 --start-period=40s \
CMD curl -f http://localhost:8000/health || exit 1
EXPOSE 8000
# Run server with web UI
CMD ["uvicorn", "backend.main:app", "--host", "0.0.0.0", "--port", "8000"]
+245
View File
@@ -0,0 +1,245 @@
# Voicebox Makefile
# Unix-only (macOS/Linux). Windows users should use WSL.
SHELL := /bin/bash
.DEFAULT_GOAL := help
# Directories
BACKEND_DIR := backend
TAURI_DIR := tauri
WEB_DIR := web
APP_DIR := app
# Python (prefer 3.12, fallback to 3.13, then python3)
PYTHON := $(shell command -v python3.12 2>/dev/null || command -v python3.13 2>/dev/null || echo python3)
VENV := $(CURDIR)/$(BACKEND_DIR)/venv
VENV_BIN := $(VENV)/bin
PIP := $(VENV_BIN)/pip
PYTHON_VENV := $(VENV_BIN)/python
# Colors for output
BLUE := \033[0;34m
GREEN := \033[0;32m
YELLOW := \033[0;33m
NC := \033[0m # No Color
.PHONY: help
help: ## Show this help message
@echo -e "$(BLUE)Voicebox$(NC) - Development Commands"
@echo ""
@grep -E '^[a-zA-Z_-]+:.*?## .*$$' $(MAKEFILE_LIST) | sort | \
awk 'BEGIN {FS = ":.*?## "}; {printf " $(GREEN)%-20s$(NC) %s\n", $$1, $$2}'
# =============================================================================
# SETUP
# =============================================================================
.PHONY: setup setup-js setup-python setup-rust
setup: setup-js setup-python ## Full project setup (all dependencies)
@echo -e "$(GREEN)✓ Setup complete!$(NC)"
@echo -e " Run $(YELLOW)make dev$(NC) to start development servers"
setup-js: ## Install JavaScript dependencies (bun)
@echo -e "$(BLUE)Installing JavaScript dependencies...$(NC)"
bun install
setup-python: $(VENV)/bin/activate ## Set up Python virtual environment and dependencies
@echo -e "$(BLUE)Installing Python dependencies...$(NC)"
$(PIP) install --upgrade pip
$(PIP) install -r $(BACKEND_DIR)/requirements.txt
@if [ "$$(uname -m)" = "arm64" ] && [ "$$(uname)" = "Darwin" ]; then \
echo -e "$(BLUE)Detected Apple Silicon - installing MLX dependencies...$(NC)"; \
$(PIP) install -r $(BACKEND_DIR)/requirements-mlx.txt; \
echo -e "$(GREEN)✓ MLX backend enabled (native Metal acceleration)$(NC)"; \
fi
$(PIP) install git+https://github.com/QwenLM/Qwen3-TTS.git
@echo -e "$(GREEN)✓ Python environment ready$(NC)"
$(VENV)/bin/activate:
@echo -e "$(BLUE)Creating Python virtual environment...$(NC)"
@PY_MINOR=$$($(PYTHON) -c "import sys; print(sys.version_info[1])"); \
if [ "$$PY_MINOR" -gt 13 ]; then \
echo -e "$(YELLOW)Warning: Python 3.$$PY_MINOR detected. ML packages may not be compatible.$(NC)"; \
echo -e "$(YELLOW)Recommended: Use Python 3.12 or 3.13 (brew install [email protected])$(NC)"; \
fi
$(PYTHON) -m venv $(VENV)
setup-rust: ## Install Rust toolchain (if not present)
@command -v rustc >/dev/null 2>&1 || curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
# =============================================================================
# DEVELOPMENT
# =============================================================================
.PHONY: dev dev-backend dev-frontend dev-web kill-dev
dev: ## Start backend + desktop app (parallel)
@echo -e "$(BLUE)Starting development servers...$(NC)"
@echo -e "$(YELLOW)Note: If Tauri fails, run 'make build-server' first or use separate terminals$(NC)"
@trap 'kill 0' EXIT; \
$(MAKE) dev-backend & \
sleep 2 && $(MAKE) dev-frontend & \
wait
dev-backend: ## Start FastAPI backend server
@echo -e "$(BLUE)Starting backend server on http://localhost:17493$(NC)"
$(VENV_BIN)/uvicorn backend.main:app --reload --port 17493
dev-frontend: ## Start Tauri desktop app
@echo -e "$(BLUE)Starting Tauri desktop app...$(NC)"
bun run dev
dev-web: ## Start backend + web app (parallel)
@echo -e "$(BLUE)Starting web development servers...$(NC)"
@trap 'kill 0' EXIT; \
$(MAKE) dev-backend & \
sleep 2 && cd $(WEB_DIR) && bun run dev & \
wait
kill-dev: ## Kill all development processes
@echo -e "$(YELLOW)Killing development processes...$(NC)"
-pkill -f "uvicorn main:app" 2>/dev/null || true
-pkill -f "vite" 2>/dev/null || true
@echo -e "$(GREEN)✓ Processes killed$(NC)"
# =============================================================================
# BUILD
# =============================================================================
.PHONY: build build-server build-tauri build-web
build: build-server build-tauri ## Build everything (server binary + desktop app)
@echo -e "$(GREEN)✓ Build complete!$(NC)"
build-server: ## Build Python server binary
@echo -e "$(BLUE)Building server binary...$(NC)"
PATH="$(VENV_BIN):$$PATH" ./scripts/build-server.sh
build-tauri: ## Build Tauri desktop app
@echo -e "$(BLUE)Building Tauri desktop app...$(NC)"
cd $(TAURI_DIR) && bun run tauri build
build-web: ## Build web app
@echo -e "$(BLUE)Building web app...$(NC)"
cd $(WEB_DIR) && bun run build
@echo -e "$(GREEN)✓ Web build output in $(WEB_DIR)/dist/$(NC)"
# =============================================================================
# DATABASE & API
# =============================================================================
.PHONY: db-init db-reset generate-api
db-init: $(VENV)/bin/activate ## Initialize SQLite database
@echo -e "$(BLUE)Initializing database...$(NC)"
cd $(BACKEND_DIR) && $(PYTHON_VENV) -c "from database import init_db; init_db()"
@echo -e "$(GREEN)✓ Database created at $(BACKEND_DIR)/data/voicebox.db$(NC)"
db-reset: ## Reset database (delete and reinitialize)
@echo -e "$(YELLOW)Resetting database...$(NC)"
rm -f $(BACKEND_DIR)/data/voicebox.db
$(MAKE) db-init
generate-api: ## Generate TypeScript API client from OpenAPI schema
@echo -e "$(BLUE)Generating API client...$(NC)"
@echo -e "$(YELLOW)Note: Backend must be running (make dev-backend)$(NC)"
./scripts/generate-api.sh
@echo -e "$(GREEN)✓ API client generated in $(APP_DIR)/src/lib/api/$(NC)"
# =============================================================================
# CODE QUALITY
# =============================================================================
.PHONY: lint format typecheck check
lint: ## Run linter (Biome)
@echo -e "$(BLUE)Linting...$(NC)"
bun run lint
format: ## Format code (Biome)
@echo -e "$(BLUE)Formatting...$(NC)"
bun run format
typecheck: ## Run TypeScript type checking
@echo -e "$(BLUE)Type checking...$(NC)"
bun run tsc --noEmit
check: ## Run all checks (Biome lint + format + type check)
@echo -e "$(BLUE)Running all checks...$(NC)"
bun run check
@echo -e "$(GREEN)✓ All checks passed$(NC)"
# =============================================================================
# TESTING
# =============================================================================
.PHONY: test test-backend test-frontend
test: test-backend test-frontend ## Run all tests
@echo -e "$(GREEN)✓ All tests passed$(NC)"
test-backend: ## Run Python backend tests (requires pytest)
@echo -e "$(BLUE)Running backend tests...$(NC)"
@if [ -f "$(VENV_BIN)/pytest" ]; then \
cd $(BACKEND_DIR) && $(VENV_BIN)/pytest -v; \
else \
echo -e "$(YELLOW)pytest not installed. Run: $(PIP) install pytest$(NC)"; \
exit 1; \
fi
test-frontend: ## Run frontend tests (requires test script in package.json)
@echo -e "$(BLUE)Running frontend tests...$(NC)"
@if bun run test --help >/dev/null 2>&1; then \
bun run test; \
else \
echo -e "$(YELLOW)No test script configured$(NC)"; \
exit 1; \
fi
# =============================================================================
# LOGS & DEBUGGING
# =============================================================================
.PHONY: logs docs
logs: ## Tail backend logs
@echo -e "$(BLUE)Tailing logs (Ctrl+C to stop)...$(NC)"
tail -f $(BACKEND_DIR)/logs/*.log 2>/dev/null || echo "No log files found"
docs: ## Open API documentation (backend must be running)
@echo -e "$(BLUE)Opening API docs...$(NC)"
open http://localhost:17493/docs 2>/dev/null || xdg-open http://localhost:17493/docs
# =============================================================================
# CLEAN
# =============================================================================
.PHONY: clean clean-python clean-build clean-all
clean: ## Clean build artifacts
@echo -e "$(BLUE)Cleaning build artifacts...$(NC)"
rm -rf $(TAURI_DIR)/src-tauri/target/release
rm -rf $(WEB_DIR)/dist
rm -rf $(APP_DIR)/dist
@echo -e "$(GREEN)✓ Build artifacts cleaned$(NC)"
clean-python: ## Clean Python cache and virtual environment
@echo -e "$(BLUE)Cleaning Python files...$(NC)"
rm -rf $(VENV)
find $(BACKEND_DIR) -type d -name "__pycache__" -exec rm -rf {} + 2>/dev/null || true
find $(BACKEND_DIR) -type f -name "*.pyc" -delete 2>/dev/null || true
@echo -e "$(GREEN)✓ Python environment cleaned$(NC)"
clean-build: ## Clean Rust/Tauri build cache
@echo -e "$(BLUE)Cleaning Rust build cache...$(NC)"
cd $(TAURI_DIR)/src-tauri && cargo clean
@echo -e "$(GREEN)✓ Rust cache cleaned$(NC)"
clean-all: clean clean-python clean-build ## Nuclear clean (everything)
@echo -e "$(BLUE)Cleaning node_modules...$(NC)"
rm -rf node_modules
rm -rf $(APP_DIR)/node_modules
rm -rf $(TAURI_DIR)/node_modules
rm -rf $(WEB_DIR)/node_modules
@echo -e "$(GREEN)✓ Full clean complete$(NC)"
+153 -45
View File
@@ -10,6 +10,21 @@
All running locally on your machine.
</p>
<p align="center">
<a href="https://github.com/jamiepine/voicebox/releases">
<img src="https://img.shields.io/github/downloads/jamiepine/voicebox/total?style=flat&color=blue" alt="Downloads" />
</a>
<a href="https://github.com/jamiepine/voicebox/releases/latest">
<img src="https://img.shields.io/github/v/release/jamiepine/voicebox?style=flat" alt="Release" />
</a>
<a href="https://github.com/jamiepine/voicebox/stargazers">
<img src="https://img.shields.io/github/stars/jamiepine/voicebox?style=flat" alt="Stars" />
</a>
<a href="https://github.com/jamiepine/voicebox/blob/main/LICENSE">
<img src="https://img.shields.io/github/license/jamiepine/voicebox?style=flat" alt="License" />
</a>
</p>
<p align="center">
<a href="https://voicebox.sh">voicebox.sh</a> •
<a href="#download">Download</a> •
@@ -22,7 +37,7 @@
<p align="center">
<a href="https://voicebox.sh">
<img src=".github/assets/screenshot.webp" alt="Voicebox App Screenshot" width="800" />
<img src="landing/public/assets/app-screenshot-1.webp" alt="Voicebox App Screenshot" width="800" />
</a>
</p>
@@ -32,32 +47,68 @@
<br/>
## Why Voicebox?
<p align="center">
<img src="landing/public/assets/app-screenshot-2.webp" alt="Voicebox Screenshot 2" width="800" />
</p>
Voice AI is exploding, but most tools are either cloud-locked, expensive, or a nightmare to set up. Voicebox is different:
<p align="center">
<img src="landing/public/assets/app-screenshot-3.webp" alt="Voicebox Screenshot 3" width="800" />
</p>
- **100% Local** — Your voice data never leaves your machine
- **Lightweight** — No bloated Electron, native Tauri performance
- **Fast** — Near-instant on CUDA, optimized for Apple Silicon
- **Flexible** — Use the app, integrate the API, or both
- **Open Source** — No subscriptions, no limits, no lock-in
<br/>
Built with **Tauri** (Rust), **TypeScript**, **React**, and **Python**. Native performance meets modern DX.
## What is Voicebox?
Voicebox is a **local-first voice cloning studio** with DAW-like features for professional voice synthesis. Think of it as a **local, free and open-source alternative to ElevenLabs** — download models, clone voices, and generate speech entirely on your machine.
Unlike cloud services that lock your voice data behind subscriptions, Voicebox gives you:
- **Complete privacy** — models and voice data stay on your machine
- **Professional tools** — multi-track timeline editor, audio trimming, conversation mixing
- **Model flexibility** — currently powered by Qwen3-TTS, with support for XTTS, Bark, and other models coming soon
- **API-first** — use the desktop app or integrate voice synthesis into your own projects
- **Native performance** — built with Tauri (Rust), not Electron
- **Super fast on Mac** — MLX backend with native Metal acceleration for 4-5x faster inference on Apple Silicon
Download a voice model, clone any voice from a few seconds of audio, and compose multi-voice projects with studio-grade editing tools. No Python install required, no cloud dependency, no limits.
---
## Download
Voicebox is available now for macOS and Windows.
### Desktop App
| Platform | Download |
|----------|----------|
| macOS (Apple Silicon) | [voicebox_aarch64.app.tar.gz](https://github.com/jamiepine/voicebox/releases/download/v0.1.0/voicebox_aarch64.app.tar.gz) |
| macOS (Intel) | [voicebox_x64.app.tar.gz](https://github.com/jamiepine/voicebox/releases/download/v0.1.0/voicebox_x64.app.tar.gz) |
| Windows (MSI) | [voicebox_0.1.0_x64_en-US.msi](https://github.com/jamiepine/voicebox/releases/download/v0.1.0/voicebox_0.1.0_x64_en-US.msi) |
| Windows (Setup) | [voicebox_0.1.0_x64-setup.exe](https://github.com/jamiepine/voicebox/releases/download/v0.1.0/voicebox_0.1.0_x64-setup.exe) |
| Platform | Download |
| --------------------- | --------------------------------------------------------------------------------------------------------------------------- |
| macOS (Apple Silicon) | [Download latest release](https://github.com/jamiepine/voicebox/releases/latest) |
| macOS (Intel) | [Download latest release](https://github.com/jamiepine/voicebox/releases/latest) |
| Windows (MSI) | [Download latest release](https://github.com/jamiepine/voicebox/releases/latest) |
| Windows (Setup) | [Download latest release](https://github.com/jamiepine/voicebox/releases/latest) |
| Linux (AppImage) | [Download latest release](https://github.com/jamiepine/voicebox/releases/latest) |
| Linux (Deb) | [Download latest release](https://github.com/jamiepine/voicebox/releases/latest) |
> **Linux builds coming soon** — Currently blocked by GitHub runner disk space limitations.
### Docker
Run Voicebox with the web UI in Docker - perfect for servers and headless deployments:
```bash
# CPU-only (supports amd64 and arm64)
docker run -p 8000:8000 -v voicebox-data:/app/data \
ghcr.io/jamiepine/voicebox:latest
# NVIDIA GPU (recommended for performance)
docker run --gpus all -p 8000:8000 -v voicebox-data:/app/data \
ghcr.io/jamiepine/voicebox:latest-cuda
```
Or use Docker Compose:
```bash
docker compose up -d
```
Open http://localhost:8000 to access the web UI.
See [Docker Deployment Guide](docs/overview/docker.mdx) for cloud deployments, GPU setup, and more.
---
@@ -70,11 +121,13 @@ Powered by Alibaba's **Qwen3-TTS** — a breakthrough model that achieves near-p
- **Instant cloning** — Upload a sample, get a voice profile
- **High fidelity** — Natural prosody, emotion, and cadence
- **Multi-language** — English, Chinese, and more coming
- **Lightning fast on Mac** — MLX backend leverages Apple Silicon's Neural Engine for super fast generation
### Voice Profile Management
- **Create profiles** from audio files or record directly in-app
- **Import/Export** profiles to share or backup
- **Multi-sample support** — combine multiple samples for higher quality cloning
- **Organize** with descriptions and language tags
### Speech Generation
@@ -83,9 +136,19 @@ Powered by Alibaba's **Qwen3-TTS** — a breakthrough model that achieves near-p
- **Batch generation** for long-form content
- **Smart caching** — regenerate instantly with voice prompt caching
### Stories Editor
Create multi-voice narratives, podcasts, and conversations with a timeline-based editor.
- **Multi-track composition** — arrange multiple voice tracks in a single project
- **Inline audio editing** — trim and split clips directly in the timeline
- **Auto-playback** — preview stories with synchronized playhead
- **Voice mixing** — build conversations with multiple participants
### Recording & Transcription
- **In-app recording** with waveform visualization
- **System audio capture** — record desktop audio on macOS and Windows
- **Automatic transcription** powered by Whisper
- **Export recordings** in multiple formats
@@ -97,9 +160,10 @@ Powered by Alibaba's **Qwen3-TTS** — a breakthrough model that achieves near-p
### Flexible Deployment
- **Local mode** — Everything runs on your machine
- **Remote mode** — Connect to a GPU server on your network
- **One-click server** — Turn any machine into a Voicebox server
- **Desktop app** — Native apps for macOS, Windows, and Linux
- **Docker** — Deploy to servers with the web UI included
- **Remote mode** — Connect desktop app to a remote GPU server
- **Cloud ready** — Deploy to AWS, GCP, DigitalOcean, or any cloud provider
---
@@ -109,17 +173,17 @@ Voicebox exposes a full REST API, so you can integrate voice synthesis into your
```bash
# Generate speech
curl -X POST http://localhost:8000/api/generate \
curl -X POST http://localhost:8000/generate \
-H "Content-Type: application/json" \
-d '{"text": "Hello world", "profile_id": "abc123"}'
-d '{"text": "Hello world", "profile_id": "abc123", "language": "en"}'
# List voice profiles
curl http://localhost:8000/api/profiles
curl http://localhost:8000/profiles
# Create a profile from audio
curl -X POST http://localhost:8000/api/profiles \
-F "[email protected]" \
-F "name=My Voice"
# Create a profile
curl -X POST http://localhost:8000/profiles \
-H "Content-Type: application/json" \
-d '{"name": "My Voice", "language": "en"}'
```
**Use cases:**
@@ -136,16 +200,17 @@ Full API documentation available at `http://localhost:8000/docs` when running.
## Tech Stack
| Layer | Technology |
|-------|------------|
| Desktop App | Tauri (Rust) |
| Frontend | React, TypeScript, Tailwind CSS |
| State | Zustand, React Query |
| Backend | FastAPI (Python) |
| Voice Model | Qwen3-TTS |
| Transcription | Whisper |
| Database | SQLite |
| Audio | WaveSurfer.js, librosa |
| Layer | Technology |
| ---------------- | --------------------------------------------------- |
| Desktop App | Tauri (Rust) |
| Frontend | React, TypeScript, Tailwind CSS |
| State | Zustand, React Query |
| Backend | FastAPI (Python) |
| Voice Model | Qwen3-TTS (PyTorch or MLX) |
| Transcription | Whisper (PyTorch or MLX) |
| Inference Engine | MLX (Apple Silicon) / PyTorch (Windows/Linux/Intel) |
| Database | SQLite |
| Audio | WaveSurfer.js, librosa |
**Why this stack?**
@@ -153,6 +218,26 @@ Full API documentation available at `http://localhost:8000/docs` when running.
- **FastAPI** — Async Python with automatic OpenAPI schema generation
- **Type-safe end-to-end** — Generated TypeScript client from OpenAPI spec
### TTS Provider Architecture
Voicebox uses a modular provider system to support different inference backends:
- **`apple-mlx`** — Bundled with macOS Apple Silicon builds
- Uses MLX with native Metal acceleration (4-5x faster)
- Works out of the box, no download required
- **`pytorch-cpu`** — Universal CPU provider (bundled or downloaded)
- Bundled with Windows and macOS Intel builds
- Downloaded on first use for Linux (~300MB)
- **`pytorch-cuda`** — Optional NVIDIA GPU-accelerated provider
- Windows/Linux only (~2.4GB)
- 4-5x faster inference on CUDA-capable GPUs
macOS and Windows builds work out of the box with bundled providers. Linux users download a provider on first launch. The app automatically detects your hardware and recommends the best option. All downloadable providers are distributed via Cloudflare R2 for fast, global delivery.
---
## Roadmap
@@ -161,13 +246,13 @@ Voicebox is the beginning of something bigger. Here's what's coming:
### Coming Soon
| Feature | Description |
|---------|-------------|
| **Real-time Synthesis** | Stream audio as it generates, word by word |
| **Conversation Mode** | Multi-speaker dialogues with automatic turn-taking |
| **Voice Effects** | Pitch shift, reverb, M3GAN-style effects |
| **Timeline Editor** | Audio studio with word-level precision editing |
| **More Models** | XTTS, Bark, and other open-source voice models |
| Feature | Description |
| ----------------------- | -------------------------------------------------- |
| **Real-time Synthesis** | Stream audio as it generates, word by word |
| **Conversation Mode** | Multi-speaker dialogues with automatic turn-taking |
| **Voice Effects** | Pitch shift, reverb, M3GAN-style effects |
| **Timeline Editor** | Audio studio with word-level precision editing |
| **More Models** | XTTS, Bark, and other open-source voice models |
### Future Vision
@@ -184,8 +269,26 @@ Voicebox aims to be the **one-stop shop for everything voice** — cloning, synt
See [CONTRIBUTING.md](CONTRIBUTING.md) for detailed setup and contribution guidelines.
**Using the Makefile (recommended):** Run `make help` to see all available commands for setup, development, building, and testing.
### Quick Start
**With Makefile (Unix/macOS/Linux):**
```bash
# Clone the repo
git clone https://github.com/voicebox-sh/voicebox.git
cd voicebox
# Setup everything
make setup
# Start development
make dev
```
**Manual setup (all platforms):**
```bash
# Clone the repo
git clone https://github.com/voicebox-sh/voicebox.git
@@ -201,7 +304,12 @@ cd backend && pip install -r requirements.txt && cd ..
bun run dev
```
**Prerequisites:** [Bun](https://bun.sh), [Rust](https://rustup.rs), [Python 3.11+](https://python.org). CUDA-capable GPU recommended (CPU inference supported but slower).
**Prerequisites:** [Bun](https://bun.sh), [Rust](https://rustup.rs), [Python 3.11+](https://python.org).
**Performance:**
- **Apple Silicon (M1/M2/M3)**: Uses MLX backend with native Metal acceleration for 4-5x faster inference
- **Windows/Linux/Intel Mac**: Uses PyTorch backend (CUDA GPU recommended, CPU supported but slower)
### Project Structure
+7 -2
View File
@@ -1,6 +1,6 @@
{
"name": "@voicebox/app",
"version": "0.1.6",
"version": "0.1.13",
"private": true,
"type": "module",
"scripts": {
@@ -17,6 +17,10 @@
"@dnd-kit/sortable": "^10.0.0",
"@dnd-kit/utilities": "^3.2.2",
"@hookform/resolvers": "^3.9.0",
"@hugeicons/core-free-icons": "^3.1.1",
"@hugeicons/react": "^1.1.4",
"@iconify-json/svg-spinners": "^1.2.4",
"@iconify/react": "^6.0.2",
"@radix-ui/react-alert-dialog": "^1.1.1",
"@radix-ui/react-avatar": "^1.1.0",
"@radix-ui/react-dialog": "^1.1.1",
@@ -24,6 +28,7 @@
"@radix-ui/react-label": "^2.1.0",
"@radix-ui/react-popover": "^1.1.1",
"@radix-ui/react-progress": "^1.1.0",
"@radix-ui/react-radio-group": "^1.2.0",
"@radix-ui/react-scroll-area": "^1.1.0",
"@radix-ui/react-select": "^2.1.1",
"@radix-ui/react-separator": "^1.1.0",
@@ -43,11 +48,11 @@
"clsx": "^2.1.1",
"date-fns": "^3.6.0",
"framer-motion": "^12.29.0",
"lucide-react": "^0.454.0",
"motion": "^12.29.0",
"react": "^18.3.0",
"react-dom": "^18.3.0",
"react-hook-form": "^7.53.0",
"react-sound-visualizer": "^1.4.0",
"tailwind-merge": "^2.5.4",
"wavesurfer.js": "^7.0.0",
"zod": "^3.23.8",
+35 -21
View File
@@ -1,16 +1,12 @@
import { useEffect, useRef, useState } from 'react';
import { RouterProvider } from '@tanstack/react-router';
import { useEffect, useRef, useState } from 'react';
import voiceboxLogo from '@/assets/voicebox-logo.png';
import ShinyText from '@/components/ShinyText';
import { TitleBarDragRegion } from '@/components/TitleBarDragRegion';
import { useAutoUpdater } from '@/hooks/useAutoUpdater';
import { TOP_SAFE_AREA_PADDING } from '@/lib/constants/ui';
import {
isTauri,
setKeepServerRunning,
setupWindowCloseHandler,
startServer,
} from '@/lib/tauri';
import { cn } from '@/lib/utils/cn';
import { usePlatform } from '@/platform/PlatformContext';
import { router } from '@/router';
import { useServerStore } from '@/stores/serverStore';
@@ -38,29 +34,45 @@ const LOADING_MESSAGES = [
];
function App() {
const platform = usePlatform();
const [serverReady, setServerReady] = useState(false);
const [loadingMessageIndex, setLoadingMessageIndex] = useState(0);
const serverStartingRef = useRef(false);
// Automatically check for app updates on startup and show toast notifications
useAutoUpdater(true);
// Sync stored setting to Rust on startup
useEffect(() => {
if (isTauri()) {
if (platform.metadata.isTauri) {
const keepRunning = useServerStore.getState().keepServerRunningOnClose;
setKeepServerRunning(keepRunning).catch((error) => {
platform.lifecycle.setKeepServerRunning(keepRunning).catch((error) => {
console.error('Failed to sync initial setting to Rust:', error);
});
}
}, []);
// Empty dependency array - platform is stable from context, only run once
// eslint-disable-next-line react-hooks/exhaustive-deps
}, [platform.metadata.isTauri, platform.lifecycle]);
// Setup lifecycle callbacks
useEffect(() => {
platform.lifecycle.onServerReady = () => {
setServerReady(true);
};
// Empty dependency array - platform is stable from context, only run once
// eslint-disable-next-line react-hooks/exhaustive-deps
}, [platform.lifecycle]);
// Setup window close handler and auto-start server when running in Tauri (production only)
useEffect(() => {
if (!isTauri()) {
if (!platform.metadata.isTauri) {
setServerReady(true); // Web assumes server is running
return;
}
// Setup window close handler to check setting and stop server if needed
// This works in both dev and prod, but will only stop server if it was started by the app
setupWindowCloseHandler().catch((error) => {
platform.lifecycle.setupWindowCloseHandler().catch((error) => {
console.error('Failed to setup window close handler:', error);
});
@@ -70,8 +82,7 @@ function App() {
console.log('Dev mode: Skipping auto-start of server (run it separately)');
setServerReady(true); // Mark as ready so UI doesn't show loading screen
// Mark that server was not started by app (so we don't try to stop it on close)
// @ts-expect-error - adding property to window
window.__voiceboxServerStartedByApp = false;
(window as any).__voiceboxServerStartedByApp = false;
return;
}
@@ -83,19 +94,20 @@ function App() {
serverStartingRef.current = true;
console.log('Production mode: Starting bundled server...');
startServer(false)
platform.lifecycle
.startServer(false)
.then((serverUrl) => {
console.log('Server is ready at:', serverUrl);
// Update the server URL in the store with the dynamically assigned port
useServerStore.getState().setServerUrl(serverUrl);
setServerReady(true);
// Mark that we started the server (so we know to stop it on close)
window.__voiceboxServerStartedByApp = true;
(window as any).__voiceboxServerStartedByApp = true;
})
.catch((error) => {
console.error('Failed to auto-start server:', error);
serverStartingRef.current = false;
window.__voiceboxServerStartedByApp = false;
(window as any).__voiceboxServerStartedByApp = false;
});
// Cleanup: stop server on actual unmount (not StrictMode remount)
@@ -104,11 +116,13 @@ function App() {
// Window close event handles server shutdown based on setting
serverStartingRef.current = false;
};
}, []);
// Empty dependency array - platform is stable from context, only run once
// eslint-disable-next-line react-hooks/exhaustive-deps
}, [platform.metadata.isTauri, platform.lifecycle]);
// Cycle through loading messages every 3 seconds
useEffect(() => {
if (!isTauri() || serverReady) {
if (!platform.metadata.isTauri || serverReady) {
return;
}
@@ -117,10 +131,10 @@ function App() {
}, 3000);
return () => clearInterval(interval);
}, [serverReady]);
}, [serverReady, platform.metadata.isTauri]);
// Show loading screen while server is starting in Tauri
if (isTauri() && !serverReady) {
if (platform.metadata.isTauri && !serverReady) {
return (
<div
className={cn(
+48 -45
View File
@@ -1,17 +1,18 @@
import { useQuery } from '@tanstack/react-query';
import { invoke } from '@tauri-apps/api/core';
import { Pause, Play, Repeat, Volume2, VolumeX, X } from 'lucide-react';
import { HugeiconsIcon } from '@hugeicons/react';
import { PauseIcon, PlayIcon, RepeatIcon, VolumeHighIcon, VolumeMuteIcon, Cancel01Icon } from '@hugeicons/core-free-icons';
import { useEffect, useMemo, useRef, useState } from 'react';
import WaveSurfer from 'wavesurfer.js';
import { Button } from '@/components/ui/button';
import { Slider } from '@/components/ui/slider';
import { apiClient } from '@/lib/api/client';
import { isTauri } from '@/lib/tauri';
import { formatAudioDuration } from '@/lib/utils/audio';
import { debug } from '@/lib/utils/debug';
import { usePlayerStore } from '@/stores/playerStore';
import { usePlatform } from '@/platform/PlatformContext';
export function AudioPlayer() {
const platform = usePlatform();
const {
audioUrl,
audioId,
@@ -39,7 +40,7 @@ export function AudioPlayer() {
if (!profileId) return { channel_ids: [] };
return apiClient.getProfileChannels(profileId);
},
enabled: !!profileId && isTauri(),
enabled: !!profileId && platform.metadata.isTauri,
});
const { data: channels } = useQuery({
@@ -50,7 +51,7 @@ export function AudioPlayer() {
// Determine if we should use native playback
const useNativePlayback = useMemo(() => {
if (!isTauri() || !profileChannels || !channels) {
if (!platform.metadata.isTauri || !profileChannels || !channels) {
return false;
}
@@ -195,7 +196,7 @@ export function AudioPlayer() {
let runtimeProfileChannels = null;
let runtimeChannels = null;
if (isTauri() && currentProfileId) {
if (platform.metadata.isTauri && currentProfileId) {
try {
runtimeProfileChannels = await apiClient.getProfileChannels(currentProfileId);
debug.log('Runtime profileChannels:', runtimeProfileChannels);
@@ -210,7 +211,7 @@ export function AudioPlayer() {
}
debug.log('Auto-play check:', {
isTauri: isTauri(),
isTauri: platform.metadata.isTauri,
currentAudioUrl,
currentProfileId,
hasProfileChannels: !!runtimeProfileChannels,
@@ -218,7 +219,7 @@ export function AudioPlayer() {
});
if (
isTauri() &&
platform.metadata.isTauri &&
currentAudioUrl &&
currentProfileId &&
runtimeProfileChannels &&
@@ -229,7 +230,7 @@ export function AudioPlayer() {
// Stop any existing native playback first
if (isUsingNativePlaybackRef.current) {
try {
await invoke('stop_audio_playback');
platform.audio.stopPlayback();
debug.log('Stopped existing native playback before starting new one');
} catch (error) {
debug.error('Failed to stop existing playback:', error);
@@ -279,11 +280,8 @@ export function AudioPlayer() {
// Play via native audio
debug.log('Invoking play_audio_to_devices...');
try {
const result = await invoke('play_audio_to_devices', {
audioData: Array.from(audioData),
deviceIds: deviceIds,
});
debug.log('play_audio_to_devices completed successfully, result:', result);
await platform.audio.playToDevices(audioData, deviceIds);
debug.log('play_audio_to_devices completed successfully');
// Mark that we're using native playback
isUsingNativePlaybackRef.current = true;
@@ -357,14 +355,22 @@ export function AudioPlayer() {
}
}
// Standard WaveSurfer auto-play
// Use a small delay to ensure audio element is fully ready
setTimeout(() => {
wavesurfer.play().catch((error) => {
debug.error('Failed to autoplay:', error);
// Don't show error for autoplay failures (browser restrictions)
});
}, 100);
// Only auto-play if shouldAutoPlay flag is set (user explicitly clicked to play)
const shouldAutoPlayNow = usePlayerStore.getState().shouldAutoPlay;
if (shouldAutoPlayNow) {
// Clear the flag first
usePlayerStore.getState().clearAutoPlayFlag();
// Use a small delay to ensure audio element is fully ready
setTimeout(() => {
wavesurfer.play().catch((error) => {
debug.error('Failed to autoplay:', error);
// Don't show error for autoplay failures (browser restrictions)
});
}, 100);
} else {
debug.log('Skipping auto-play - shouldAutoPlay is false');
}
});
// Handle play/pause
@@ -454,7 +460,7 @@ export function AudioPlayer() {
// Use double requestAnimationFrame to ensure DOM is fully rendered
let rafId1: number;
let rafId2: number;
let timeoutId: number | null = null;
let timeoutId: ReturnType<typeof setTimeout> | null = null;
rafId1 = requestAnimationFrame(() => {
rafId2 = requestAnimationFrame(() => {
@@ -508,15 +514,13 @@ export function AudioPlayer() {
}
// Stop native playback if it was active
if (isUsingNativePlaybackRef.current && isTauri()) {
(async () => {
try {
await invoke('stop_audio_playback');
debug.log('Stopped native audio playback');
} catch (error) {
debug.error('Failed to stop native playback:', error);
}
})();
if (isUsingNativePlaybackRef.current && platform.metadata.isTauri) {
try {
platform.audio.stopPlayback();
debug.log('Stopped native audio playback');
} catch (error) {
debug.error('Failed to stop native playback:', error);
}
}
// Reset native playback flag when loading new audio
@@ -703,7 +707,7 @@ export function AudioPlayer() {
if (isPlaying) {
// Pause: stop native playback and pause WaveSurfer visualization
try {
await invoke('stop_audio_playback');
platform.audio.stopPlayback();
debug.log('Stopped native audio playback');
} catch (error) {
debug.error('Failed to stop native playback:', error);
@@ -716,7 +720,7 @@ export function AudioPlayer() {
try {
// Stop any existing native playback first
try {
await invoke('stop_audio_playback');
platform.audio.stopPlayback();
} catch (_error) {
// Ignore errors when stopping (might not be playing)
debug.log('No existing playback to stop');
@@ -734,10 +738,7 @@ export function AudioPlayer() {
const audioData = new Uint8Array(await response.arrayBuffer());
// Play via native audio
await invoke('play_audio_to_devices', {
audioData: Array.from(audioData),
deviceIds: deviceIds,
});
await platform.audio.playToDevices(audioData, deviceIds);
// Mark that we're using native playback
isUsingNativePlaybackRef.current = true;
@@ -798,10 +799,12 @@ export function AudioPlayer() {
const handleClose = () => {
// Stop any native playback
if (isUsingNativePlaybackRef.current && isTauri()) {
invoke('stop_audio_playback').catch((error) => {
if (isUsingNativePlaybackRef.current && platform.metadata.isTauri) {
try {
platform.audio.stopPlayback();
} catch (error) {
debug.error('Failed to stop native playback:', error);
});
}
}
// Stop WaveSurfer
if (wavesurferRef.current) {
@@ -830,7 +833,7 @@ export function AudioPlayer() {
className="shrink-0"
title={duration === 0 && !isLoading ? 'Audio not loaded' : ''}
>
{isPlaying ? <Pause className="h-5 w-5" /> : <Play className="h-5 w-5" />}
{isPlaying ? <HugeiconsIcon icon={PauseIcon} size={20} className="h-5 w-5" /> : <HugeiconsIcon icon={PlayIcon} size={20} className="h-5 w-5" />}
</Button>
{/* Waveform */}
@@ -871,7 +874,7 @@ export function AudioPlayer() {
className={isLooping ? 'text-primary' : ''}
title="Toggle loop"
>
<Repeat className="h-4 w-4" />
<HugeiconsIcon icon={RepeatIcon} size={16} className="h-4 w-4" />
</Button>
{/* Volume Control */}
@@ -882,7 +885,7 @@ export function AudioPlayer() {
onClick={() => setVolume(volume > 0 ? 0 : 1)}
className="h-8 w-8"
>
{volume > 0 ? <Volume2 className="h-4 w-4" /> : <VolumeX className="h-4 w-4" />}
{volume > 0 ? <HugeiconsIcon icon={VolumeHighIcon} size={16} className="h-4 w-4" /> : <HugeiconsIcon icon={VolumeMuteIcon} size={16} className="h-4 w-4" />}
</Button>
<Slider
value={[volume * 100]}
@@ -901,7 +904,7 @@ export function AudioPlayer() {
className="shrink-0"
title="Close player"
>
<X className="h-5 w-5" />
<HugeiconsIcon icon={Cancel01Icon} size={20} className="h-5 w-5" />
</Button>
</div>
</div>
+20 -20
View File
@@ -1,6 +1,6 @@
import { useMutation, useQuery, useQueryClient } from '@tanstack/react-query';
import { invoke } from '@tauri-apps/api/core';
import { Check, CheckCircle2, Edit, Plus, Speaker, Trash2 } from 'lucide-react';
import { HugeiconsIcon } from '@hugeicons/react';
import { CheckmarkCircle01Icon, CheckmarkCircle02Icon, Edit01Icon, Add01Icon, SpeakerIcon, Delete01Icon } from '@hugeicons/core-free-icons';
import { useState } from 'react';
import { Badge } from '@/components/ui/badge';
import { Button } from '@/components/ui/button';
@@ -23,9 +23,9 @@ import {
} from '@/components/ui/select';
import { apiClient } from '@/lib/api/client';
import { BOTTOM_SAFE_AREA_PADDING } from '@/lib/constants/ui';
import { isTauri } from '@/lib/tauri';
import { cn } from '@/lib/utils/cn';
import { usePlayerStore } from '@/stores/playerStore';
import { usePlatform } from '@/platform/PlatformContext';
interface AudioDevice {
id: string;
@@ -34,6 +34,7 @@ interface AudioDevice {
}
export function AudioTab() {
const platform = usePlatform();
const [createDialogOpen, setCreateDialogOpen] = useState(false);
const [editingChannel, setEditingChannel] = useState<string | null>(null);
const [selectedChannelId, setSelectedChannelId] = useState<string | null>(null);
@@ -49,18 +50,17 @@ export function AudioTab() {
const { data: devices, isLoading: devicesLoading } = useQuery({
queryKey: ['audio-devices'],
queryFn: async () => {
if (!isTauri()) {
if (!platform.metadata.isTauri) {
return [];
}
try {
const result = await invoke<AudioDevice[]>('list_audio_output_devices');
return result;
return await platform.audio.listOutputDevices();
} catch (error) {
console.error('Failed to list audio devices:', error);
return [];
}
},
enabled: isTauri(),
enabled: platform.metadata.isTauri,
});
const { data: profiles } = useQuery({
@@ -136,7 +136,7 @@ export function AudioTab() {
<div className="flex items-center justify-between mb-6 shrink-0">
<h2 className="text-2xl font-bold">Audio Channels</h2>
<Button onClick={() => setCreateDialogOpen(true)}>
<Plus className="h-4 w-4 mr-2" />
<HugeiconsIcon icon={Add01Icon} size={16} className="h-4 w-4 mr-2" />
New Channel
</Button>
</div>
@@ -151,13 +151,13 @@ export function AudioTab() {
>
{allChannels.length === 0 ? (
<div className="flex flex-col items-center justify-center py-12 border-2 border-dashed border-muted rounded-md">
<Speaker className="h-12 w-12 text-muted-foreground mb-4" />
<HugeiconsIcon icon={SpeakerIcon} size={48} className="h-12 w-12 text-muted-foreground mb-4" />
<p className="text-muted-foreground mb-4">
No audio channels yet. Create your first channel to route voices to specific
devices.
</p>
<Button onClick={() => setCreateDialogOpen(true)}>
<Plus className="h-4 w-4 mr-2" />
<HugeiconsIcon icon={Add01Icon} size={16} className="h-4 w-4 mr-2" />
Create Channel
</Button>
</div>
@@ -179,7 +179,7 @@ export function AudioTab() {
<div className="flex-1 min-w-0">
<div className="flex items-center gap-2 mb-3">
<div className="h-8 w-8 rounded-lg bg-muted flex items-center justify-center shrink-0">
<Speaker className="h-4 w-4 text-muted-foreground" />
<HugeiconsIcon icon={SpeakerIcon} size={16} className="h-4 w-4 text-muted-foreground" />
</div>
<div className="flex items-center gap-2 min-w-0">
<h3 className="font-semibold text-base truncate">{channel.name}</h3>
@@ -236,7 +236,7 @@ export function AudioTab() {
setEditingChannel(channel.id);
}}
>
<Edit className="h-4 w-4" />
<HugeiconsIcon icon={Edit01Icon} size={16} className="h-4 w-4" />
</Button>
<Button
variant="ghost"
@@ -249,7 +249,7 @@ export function AudioTab() {
}
}}
>
<Trash2 className="h-4 w-4" />
<HugeiconsIcon icon={Delete01Icon} size={16} className="h-4 w-4" />
</Button>
</div>
)}
@@ -326,10 +326,10 @@ export function AudioTab() {
isConnected ? 'bg-accent border-accent' : 'border-muted-foreground/30',
)}
>
{isConnected && <Check className="h-3 w-3 text-accent-foreground" />}
{isConnected && <HugeiconsIcon icon={CheckmarkCircle01Icon} size={12} className="h-3 w-3 text-accent-foreground" />}
</div>
) : device.is_default ? (
<CheckCircle2 className="h-4 w-4 text-primary shrink-0" />
<HugeiconsIcon icon={CheckmarkCircle02Icon} size={16} className="h-4 w-4 text-primary shrink-0" />
) : null}
<span className={cn('truncate flex-1', device.is_default && 'font-medium')}>
{device.name}
@@ -340,9 +340,9 @@ export function AudioTab() {
</div>
) : (
<div className="flex flex-col items-center justify-center py-12 border-2 border-dashed border-muted rounded-md">
<CheckCircle2 className="h-12 w-12 text-muted-foreground mb-4" />
<HugeiconsIcon icon={CheckmarkCircle02Icon} size={48} className="h-12 w-12 text-muted-foreground mb-4" />
<p className="text-muted-foreground text-center">
{isTauri() ? 'No audio devices found' : 'Audio device selection requires Tauri'}
{platform.metadata.isTauri ? 'No audio devices found' : 'Audio device selection requires Tauri'}
</p>
</div>
)}
@@ -495,7 +495,7 @@ function CreateChannelDialog({ open, onOpenChange, devices, onCreate }: CreateCh
setSelectedDevices(selectedDevices.filter((id) => id !== deviceId))
}
>
<Trash2 className="h-3 w-3" />
<HugeiconsIcon icon={Delete01Icon} size={12} className="h-3 w-3" />
</Button>
</div>
);
@@ -603,7 +603,7 @@ function EditChannelDialog({
setSelectedDevices(selectedDevices.filter((id) => id !== deviceId))
}
>
<Trash2 className="h-3 w-3" />
<HugeiconsIcon icon={Delete01Icon} size={12} className="h-3 w-3" />
</Button>
</div>
);
@@ -649,7 +649,7 @@ function EditChannelDialog({
setSelectedVoices(selectedVoices.filter((id) => id !== profileId))
}
>
<Trash2 className="h-3 w-3" />
<HugeiconsIcon icon={Delete01Icon} size={12} className="h-3 w-3" />
</Button>
</div>
);
@@ -1,6 +1,8 @@
import { SparklesIcon, TextSquareIcon } from '@hugeicons/core-free-icons';
import { HugeiconsIcon } from '@hugeicons/react';
import { Icon } from '@iconify/react';
import { useMatchRoute } from '@tanstack/react-router';
import { AnimatePresence, motion } from 'framer-motion';
import { Loader2, MessageSquare, Sparkles } from 'lucide-react';
import { useEffect, useRef, useState } from 'react';
import { Button } from '@/components/ui/button';
import { Form, FormControl, FormField, FormItem, FormMessage } from '@/components/ui/form';
@@ -112,9 +114,6 @@ export function FloatingGenerateBox({
}
}, [selectedProfileId, profiles, setSelectedProfileId]);
// Get current form value to trigger resize when it changes
const formValue = form.watch(isInstructMode ? 'instruct' : 'text');
// Auto-resize textarea based on content (only when expanded)
useEffect(() => {
if (!isExpanded) {
@@ -190,80 +189,134 @@ export function FloatingGenerateBox({
}}
>
<motion.div
className="bg-background/30 backdrop-blur-2xl border border-accent/20 rounded-[2rem] shadow-2xl hover:bg-background/40 hover:border-accent/20 transition-all duration-300 overflow-hidden p-3"
className="bg-background/30 backdrop-blur-2xl border border-accent/20 rounded-[2rem] shadow-2xl hover:bg-background/40 hover:border-accent/20 transition-all duration-300 p-3"
transition={{ duration: 0.6, ease: 'easeInOut' }}
>
<Form {...form}>
<form onSubmit={form.handleSubmit(onSubmit)}>
<div className="flex gap-2">
<motion.div className="flex-1" transition={{ duration: 0.3, ease: 'easeOut' }}>
{isInstructMode && (
<span className="text-xs text-accent font-medium mb-1 block">
Delivery instructions:
</span>
)}
<FormField
control={form.control}
name={isInstructMode ? 'instruct' : 'text'}
render={({ field }) => (
<FormItem>
<FormControl>
<motion.div
animate={{
height: isExpanded ? 'auto' : '32px',
}}
transition={{ duration: 0.15, ease: 'easeOut' }}
style={{ overflow: 'hidden' }}
>
<Textarea
{...field}
ref={(node: HTMLTextAreaElement | null) => {
// Store ref for auto-resize
textareaRef.current = node;
// Forward ref to react-hook-form
if (typeof field.ref === 'function') {
field.ref(node);
}
<motion.div
className={cn('flex-1', isExpanded && 'mr-12')}
transition={{ duration: 0.3, ease: 'easeOut' }}
>
{/* Text field - hidden when in instruct mode */}
<div style={{ display: isInstructMode ? 'none' : 'block' }}>
<FormField
control={form.control}
name="text"
render={({ field }) => (
<FormItem>
<FormControl>
<motion.div
animate={{
height: isExpanded ? 'auto' : '32px',
}}
placeholder={
isInstructMode
? 'Add delivery instructions...'
: isStoriesRoute && currentStory
transition={{ duration: 0.15, ease: 'easeOut' }}
style={{ overflow: 'hidden' }}
>
<Textarea
{...field}
ref={(node: HTMLTextAreaElement | null) => {
// Store ref for auto-resize (only for active field)
if (!isInstructMode) {
textareaRef.current = node;
}
// Forward ref to react-hook-form
if (typeof field.ref === 'function') {
field.ref(node);
}
}}
placeholder={
isStoriesRoute && currentStory
? `Generate speech for "${currentStory.name}"...`
: selectedProfile
? `Generate speech using ${selectedProfile.name}...`
: 'Select a voice profile above...'
}
className="resize-none bg-transparent border-none focus-visible:ring-0 focus-visible:ring-offset-0 focus:outline-none focus:ring-0 outline-none ring-0 rounded-2xl text-sm placeholder:text-muted-foreground/60 w-full"
style={{
minHeight: isExpanded ? '100px' : '32px',
maxHeight: '300px',
}
className="resize-none bg-transparent border-none focus-visible:ring-0 focus-visible:ring-offset-0 focus:outline-none focus:ring-0 outline-none ring-0 rounded-2xl text-sm placeholder:text-muted-foreground/60 w-full"
style={{
minHeight: isExpanded ? '100px' : '32px',
maxHeight: '300px',
}}
disabled={!selectedProfileId}
onClick={() => setIsExpanded(true)}
onFocus={() => setIsExpanded(true)}
/>
</motion.div>
</FormControl>
<FormMessage className="text-xs" />
</FormItem>
)}
/>
</div>
{/* Instruct field - hidden when in text mode */}
<div style={{ display: isInstructMode ? 'block' : 'none' }}>
<FormField
control={form.control}
name="instruct"
render={({ field }) => (
<FormItem>
<FormControl>
<motion.div
animate={{
height: isExpanded ? 'auto' : '32px',
}}
disabled={!selectedProfileId}
onClick={() => setIsExpanded(true)}
onFocus={() => setIsExpanded(true)}
/>
</motion.div>
</FormControl>
<FormMessage className="text-xs" />
</FormItem>
)}
/>
transition={{ duration: 0.15, ease: 'easeOut' }}
style={{ overflow: 'hidden' }}
>
<Textarea
{...field}
ref={(node: HTMLTextAreaElement | null) => {
// Store ref for auto-resize (only for active field)
if (isInstructMode) {
textareaRef.current = node;
}
// Forward ref to react-hook-form
if (typeof field.ref === 'function') {
field.ref(node);
}
}}
placeholder="e.g. very happy and excited"
className="resize-none bg-transparent border-none focus-visible:ring-0 focus-visible:ring-offset-0 focus:outline-none focus:ring-0 outline-none ring-0 rounded-2xl text-sm placeholder:text-muted-foreground/60 w-full"
style={{
minHeight: isExpanded ? '100px' : '32px',
maxHeight: '300px',
}}
disabled={!selectedProfileId}
onClick={() => setIsExpanded(true)}
onFocus={() => setIsExpanded(true)}
/>
</motion.div>
</FormControl>
<FormMessage className="text-xs" />
</FormItem>
)}
/>
</div>
</motion.div>
<div className="relative shrink-0">
<Button
type="submit"
disabled={isPending || !selectedProfileId}
className="h-10 w-10 rounded-full bg-accent hover:bg-accent/90 hover:scale-105 text-accent-foreground shadow-lg hover:shadow-accent/50 transition-all duration-200"
size="icon"
>
{isPending ? (
<Loader2 className="h-4 w-4 animate-spin" />
) : (
<Sparkles className="h-4 w-4" />
)}
</Button>
<div className="group relative">
<Button
type="submit"
disabled={isPending || !selectedProfileId}
className="h-10 w-10 rounded-full bg-accent hover:bg-accent/90 hover:scale-105 text-accent-foreground shadow-lg transition-all duration-200"
size="icon"
>
{isPending ? (
<Icon icon="svg-spinners:ring-resize" className="h-4 w-4 animate-spin" />
) : (
<HugeiconsIcon icon={SparklesIcon} size={16} className="h-4 w-4" />
)}
</Button>
<span className="pointer-events-none absolute bottom-full left-1/2 -translate-x-1/2 mb-2 whitespace-nowrap rounded-md bg-popover px-3 py-1.5 text-xs text-popover-foreground border border-border opacity-0 transition-opacity group-hover:opacity-100 z-[9999]">
{isPending
? 'Generating...'
: !selectedProfileId
? 'Select a voice profile first'
: 'Generate speech'}
</span>
</div>
<AnimatePresence>
{isExpanded && (
<motion.div
@@ -273,17 +326,25 @@ export function FloatingGenerateBox({
transition={{ duration: 0.2 }}
className="absolute top-0 right-[calc(100%+0.5rem)]"
>
<Button
type="button"
variant="ghost"
size="icon"
onClick={() => setIsInstructMode(!isInstructMode)}
className={`h-10 w-10 rounded-full bg-card border border-border hover:bg-background/50 transition-all duration-200 ${
isInstructMode ? 'text-accent' : ''
}`}
>
<MessageSquare className="h-4 w-4" />
</Button>
<div className="group relative">
<Button
type="button"
variant="ghost"
size="icon"
onClick={() => setIsInstructMode(!isInstructMode)}
className={cn(
'h-10 w-10 rounded-full transition-all duration-200',
isInstructMode
? 'bg-accent text-accent-foreground border border-accent hover:bg-accent/90'
: 'bg-card border border-border hover:bg-background/50',
)}
>
<HugeiconsIcon icon={TextSquareIcon} size={16} className="h-4 w-4" />
</Button>
<span className="pointer-events-none absolute bottom-full left-1/2 -translate-x-1/2 mb-2 whitespace-nowrap rounded-md bg-popover px-3 py-1.5 text-xs text-popover-foreground border border-border opacity-0 transition-opacity group-hover:opacity-100 z-[9999]">
Fine tune instructions
</span>
</div>
</motion.div>
)}
</AnimatePresence>
@@ -1,4 +1,6 @@
import { Loader2, Mic } from 'lucide-react';
import { Mic01Icon } from '@hugeicons/core-free-icons';
import { HugeiconsIcon } from '@hugeicons/react';
import { Icon } from '@iconify/react';
import { Button } from '@/components/ui/button';
import { Card, CardContent, CardHeader, CardTitle } from '@/components/ui/card';
import {
@@ -46,7 +48,11 @@ export function GenerationForm() {
<FormLabel>Voice Profile</FormLabel>
{selectedProfile ? (
<div className="mt-2 p-3 border rounded-md bg-muted/50 flex items-center gap-2">
<Mic className="h-4 w-4 text-muted-foreground" />
<HugeiconsIcon
icon={Mic01Icon}
size={16}
className="h-4 w-4 text-muted-foreground"
/>
<span className="font-medium">{selectedProfile.name}</span>
<span className="text-sm text-muted-foreground">{selectedProfile.language}</span>
</div>
@@ -170,14 +176,10 @@ export function GenerationForm() {
/>
</div>
<Button
type="submit"
className="w-full"
disabled={isPending || !selectedProfileId}
>
<Button type="submit" className="w-full" disabled={isPending || !selectedProfileId}>
{isPending ? (
<>
<Loader2 className="mr-2 h-4 w-4 animate-spin" />
<Icon icon="svg-spinners:ring-resize" className="mr-2 h-4 w-4 animate-spin" />
Generating...
</>
) : (
+163 -38
View File
@@ -1,4 +1,13 @@
import { AudioWaveform, Download, FileArchive, MoreHorizontal, Play, Trash2 } from 'lucide-react';
import {
Archive01Icon,
Delete01Icon,
Download01Icon,
MoreHorizontalIcon,
PlayIcon,
WaveIcon,
} from '@hugeicons/core-free-icons';
import { HugeiconsIcon } from '@hugeicons/react';
import { Icon } from '@iconify/react';
import { useEffect, useRef, useState } from 'react';
import { Button } from '@/components/ui/button';
import {
@@ -18,6 +27,7 @@ import {
import { Textarea } from '@/components/ui/textarea';
import { useToast } from '@/components/ui/use-toast';
import { apiClient } from '@/lib/api/client';
import type { HistoryResponse } from '@/lib/api/types';
import { BOTTOM_SAFE_AREA_PADDING } from '@/lib/constants/ui';
import {
useDeleteGeneration,
@@ -33,18 +43,29 @@ import { usePlayerStore } from '@/stores/playerStore';
// OLD TABLE-BASED COMPONENT - REMOVED (can be found in git history)
// This is the new alternate history view with fixed height rows
// NEW ALTERNATE HISTORY VIEW - FIXED HEIGHT ROWS
// NEW ALTERNATE HISTORY VIEW - FIXED HEIGHT ROWS WITH INFINITE SCROLL
export function HistoryTable() {
const [page, _setPage] = useState(0);
const [page, setPage] = useState(0);
const [allHistory, setAllHistory] = useState<HistoryResponse[]>([]);
const [total, setTotal] = useState(0);
const [isScrolled, setIsScrolled] = useState(false);
const scrollRef = useRef<HTMLDivElement>(null);
const loadMoreRef = useRef<HTMLDivElement>(null);
const fileInputRef = useRef<HTMLInputElement>(null);
const [importDialogOpen, setImportDialogOpen] = useState(false);
const [selectedFile, setSelectedFile] = useState<File | null>(null);
const [deleteDialogOpen, setDeleteDialogOpen] = useState(false);
const [generationToDelete, setGenerationToDelete] = useState<{ id: string; name: string } | null>(
null,
);
const limit = 20;
const { toast } = useToast();
const { data: historyData, isLoading } = useHistory({
const {
data: historyData,
isLoading,
isFetching,
} = useHistory({
limit,
offset: page * limit,
});
@@ -53,13 +74,63 @@ export function HistoryTable() {
const exportGeneration = useExportGeneration();
const exportGenerationAudio = useExportGenerationAudio();
const importGeneration = useImportGeneration();
const setAudio = usePlayerStore((state) => state.setAudio);
const setAudioWithAutoPlay = usePlayerStore((state) => state.setAudioWithAutoPlay);
const restartCurrentAudio = usePlayerStore((state) => state.restartCurrentAudio);
const currentAudioId = usePlayerStore((state) => state.audioId);
const isPlaying = usePlayerStore((state) => state.isPlaying);
const audioUrl = usePlayerStore((state) => state.audioUrl);
const isPlayerVisible = !!audioUrl;
// Update accumulated history when new data arrives
useEffect(() => {
if (historyData?.items) {
setTotal(historyData.total);
if (page === 0) {
// Reset to first page
setAllHistory(historyData.items);
} else {
// Append new items, avoiding duplicates
setAllHistory((prev) => {
const existingIds = new Set(prev.map((item) => item.id));
const newItems = historyData.items.filter((item) => !existingIds.has(item.id));
return [...prev, ...newItems];
});
}
}
}, [historyData, page]);
// Reset to page 0 when deletions or imports occur
useEffect(() => {
if (deleteGeneration.isSuccess || importGeneration.isSuccess) {
setPage(0);
setAllHistory([]);
}
}, [deleteGeneration.isSuccess, importGeneration.isSuccess]);
// Intersection Observer for infinite scroll
useEffect(() => {
const loadMoreEl = loadMoreRef.current;
if (!loadMoreEl) return;
const observer = new IntersectionObserver(
(entries) => {
const target = entries[0];
if (target.isIntersecting && !isFetching && allHistory.length < total) {
setPage((prev) => prev + 1);
}
},
{
root: scrollRef.current,
rootMargin: '100px',
threshold: 0.1,
},
);
observer.observe(loadMoreEl);
return () => observer.disconnect();
}, [isFetching, allHistory.length, total]);
// Track scroll position for gradient effect
useEffect(() => {
const scrollEl = scrollRef.current;
if (!scrollEl) return;
@@ -77,9 +148,9 @@ export function HistoryTable() {
if (currentAudioId === audioId) {
restartCurrentAudio();
} else {
// Otherwise, load the new audio
// Otherwise, load the new audio and auto-play it
const audioUrl = apiClient.getAudioUrl(audioId);
setAudio(audioUrl, audioId, profileId, text.substring(0, 50));
setAudioWithAutoPlay(audioUrl, audioId, profileId, text.substring(0, 50));
}
};
@@ -113,24 +184,16 @@ export function HistoryTable() {
);
};
const _handleImportClick = () => {
file_handleImportClickk.click();
const handleDeleteClick = (generationId: string, profileName: string) => {
setGenerationToDelete({ id: generationId, name: profileName });
setDeleteDialogOpen(true);
};
const _handleFileChange = (_e: React.ChangeEvent<HTMLInputElement>) => {
cons_handleFileChangeet.files?.[0];
if (file) {
// Validate file extension
if (!file.name.endsWith('.voicebox.zip')) {
toast({
title: 'Invalid file type',
description: 'Please select a valid .voicebox.zip file',
variant: 'destructive',
});
return;
}
setSelectedFile(file);
setImportDialogOpen(true);
const handleDeleteConfirm = () => {
if (generationToDelete) {
deleteGeneration.mutate(generationToDelete.id);
setDeleteDialogOpen(false);
setGenerationToDelete(null);
}
};
@@ -159,13 +222,19 @@ export function HistoryTable() {
}
};
if (isLoading) {
return null;
if (isLoading && page === 0) {
return (
<div className="flex items-center justify-center h-full">
<Icon
icon="svg-spinners:ring-resize"
className="h-8 w-8 animate-spin text-muted-foreground"
/>
</div>
);
}
const history = historyData?.items || [];
const total = historyData?.total || 0;
const _hasMore = history.length === limit && (page + 1) * limit < total;
const history = allHistory;
const hasMore = allHistory.length < total;
return (
<div className="flex flex-col h-full min-h-0 relative">
@@ -205,7 +274,11 @@ export function HistoryTable() {
>
{/* Waveform icon */}
<div className="flex items-center shrink-0">
<AudioWaveform className="h-5 w-5 text-muted-foreground" />
<HugeiconsIcon
icon={WaveIcon}
size={20}
className="h-5 w-5 text-muted-foreground"
/>
</div>
{/* Left side - Meta information */}
@@ -229,11 +302,16 @@ export function HistoryTable() {
<Textarea
value={gen.text}
className="flex-1 resize-none text-sm text-muted-foreground select-text"
readOnly
/>
</div>
{/* Far right - Ellipsis actions */}
<div className="w-10 shrink-0 flex justify-end">
<div
className="w-10 shrink-0 flex justify-end"
onMouseDown={(e) => e.stopPropagation()}
onClick={(e) => e.stopPropagation()}
>
<DropdownMenu>
<DropdownMenuTrigger asChild>
<Button
@@ -241,38 +319,36 @@ export function HistoryTable() {
size="icon"
className="h-8 w-8"
aria-label="Actions"
onClick={(e) => e.stopPropagation()}
>
<MoreHorizontal className="h-4 w-4" />
<HugeiconsIcon icon={MoreHorizontalIcon} size={16} className="h-4 w-4" />
</Button>
</DropdownMenuTrigger>
<DropdownMenuContent align="end">
<DropdownMenuItem
onClick={() => handlePlay(gen.id, gen.text, gen.profile_id)}
>
<Play className="mr-2 h-4 w-4" />
<HugeiconsIcon icon={PlayIcon} size={16} className="mr-2 h-4 w-4" />
Play
</DropdownMenuItem>
<DropdownMenuItem
onClick={() => handleDownloadAudio(gen.id, gen.text)}
disabled={exportGenerationAudio.isPending}
>
<Download className="mr-2 h-4 w-4" />
<HugeiconsIcon icon={Download01Icon} size={16} className="mr-2 h-4 w-4" />
Export Audio
</DropdownMenuItem>
<DropdownMenuItem
onClick={() => handleExportPackage(gen.id, gen.text)}
disabled={exportGeneration.isPending}
>
<FileArchive className="mr-2 h-4 w-4" />
<HugeiconsIcon icon={Archive01Icon} size={16} className="mr-2 h-4 w-4" />
Export Package
</DropdownMenuItem>
<DropdownMenuItem
onClick={() => deleteGeneration.mutate(gen.id)}
onClick={() => handleDeleteClick(gen.id, gen.profile_name)}
disabled={deleteGeneration.isPending}
className="text-destructive focus:text-destructive"
>
<Trash2 className="mr-2 h-4 w-4" />
<HugeiconsIcon icon={Delete01Icon} size={16} className="mr-2 h-4 w-4" />
Delete
</DropdownMenuItem>
</DropdownMenuContent>
@@ -281,10 +357,59 @@ export function HistoryTable() {
</div>
);
})}
{/* Load more trigger element */}
{hasMore && (
<div ref={loadMoreRef} className="flex items-center justify-center py-4">
{isFetching && (
<Icon
icon="svg-spinners:ring-resize"
className="h-6 w-6 animate-spin text-muted-foreground"
/>
)}
</div>
)}
{/* End of list indicator */}
{!hasMore && history.length > 0 && (
<div className="text-center py-4 text-xs text-muted-foreground">
You've reached the end
</div>
)}
</div>
</>
)}
<Dialog open={deleteDialogOpen} onOpenChange={setDeleteDialogOpen}>
<DialogContent>
<DialogHeader>
<DialogTitle>Delete Generation</DialogTitle>
<DialogDescription>
Are you sure you want to delete this generation from "{generationToDelete?.name}"?
This action cannot be undone.
</DialogDescription>
</DialogHeader>
<DialogFooter>
<Button
variant="outline"
onClick={() => {
setDeleteDialogOpen(false);
setGenerationToDelete(null);
}}
>
Cancel
</Button>
<Button
variant="destructive"
onClick={handleDeleteConfirm}
disabled={deleteGeneration.isPending}
>
{deleteGeneration.isPending ? 'Deleting...' : 'Delete'}
</Button>
</DialogFooter>
</DialogContent>
</Dialog>
<Dialog open={importDialogOpen} onOpenChange={setImportDialogOpen}>
<DialogContent>
<DialogHeader>
+4 -3
View File
@@ -1,4 +1,5 @@
import { Sparkles, Upload } from 'lucide-react';
import { HugeiconsIcon } from '@hugeicons/react';
import { SparklesIcon, Upload01Icon } from '@hugeicons/core-free-icons';
import { useRef, useState } from 'react';
import { FloatingGenerateBox } from '@/components/Generation/FloatingGenerateBox';
import { HistoryTable } from '@/components/History/HistoryTable';
@@ -89,7 +90,7 @@ export function MainEditor() {
<h2 className="text-2xl font-bold">Voicebox</h2>
<div className="flex gap-2">
<Button variant="outline" onClick={handleImportClick}>
<Upload className="mr-2 h-4 w-4" />
<HugeiconsIcon icon={Upload01Icon} size={16} className="mr-2 h-4 w-4" />
Import Voice
</Button>
<input
@@ -100,7 +101,7 @@ export function MainEditor() {
className="hidden"
/>
<Button onClick={() => setDialogOpen(true)}>
<Sparkles className="mr-2 h-4 w-4" />
<HugeiconsIcon icon={SparklesIcon} size={16} className="mr-2 h-4 w-4" />
Create Voice
</Button>
</div>
@@ -17,7 +17,7 @@ import { Input } from '@/components/ui/input';
import { Checkbox } from '@/components/ui/checkbox';
import { useToast } from '@/components/ui/use-toast';
import { useServerStore } from '@/stores/serverStore';
import { setKeepServerRunning } from '@/lib/tauri';
import { usePlatform } from '@/platform/PlatformContext';
const connectionSchema = z.object({
serverUrl: z.string().url('Please enter a valid URL'),
@@ -26,6 +26,7 @@ const connectionSchema = z.object({
type ConnectionFormValues = z.infer<typeof connectionSchema>;
export function ConnectionForm() {
const platform = usePlatform();
const serverUrl = useServerStore((state) => state.serverUrl);
const setServerUrl = useServerStore((state) => state.setServerUrl);
const keepServerRunningOnClose = useServerStore((state) => state.keepServerRunningOnClose);
@@ -89,7 +90,7 @@ export function ConnectionForm() {
checked={keepServerRunningOnClose}
onCheckedChange={(checked: boolean) => {
setKeepServerRunningOnClose(checked);
setKeepServerRunning(checked).catch((error) => {
platform.lifecycle.setKeepServerRunning(checked).catch((error) => {
console.error('Failed to sync setting to Rust:', error);
});
toast({
@@ -0,0 +1,112 @@
import { Folder01Icon, FolderOpenIcon } from '@hugeicons/core-free-icons';
import { HugeiconsIcon } from '@hugeicons/react';
import { Icon } from '@iconify/react';
import { Button } from '@/components/ui/button';
import { Card, CardContent, CardDescription, CardHeader, CardTitle } from '@/components/ui/card';
import { Input } from '@/components/ui/input';
import { useSystemFolders } from '@/lib/hooks/useSystemFolders';
import { usePlatform } from '@/platform/PlatformContext';
interface FolderRowProps {
label: string;
description: string;
path: string | undefined;
isLoading: boolean;
canOpen: boolean;
onOpen: () => void;
}
function FolderRow({ label, description, path, isLoading, canOpen, onOpen }: FolderRowProps) {
return (
<div className="space-y-1.5">
<div className="flex items-center justify-between">
<div>
<div className="text-sm font-medium">{label}</div>
<div className="text-xs text-muted-foreground">{description}</div>
</div>
{canOpen && path && (
<Button
variant="outline"
size="sm"
onClick={onOpen}
disabled={isLoading || !path}
className="shrink-0"
>
<HugeiconsIcon icon={FolderOpenIcon} size={16} className="h-4 w-4 mr-2" />
Open
</Button>
)}
</div>
<Input
value={isLoading ? 'Loading...' : path || 'Not available'}
readOnly
className="font-mono text-xs text-muted-foreground select-all cursor-text"
/>
</div>
);
}
export function DataFolders() {
const { data: folders, isLoading, error } = useSystemFolders();
const platform = usePlatform();
const isTauri = platform.metadata.isTauri;
const handleOpenFolder = async (path: string | undefined) => {
if (!path) return;
const success = await platform.filesystem.openFolder(path);
if (!success && isTauri) {
console.error('Failed to open folder:', path);
}
};
return (
<Card>
<CardHeader>
<CardTitle className="flex items-center gap-2">
<HugeiconsIcon icon={Folder01Icon} size={20} className="h-5 w-5" />
Data Folders
</CardTitle>
<CardDescription>
{isTauri
? 'Click "Open" to view folders in your file explorer, or copy the paths below.'
: 'These are the server-side folder paths where your data is stored.'}
</CardDescription>
</CardHeader>
<CardContent className="space-y-4">
{error ? (
<div className="flex items-center gap-2 text-sm text-destructive">
<Icon icon="lucide:alert-circle" className="h-4 w-4" />
<span>Failed to load folder paths: {error.message}</span>
</div>
) : (
<>
<FolderRow
label="App Data"
description="Voices, generations, and app database"
path={folders?.data_dir}
isLoading={isLoading}
canOpen={isTauri}
onOpen={() => handleOpenFolder(folders?.data_dir)}
/>
<FolderRow
label="Models"
description="Downloaded AI models from HuggingFace Hub"
path={folders?.models_dir}
isLoading={isLoading}
canOpen={isTauri}
onOpen={() => handleOpenFolder(folders?.models_dir)}
/>
<FolderRow
label="Providers"
description="External TTS provider binaries (PyTorch CPU/CUDA)"
path={folders?.providers_dir}
isLoading={isLoading}
canOpen={isTauri}
onOpen={() => handleOpenFolder(folders?.providers_dir)}
/>
</>
)}
</CardContent>
</Card>
);
}
@@ -1,6 +1,8 @@
import { Delete01Icon, Download01Icon } from '@hugeicons/core-free-icons';
import { HugeiconsIcon } from '@hugeicons/react';
import { Icon } from '@iconify/react';
import { useMutation, useQuery, useQueryClient } from '@tanstack/react-query';
import { Download, Loader2, Trash2 } from 'lucide-react';
import { useState } from 'react';
import { useCallback, useState } from 'react';
import {
AlertDialog,
AlertDialogAction,
@@ -17,7 +19,6 @@ import { Card, CardContent, CardDescription, CardHeader, CardTitle } from '@/com
import { useToast } from '@/components/ui/use-toast';
import { apiClient } from '@/lib/api/client';
import { useModelDownloadToast } from '@/lib/hooks/useModelDownloadToast';
import { ModelProgress } from './ModelProgress';
export function ModelManagement() {
const { toast } = useToast();
@@ -27,15 +28,36 @@ export function ModelManagement() {
const { data: modelStatus, isLoading } = useQuery({
queryKey: ['modelStatus'],
queryFn: () => apiClient.getModelStatus(),
queryFn: async () => {
console.log('[Query] Fetching model status');
const result = await apiClient.getModelStatus();
console.log('[Query] Model status fetched:', result);
return result;
},
refetchInterval: 5000, // Refresh every 5 seconds
});
// Callbacks for download completion
const handleDownloadComplete = useCallback(() => {
console.log('[ModelManagement] Download complete, clearing state');
setDownloadingModel(null);
setDownloadingDisplayName(null);
queryClient.invalidateQueries({ queryKey: ['modelStatus'] });
}, [queryClient]);
const handleDownloadError = useCallback(() => {
console.log('[ModelManagement] Download error, clearing state');
setDownloadingModel(null);
setDownloadingDisplayName(null);
}, []);
// Use progress toast hook for the downloading model
useModelDownloadToast({
modelName: downloadingModel || '',
displayName: downloadingDisplayName || '',
enabled: !!downloadingModel && !!downloadingDisplayName,
onComplete: handleDownloadComplete,
onError: handleDownloadError,
});
const [deleteDialogOpen, setDeleteDialogOpen] = useState(false);
@@ -45,44 +67,69 @@ export function ModelManagement() {
sizeMb?: number;
} | null>(null);
const downloadMutation = useMutation({
mutationFn: (modelName: string) => {
const handleDownload = async (modelName: string) => {
console.log('[Download] Button clicked for:', modelName, 'at', new Date().toISOString());
// Find display name
const model = modelStatus?.models.find((m) => m.model_name === modelName);
const displayName = model?.display_name || modelName;
try {
// IMPORTANT: Call the API FIRST before setting state
// Setting state enables the SSE EventSource in useModelDownloadToast,
// which can block/delay the download fetch due to HTTP/1.1 connection limits
console.log('[Download] Calling download API for:', modelName);
const result = await apiClient.triggerModelDownload(modelName);
console.log('[Download] Download API responded:', result);
// NOW set state to enable SSE tracking (after download has started on backend)
setDownloadingModel(modelName);
// Find display name from model status
const model = modelStatus?.models.find((m) => m.model_name === modelName);
setDownloadingDisplayName(model?.display_name || modelName);
return apiClient.triggerModelDownload(modelName);
},
onSuccess: () => {
// Download completed - clear state and refetch status
setDownloadingModel(null);
setDownloadingDisplayName(null);
setDownloadingDisplayName(displayName);
// Download initiated successfully - state will be cleared when SSE reports completion
// or by the polling interval detecting the model is downloaded
queryClient.invalidateQueries({ queryKey: ['modelStatus'] });
},
onError: (error: Error) => {
} catch (error) {
console.error('[Download] Download failed:', error);
setDownloadingModel(null);
setDownloadingDisplayName(null);
toast({
title: 'Download failed',
description: error.message,
description: error instanceof Error ? error.message : 'Unknown error',
variant: 'destructive',
});
},
});
}
};
const deleteMutation = useMutation({
mutationFn: (modelName: string) => apiClient.deleteModel(modelName),
onSuccess: () => {
mutationFn: async (modelName: string) => {
console.log('[Delete] Deleting model:', modelName);
const result = await apiClient.deleteModel(modelName);
console.log('[Delete] Model deleted successfully:', modelName);
return result;
},
onSuccess: async (_data, _modelName) => {
console.log('[Delete] onSuccess - showing toast and invalidating queries');
toast({
title: 'Model deleted',
description: `${modelToDelete?.displayName || 'Model'} has been deleted successfully.`,
});
setDeleteDialogOpen(false);
setModelToDelete(null);
// Refetch status to update UI
queryClient.invalidateQueries({ queryKey: ['modelStatus'] });
// Invalidate AND explicitly refetch to ensure UI updates
// Using refetchType: 'all' ensures we refetch even if the query is stale
console.log('[Delete] Invalidating modelStatus query');
await queryClient.invalidateQueries({
queryKey: ['modelStatus'],
refetchType: 'all',
});
// Also explicitly refetch to guarantee fresh data
console.log('[Delete] Explicitly refetching modelStatus query');
await queryClient.refetchQueries({ queryKey: ['modelStatus'] });
console.log('[Delete] Query refetched');
},
onError: (error: Error) => {
console.log('[Delete] onError:', error);
toast({
title: 'Delete failed',
description: error.message,
@@ -108,7 +155,10 @@ export function ModelManagement() {
<CardContent className="space-y-4">
{isLoading ? (
<div className="flex items-center justify-center py-8">
<Loader2 className="h-6 w-6 animate-spin text-muted-foreground" />
<Icon
icon="svg-spinners:ring-resize"
className="h-6 w-6 animate-spin text-muted-foreground"
/>
</div>
) : modelStatus ? (
<div className="space-y-4">
@@ -124,7 +174,7 @@ export function ModelManagement() {
<ModelItem
key={model.model_name}
model={model}
onDownload={() => downloadMutation.mutate(model.model_name)}
onDownload={() => handleDownload(model.model_name)}
onDelete={() => {
setModelToDelete({
name: model.model_name,
@@ -152,7 +202,7 @@ export function ModelManagement() {
<ModelItem
key={model.model_name}
model={model}
onDownload={() => downloadMutation.mutate(model.model_name)}
onDownload={() => handleDownload(model.model_name)}
onDelete={() => {
setModelToDelete({
name: model.model_name,
@@ -167,22 +217,6 @@ export function ModelManagement() {
))}
</div>
</div>
{/* Progress indicators */}
<div className="pt-4 border-t">
<h3 className="text-sm font-semibold mb-3 text-muted-foreground">
Download Progress
</h3>
<div className="space-y-2">
{modelStatus.models.map((model) => (
<ModelProgress
key={model.model_name}
modelName={model.model_name}
displayName={model.display_name}
/>
))}
</div>
</div>
</div>
) : null}
</CardContent>
@@ -216,7 +250,7 @@ export function ModelManagement() {
>
{deleteMutation.isPending ? (
<>
<Loader2 className="h-4 w-4 mr-2 animate-spin" />
<Icon icon="svg-spinners:ring-resize" className="h-4 w-4 mr-2 animate-spin" />
Deleting...
</>
) : (
@@ -235,16 +269,20 @@ interface ModelItemProps {
model_name: string;
display_name: string;
downloaded: boolean;
downloading?: boolean; // From server - true if download in progress
size_mb?: number;
loaded: boolean;
};
onDownload: () => void;
onDelete: () => void;
isDownloading: boolean;
isDownloading: boolean; // Local state - true if user just clicked download
formatSize: (sizeMb?: number) => string;
}
function ModelItem({ model, onDownload, onDelete, isDownloading, formatSize }: ModelItemProps) {
// Use server's downloading state OR local state (for immediate feedback before server updates)
const showDownloading = model.downloading || isDownloading;
return (
<div className="flex items-center justify-between p-3 border rounded-lg">
<div className="flex-1">
@@ -255,20 +293,21 @@ function ModelItem({ model, onDownload, onDelete, isDownloading, formatSize }: M
Loaded
</Badge>
)}
{model.downloaded && !model.loaded && (
{/* Only show Downloaded if actually downloaded AND not downloading */}
{model.downloaded && !model.loaded && !showDownloading && (
<Badge variant="secondary" className="text-xs">
Downloaded
</Badge>
)}
</div>
{model.downloaded && model.size_mb && (
{model.downloaded && model.size_mb && !showDownloading && (
<div className="text-xs text-muted-foreground mt-1">
Size: {formatSize(model.size_mb)}
</div>
)}
</div>
<div className="flex items-center gap-2">
{model.downloaded ? (
{model.downloaded && !showDownloading ? (
<div className="flex items-center gap-2">
<div className="flex items-center gap-1 text-sm text-muted-foreground">
<span>Ready</span>
@@ -280,22 +319,18 @@ function ModelItem({ model, onDownload, onDelete, isDownloading, formatSize }: M
disabled={model.loaded}
title={model.loaded ? 'Unload model before deleting' : 'Delete model'}
>
<Trash2 className="h-4 w-4" />
<HugeiconsIcon icon={Delete01Icon} size={16} className="h-4 w-4" />
</Button>
</div>
) : showDownloading ? (
<Button size="sm" variant="outline" disabled>
<Icon icon="svg-spinners:ring-resize" className="h-4 w-4 mr-2 animate-spin" />
Downloading...
</Button>
) : (
<Button size="sm" onClick={onDownload} disabled={isDownloading} variant="outline">
{isDownloading ? (
<>
<Loader2 className="h-4 w-4 mr-2 animate-spin" />
Downloading...
</>
) : (
<>
<Download className="h-4 w-4 mr-2" />
Download
</>
)}
<Button size="sm" onClick={onDownload} variant="outline">
<HugeiconsIcon icon={Download01Icon} size={16} className="h-4 w-4 mr-2" />
Download
</Button>
)}
</div>
@@ -1,4 +1,6 @@
import { Loader2, XCircle } from 'lucide-react';
import { CancelCircleIcon } from '@hugeicons/core-free-icons';
import { HugeiconsIcon } from '@hugeicons/react';
import { Icon } from '@iconify/react';
import { useEffect, useState } from 'react';
import { Card, CardContent, CardHeader, CardTitle } from '@/components/ui/card';
import { Progress } from '@/components/ui/progress';
@@ -8,15 +10,27 @@ import { useServerStore } from '@/stores/serverStore';
interface ModelProgressProps {
modelName: string;
displayName: string;
/** Only connect to SSE when actively downloading - prevents connection exhaustion */
isDownloading?: boolean;
}
export function ModelProgress({ modelName, displayName }: ModelProgressProps) {
export function ModelProgress({
modelName,
displayName,
isDownloading = false,
}: ModelProgressProps) {
const [progress, setProgress] = useState<ModelProgressType | null>(null);
const [isSubscribed, setIsSubscribed] = useState(false);
const serverUrl = useServerStore((state) => state.serverUrl);
useEffect(() => {
if (!serverUrl || isSubscribed) return;
// IMPORTANT: Only connect to SSE when this specific model is downloading
// Opening SSE connections for all models exhausts HTTP/1.1 connection limits (6 per origin)
// which causes other fetches (like the download trigger) to be queued/blocked
if (!serverUrl || !isDownloading) {
return;
}
console.log(`[ModelProgress] Connecting SSE for ${modelName}`);
// Subscribe to progress updates via Server-Sent Events
const eventSource = new EventSource(`${serverUrl}/models/progress/${modelName}`);
@@ -28,8 +42,8 @@ export function ModelProgress({ modelName, displayName }: ModelProgressProps) {
// Close connection if complete or error
if (data.status === 'complete' || data.status === 'error') {
console.log(`[ModelProgress] Download ${data.status} for ${modelName}, closing SSE`);
eventSource.close();
setIsSubscribed(false);
}
} catch (error) {
console.error('Error parsing progress event:', error);
@@ -37,18 +51,15 @@ export function ModelProgress({ modelName, displayName }: ModelProgressProps) {
};
eventSource.onerror = (error) => {
console.error('SSE error:', error);
console.error(`[ModelProgress] SSE error for ${modelName}:`, error);
eventSource.close();
setIsSubscribed(false);
};
setIsSubscribed(true);
return () => {
console.log(`[ModelProgress] Cleanup - closing SSE for ${modelName}`);
eventSource.close();
setIsSubscribed(false);
};
}, [serverUrl, modelName, isSubscribed]);
}, [serverUrl, modelName, isDownloading]);
// Don't render if no progress or if complete/error and some time has passed
if (
@@ -69,10 +80,12 @@ export function ModelProgress({ modelName, displayName }: ModelProgressProps) {
const getStatusIcon = () => {
switch (progress.status) {
case 'error':
return <XCircle className="h-4 w-4 text-destructive" />;
return (
<HugeiconsIcon icon={CancelCircleIcon} size={16} className="h-4 w-4 text-destructive" />
);
case 'downloading':
case 'extracting':
return <Loader2 className="h-4 w-4 animate-spin" />;
return <Icon icon="svg-spinners:ring-resize" className="h-4 w-4 animate-spin" />;
default:
return null;
}
@@ -0,0 +1,400 @@
import { Delete01Icon, Download01Icon } from '@hugeicons/core-free-icons';
import { HugeiconsIcon } from '@hugeicons/react';
import { Icon } from '@iconify/react';
import { useMutation, useQuery, useQueryClient } from '@tanstack/react-query';
import { useCallback, useState } from 'react';
import {
AlertDialog,
AlertDialogAction,
AlertDialogCancel,
AlertDialogContent,
AlertDialogDescription,
AlertDialogFooter,
AlertDialogHeader,
AlertDialogTitle,
} from '@/components/ui/alert-dialog';
import { Button } from '@/components/ui/button';
import { Card, CardContent, CardDescription, CardHeader, CardTitle } from '@/components/ui/card';
import { Label } from '@/components/ui/label';
import { RadioGroup, RadioGroupItem } from '@/components/ui/radio-group';
import { useToast } from '@/components/ui/use-toast';
import { apiClient } from '@/lib/api/client';
import { useModelDownloadToast } from '@/lib/hooks/useModelDownloadToast';
const isMacOS = () => navigator.platform.toLowerCase().includes('mac');
const isWindows = () => navigator.platform.toLowerCase().includes('win');
const getPlatformName = () => {
if (isMacOS()) return 'macOS';
if (isWindows()) return 'Windows';
return 'Linux';
};
type ProviderType =
| 'auto'
| 'apple-mlx'
| 'bundled-pytorch'
| 'pytorch-cpu'
| 'pytorch-cuda'
| 'remote'
| 'openai';
export function ProviderSettings() {
const { toast } = useToast();
const queryClient = useQueryClient();
const [downloadingProvider, setDownloadingProvider] = useState<string | null>(null);
const { data: providersData, isLoading } = useQuery({
queryKey: ['providers'],
queryFn: async () => {
return await apiClient.listProviders();
},
refetchInterval: 5000,
});
const { data: activeProvider } = useQuery({
queryKey: ['activeProvider'],
queryFn: async () => {
return await apiClient.getActiveProvider();
},
refetchInterval: 5000,
});
// Callbacks for download completion
const handleDownloadComplete = useCallback(() => {
setDownloadingProvider(null);
queryClient.invalidateQueries({ queryKey: ['providers'] });
}, [queryClient]);
const handleDownloadError = useCallback(() => {
setDownloadingProvider(null);
}, []);
// Use progress toast hook for the downloading provider
useModelDownloadToast({
modelName: downloadingProvider || '',
displayName: downloadingProvider || '',
enabled: !!downloadingProvider,
onComplete: handleDownloadComplete,
onError: handleDownloadError,
});
const [deleteDialogOpen, setDeleteDialogOpen] = useState(false);
const [providerToDelete, setProviderToDelete] = useState<string | null>(null);
const downloadMutation = useMutation({
mutationFn: async (providerType: string) => {
return await apiClient.downloadProvider(providerType);
},
onSuccess: (_, providerType) => {
setDownloadingProvider(providerType);
queryClient.invalidateQueries({ queryKey: ['providers'] });
},
onError: (error: Error) => {
toast({
title: 'Download failed',
description: error.message,
variant: 'destructive',
});
},
});
const startMutation = useMutation({
mutationFn: async (providerType: string) => {
return await apiClient.startProvider(providerType);
},
onSuccess: () => {
queryClient.invalidateQueries({ queryKey: ['activeProvider'] });
toast({
title: 'Provider started',
description: 'The provider has been started successfully',
});
},
onError: (error: Error) => {
toast({
title: 'Failed to start provider',
description: error.message,
variant: 'destructive',
});
},
});
const deleteMutation = useMutation({
mutationFn: async (providerType: string) => {
return await apiClient.deleteProvider(providerType);
},
onSuccess: () => {
queryClient.invalidateQueries({ queryKey: ['providers'] });
toast({
title: 'Provider deleted',
description: 'The provider has been deleted successfully',
});
},
onError: (error: Error) => {
toast({
title: 'Failed to delete provider',
description: error.message,
variant: 'destructive',
});
},
});
const handleDownload = async (providerType: string) => {
downloadMutation.mutate(providerType);
};
const handleStart = async (providerType: string) => {
startMutation.mutate(providerType);
};
const handleDelete = (providerType: string) => {
setProviderToDelete(providerType);
setDeleteDialogOpen(true);
};
const confirmDelete = () => {
if (providerToDelete) {
deleteMutation.mutate(providerToDelete);
setDeleteDialogOpen(false);
setProviderToDelete(null);
}
};
if (isLoading) {
return (
<Card>
<CardHeader>
<CardTitle>TTS Provider</CardTitle>
<CardDescription>Choose how Voicebox generates speech</CardDescription>
</CardHeader>
<CardContent>
<div className="flex items-center justify-center py-8">
<Icon icon="svg-spinners:ring-resize" className="h-6 w-6 animate-spin" />
</div>
</CardContent>
</Card>
);
}
const installedProviders = providersData?.installed || [];
// Determine current active provider
const currentProvider = activeProvider?.provider;
console.log('currentProvider', currentProvider);
const selectedProvider = currentProvider as ProviderType;
const isStarting = startMutation.isPending;
return (
<>
<Card>
<CardHeader>
<CardTitle>TTS Provider</CardTitle>
<CardDescription>Choose how Voicebox generates speech.</CardDescription>
</CardHeader>
<CardContent className="relative">
{isStarting && (
<div className="absolute inset-0 bg-background/80 backdrop-blur-sm flex items-center justify-center z-10 rounded-lg">
<div className="flex items-center gap-2 text-muted-foreground">
<Icon icon="svg-spinners:ring-resize" className="h-5 w-5" />
<span>Starting provider...</span>
</div>
</div>
)}
<RadioGroup
value={selectedProvider}
onValueChange={(value) => handleStart(value)}
disabled={isStarting}
>
{/* PyTorch CUDA */}
<div className="flex items-center justify-between py-2">
<div className={`flex items-center space-x-3 flex-1 ${isMacOS() || !installedProviders.includes('pytorch-cuda') ? 'opacity-50' : ''}`}>
<RadioGroupItem value="pytorch-cuda" id="cuda" disabled={isMacOS() || isStarting || !installedProviders.includes('pytorch-cuda')} />
<Label
htmlFor="cuda"
className={`flex-1 ${isMacOS() || isStarting || !installedProviders.includes('pytorch-cuda') ? 'cursor-not-allowed' : 'cursor-pointer'}`}
>
<div className="font-medium">PyTorch CUDA</div>
<div className="text-sm text-muted-foreground">
NVIDIA GPU-accelerated provider
</div>
</Label>
</div>
<div className="flex items-center gap-2">
{isMacOS() && (
<>
<span className="text-xs text-muted-foreground">2.4GB</span>
<Button size="sm" variant="secondary" disabled>
Not Available on macOS
</Button>
</>
)}
{!isMacOS() && !installedProviders.includes('pytorch-cuda') && (
<>
<span className="text-xs text-muted-foreground">2.4GB</span>
<Button
onClick={() => handleDownload('pytorch-cuda')}
size="sm"
variant="outline"
disabled={downloadingProvider === 'pytorch-cuda' || isStarting}
className="shrink-0"
>
{downloadingProvider === 'pytorch-cuda' ? (
<Icon icon="svg-spinners:ring-resize" className="h-4 w-4 animate-spin" />
) : (
<>
<HugeiconsIcon icon={Download01Icon} size={16} className="h-4 w-4 mr-2" />
Download
</>
)}
</Button>
</>
)}
{installedProviders.includes('pytorch-cuda') && (
<Button
onClick={() => handleDelete('pytorch-cuda')}
size="sm"
variant="outline"
disabled={isStarting}
className="shrink-0"
>
<HugeiconsIcon icon={Delete01Icon} size={16} className="h-4 w-4 mr-2" />
Uninstall
</Button>
)}
</div>
</div>
{/* PyTorch CPU */}
<div className="flex items-center justify-between py-2">
<div className={`flex items-center space-x-3 flex-1 ${!installedProviders.includes('pytorch-cpu') ? 'opacity-50' : ''}`}>
<RadioGroupItem value="pytorch-cpu" id="cpu" disabled={isStarting || !installedProviders.includes('pytorch-cpu')} />
<Label
htmlFor="cpu"
className={`flex-1 ${isStarting || !installedProviders.includes('pytorch-cpu') ? 'cursor-not-allowed' : 'cursor-pointer'}`}
>
<div className="font-medium">PyTorch CPU</div>
<div className="text-sm text-muted-foreground">
Works on any system, slower inference
</div>
</Label>
</div>
<div className="flex items-center gap-2">
{!installedProviders.includes('pytorch-cpu') && (
<>
<span className="text-xs text-muted-foreground">242MB</span>
<Button
onClick={() => handleDownload('pytorch-cpu')}
size="sm"
variant="outline"
disabled={downloadingProvider === 'pytorch-cpu' || isStarting}
className="shrink-0"
>
{downloadingProvider === 'pytorch-cpu' ? (
<Icon icon="svg-spinners:ring-resize" className="h-4 w-4 animate-spin" />
) : (
<>
<HugeiconsIcon icon={Download01Icon} size={16} className="h-4 w-4 mr-2" />
Download
</>
)}
</Button>
</>
)}
{installedProviders.includes('pytorch-cpu') && (
<Button
onClick={() => handleDelete('pytorch-cpu')}
size="sm"
variant="outline"
disabled={isStarting}
className="shrink-0"
>
<HugeiconsIcon icon={Delete01Icon} size={16} className="h-4 w-4 mr-2" />
Uninstall
</Button>
)}
</div>
</div>
{/* MLX bundled (macOS Apple Silicon only) */}
<div className="flex items-center justify-between py-2">
<div className={`flex items-center space-x-3 flex-1 ${!isMacOS() ? 'opacity-50' : ''}`}>
<RadioGroupItem value="apple-mlx" id="mlx" disabled={isStarting || !isMacOS()} />
<Label
htmlFor="mlx"
className={`flex-1 ${isStarting || !isMacOS() ? 'cursor-not-allowed' : 'cursor-pointer'}`}
>
<div className="font-medium">Apple MLX</div>
<div className="text-sm text-muted-foreground">
{isMacOS()
? 'Bundled with this version, optimized for Apple Silicon'
: 'Only available on Apple Silicon'}
</div>
</Label>
</div>
{!isMacOS() && (
<Button size="sm" variant="secondary" disabled>
Not Available on {getPlatformName()}
</Button>
)}
</div>
{/* Remote */}
<div className="flex items-center justify-between py-2">
<div className="flex items-center space-x-3 flex-1 opacity-50">
<RadioGroupItem value="remote" id="remote" disabled />
<Label htmlFor="remote" className="flex-1 cursor-not-allowed">
<div className="font-medium">Remote Server</div>
<div className="text-sm text-muted-foreground">
Connect to your own TTS server
</div>
</Label>
</div>
<Button size="sm" variant="secondary" disabled>
Coming Soon
</Button>
</div>
{/* OpenAI */}
<div className="flex items-center justify-between py-2">
<div className="flex items-center space-x-3 flex-1 opacity-50">
<RadioGroupItem value="openai" id="openai" disabled />
<Label htmlFor="openai" className="flex-1 cursor-not-allowed">
<div className="font-medium">OpenAI API</div>
<div className="text-sm text-muted-foreground">Use OpenAI's TTS API</div>
</Label>
</div>
<Button size="sm" variant="secondary" disabled>
Coming Soon
</Button>
</div>
</RadioGroup>
<p className="text-xs text-muted-foreground mt-5">
Note: PyTorch and MLX use different versions of the same model. When switching between
them, you will need to redownload the model.
</p>
</CardContent>
</Card>
<AlertDialog open={deleteDialogOpen} onOpenChange={setDeleteDialogOpen}>
<AlertDialogContent>
<AlertDialogHeader>
<AlertDialogTitle>Delete Provider</AlertDialogTitle>
<AlertDialogDescription>
Are you sure you want to delete {providerToDelete}? This will remove the provider
binary from your system. You can download it again later if needed.
</AlertDialogDescription>
</AlertDialogHeader>
<AlertDialogFooter>
<AlertDialogCancel>Cancel</AlertDialogCancel>
<AlertDialogAction
onClick={confirmDelete}
className="bg-destructive text-destructive-foreground"
>
Delete
</AlertDialogAction>
</AlertDialogFooter>
</AlertDialogContent>
</AlertDialog>
</>
);
}
@@ -1,4 +1,6 @@
import { Loader2, XCircle } from 'lucide-react';
import { CancelCircleIcon } from '@hugeicons/core-free-icons';
import { HugeiconsIcon } from '@hugeicons/react';
import { Icon } from '@iconify/react';
import { Badge } from '@/components/ui/badge';
import { Card, CardContent, CardHeader, CardTitle } from '@/components/ui/card';
import { useServerHealth } from '@/lib/hooks/useServer';
@@ -32,12 +34,12 @@ export function ServerStatus() {
{isLoading ? (
<div className="flex items-center gap-2">
<Loader2 className="h-4 w-4 animate-spin" />
<Icon icon="svg-spinners:ring-resize" className="h-4 w-4 animate-spin" />
<span className="text-sm">Checking connection...</span>
</div>
) : error ? (
<div className="flex items-center gap-2">
<XCircle className="h-4 w-4 text-destructive" />
<HugeiconsIcon icon={CancelCircleIcon} size={16} className="h-4 w-4 text-destructive" />
<span className="text-sm text-destructive">Connection failed: {error.message}</span>
</div>
) : health ? (
@@ -1,21 +1,24 @@
import { getVersion } from '@tauri-apps/api/app';
import { AlertCircle, Download, RefreshCw } from 'lucide-react';
import { HugeiconsIcon } from '@hugeicons/react';
import { AlertCircleIcon, Download01Icon, Refresh01Icon } from '@hugeicons/core-free-icons';
import { useEffect, useState } from 'react';
import { Badge } from '@/components/ui/badge';
import { Button } from '@/components/ui/button';
import { Card, CardContent, CardHeader, CardTitle } from '@/components/ui/card';
import { Progress } from '@/components/ui/progress';
import { useAutoUpdater } from '@/hooks/useAutoUpdater';
import { usePlatform } from '@/platform/PlatformContext';
export function UpdateStatus() {
const platform = usePlatform();
const { status, checkForUpdates, downloadAndInstall, restartAndInstall } = useAutoUpdater(false);
const [currentVersion, setCurrentVersion] = useState<string>('');
useEffect(() => {
getVersion()
platform.metadata
.getVersion()
.then(setCurrentVersion)
.catch(() => setCurrentVersion('0.1.0'));
}, []);
.catch(() => setCurrentVersion('Unknown'));
}, [platform]);
return (
<Card>
@@ -34,21 +37,21 @@ export function UpdateStatus() {
variant="outline"
size="sm"
>
<RefreshCw className={`h-4 w-4 mr-2 ${status.checking ? 'animate-spin' : ''}`} />
<HugeiconsIcon icon={Refresh01Icon} size={16} className={`h-4 w-4 mr-2 ${status.checking ? 'animate-spin' : ''}`} />
Check for Updates
</Button>
</div>
{status.checking && (
<div className="flex items-center gap-2 text-sm text-muted-foreground">
<RefreshCw className="h-4 w-4 animate-spin" />
<HugeiconsIcon icon={Refresh01Icon} size={16} className="h-4 w-4 animate-spin" />
Checking for updates...
</div>
)}
{status.error && (
<div className="flex items-center gap-2 text-sm text-destructive">
<AlertCircle className="h-4 w-4" />
<HugeiconsIcon icon={AlertCircleIcon} size={16} className="h-4 w-4" />
{status.error}
</div>
)}
@@ -63,7 +66,7 @@ export function UpdateStatus() {
<Badge>New</Badge>
</div>
<Button onClick={downloadAndInstall} className="w-full" size="sm">
<Download className="h-4 w-4 mr-2" />
<HugeiconsIcon icon={Download01Icon} size={16} className="h-4 w-4 mr-2" />
Download Update
</Button>
</div>
@@ -73,7 +76,7 @@ export function UpdateStatus() {
<div className="space-y-2">
<div className="flex items-center justify-between text-sm">
<div className="flex items-center gap-2">
<Download className="h-4 w-4" />
<HugeiconsIcon icon={Download01Icon} size={16} className="h-4 w-4" />
Downloading update...
</div>
{status.downloadProgress !== undefined && (
@@ -107,7 +110,7 @@ export function UpdateStatus() {
your convenience.
</div>
<Button onClick={restartAndInstall} className="w-full" size="sm">
<RefreshCw className="h-4 w-4 mr-2" />
<HugeiconsIcon icon={Refresh01Icon} size={16} className="h-4 w-4 mr-2" />
Restart Now
</Button>
</div>
+7 -2
View File
@@ -1,16 +1,21 @@
import { ConnectionForm } from '@/components/ServerSettings/ConnectionForm';
import { DataFolders } from '@/components/ServerSettings/DataFolders';
import { ProviderSettings } from '@/components/ServerSettings/ProviderSettings';
import { ServerStatus } from '@/components/ServerSettings/ServerStatus';
import { UpdateStatus } from '@/components/ServerSettings/UpdateStatus';
import { isTauri } from '@/lib/tauri';
import { usePlatform } from '@/platform/PlatformContext';
export function ServerTab() {
const platform = usePlatform();
return (
<div className="space-y-4 overflow-y-auto flex flex-col">
<div className="grid gap-4 md:grid-cols-2">
<ConnectionForm />
<ServerStatus />
</div>
{isTauri() && <UpdateStatus />}
<ProviderSettings />
<DataFolders />
{platform.metadata.isTauri && <UpdateStatus />}
<div className="py-8 text-center text-sm text-muted-foreground">
Created by{' '}
<a
+19 -12
View File
@@ -1,5 +1,14 @@
import {
Book01Icon,
Mic01Icon,
PackageIcon,
ServerStack01Icon,
SpeakerIcon,
VolumeHighIcon,
} from '@hugeicons/core-free-icons';
import { HugeiconsIcon } from '@hugeicons/react';
import { Icon } from '@iconify/react';
import { Link, useMatchRoute } from '@tanstack/react-router';
import { Box, BookOpen, Loader2, Mic, Server, Speaker, Volume2 } from 'lucide-react';
import voiceboxLogo from '@/assets/voicebox-logo.png';
import { cn } from '@/lib/utils/cn';
import { useGenerationStore } from '@/stores/generationStore';
@@ -10,12 +19,12 @@ interface SidebarProps {
}
const tabs = [
{ id: 'main', path: '/', icon: Volume2, label: 'Generate' },
{ id: 'stories', path: '/stories', icon: BookOpen, label: 'Stories' },
{ id: 'voices', path: '/voices', icon: Mic, label: 'Voices' },
{ id: 'audio', path: '/audio', icon: Speaker, label: 'Audio' },
{ id: 'models', path: '/models', icon: Box, label: 'Models' },
{ id: 'server', path: '/server', icon: Server, label: 'Server' },
{ id: 'main', path: '/', icon: VolumeHighIcon, label: 'Generate' },
{ id: 'stories', path: '/stories', icon: Book01Icon, label: 'Stories' },
{ id: 'voices', path: '/voices', icon: Mic01Icon, label: 'Voices' },
{ id: 'audio', path: '/audio', icon: SpeakerIcon, label: 'Audio' },
{ id: 'models', path: '/models', icon: PackageIcon, label: 'Models' },
{ id: 'server', path: '/server', icon: ServerStack01Icon, label: 'Server' },
];
export function Sidebar({ isMacOS }: SidebarProps) {
@@ -42,9 +51,7 @@ export function Sidebar({ isMacOS }: SidebarProps) {
const Icon = tab.icon;
// For index route, use exact match; for others, use default matching
const isActive =
tab.path === '/'
? matchRoute({ to: '/', exact: true })
: matchRoute({ to: tab.path });
tab.path === '/' ? matchRoute({ to: '/' }) : matchRoute({ to: tab.path });
return (
<Link
@@ -58,7 +65,7 @@ export function Sidebar({ isMacOS }: SidebarProps) {
title={tab.label}
aria-label={tab.label}
>
<Icon className="h-5 w-5" />
<HugeiconsIcon icon={Icon} size={20} className="h-5 w-5" />
</Link>
);
})}
@@ -75,7 +82,7 @@ export function Sidebar({ isMacOS }: SidebarProps) {
isPlayerVisible ? 'mb-[120px]' : 'mb-0',
)}
>
<Loader2 className="h-6 w-6 text-accent animate-spin" />
<Icon icon="svg-spinners:ring-resize" className="h-6 w-6 text-accent animate-spin" />
</div>
)}
</div>
@@ -1,6 +1,7 @@
import { useSortable } from '@dnd-kit/sortable';
import { CSS } from '@dnd-kit/utilities';
import { GripVertical, Mic, MoreHorizontal, Play, Trash2 } from 'lucide-react';
import { HugeiconsIcon } from '@hugeicons/react';
import { DragDropVerticalIcon, MoreHorizontalIcon, PlayIcon, Delete01Icon } from '@hugeicons/core-free-icons';
import { Button } from '@/components/ui/button';
import {
DropdownMenu,
@@ -9,6 +10,7 @@ import {
DropdownMenuTrigger,
} from '@/components/ui/dropdown-menu';
import { Textarea } from '@/components/ui/textarea';
import { ProfileAvatar } from '@/components/VoiceProfiles/ProfileAvatar';
import type { StoryItemDetail } from '@/lib/api/types';
import { cn } from '@/lib/utils/cn';
import { useStoryStore } from '@/stores/storyStore';
@@ -68,15 +70,18 @@ export function StoryChatItem({
className="shrink-0 cursor-grab active:cursor-grabbing touch-none text-muted-foreground hover:text-foreground transition-colors"
{...dragHandleProps}
>
<GripVertical className="h-5 w-5" />
<HugeiconsIcon icon={DragDropVerticalIcon} size={20} className="h-5 w-5" />
</button>
)}
{/* Voice Icon */}
{/* Voice Avatar */}
<div className="shrink-0">
<div className="h-10 w-10 rounded-full bg-muted flex items-center justify-center">
<Mic className="h-5 w-5 text-muted-foreground" />
</div>
<ProfileAvatar
profileId={item.profile_id}
size="lg"
grayscale={!isCurrentlyPlaying}
alt={`${item.profile_name} avatar`}
/>
</div>
{/* Content */}
@@ -101,16 +106,16 @@ export function StoryChatItem({
<DropdownMenu>
<DropdownMenuTrigger asChild>
<Button variant="ghost" size="icon" className="h-8 w-8" aria-label="Actions">
<MoreHorizontal className="h-4 w-4" />
<HugeiconsIcon icon={MoreHorizontalIcon} size={16} className="h-4 w-4" />
</Button>
</DropdownMenuTrigger>
<DropdownMenuContent align="end">
<DropdownMenuItem onClick={handlePlay}>
<Play className="mr-2 h-4 w-4" />
<HugeiconsIcon icon={PlayIcon} size={16} className="mr-2 h-4 w-4" />
Play from here
</DropdownMenuItem>
<DropdownMenuItem onClick={onRemove} className="text-destructive focus:text-destructive">
<Trash2 className="mr-2 h-4 w-4" />
<HugeiconsIcon icon={Delete01Icon} size={16} className="mr-2 h-4 w-4" />
Remove from Story
</DropdownMenuItem>
</DropdownMenuContent>
@@ -13,7 +13,8 @@ import {
sortableKeyboardCoordinates,
verticalListSortingStrategy,
} from '@dnd-kit/sortable';
import { Download, Plus } from 'lucide-react';
import { HugeiconsIcon } from '@hugeicons/react';
import { Download01Icon, Add01Icon } from '@hugeicons/core-free-icons';
import { useEffect, useMemo, useRef, useState } from 'react';
import { Button } from '@/components/ui/button';
import { Input } from '@/components/ui/input';
@@ -131,13 +132,13 @@ export function StoryContent() {
}
}, [isPlaying]);
const handleRemoveItem = (generationId: string) => {
const handleRemoveItem = (itemId: string) => {
if (!story) return;
removeItem.mutate(
{
storyId: story.id,
generationId,
itemId,
},
{
onError: (error) => {
@@ -271,7 +272,7 @@ export function StoryContent() {
<Popover open={isAddOpen} onOpenChange={setIsAddOpen}>
<PopoverTrigger asChild>
<Button variant="outline" size="sm">
<Plus className="mr-2 h-4 w-4" />
<HugeiconsIcon icon={Add01Icon} size={16} className="mr-2 h-4 w-4" />
Add
</Button>
</PopoverTrigger>
@@ -316,7 +317,7 @@ export function StoryContent() {
onClick={handleExportAudio}
disabled={exportAudio.isPending}
>
<Download className="mr-2 h-4 w-4" />
<HugeiconsIcon icon={Download01Icon} size={16} className="mr-2 h-4 w-4" />
Export Audio
</Button>
)}
@@ -360,7 +361,7 @@ export function StoryContent() {
item={item}
storyId={story.id}
index={index}
onRemove={() => handleRemoveItem(item.generation_id)}
onRemove={() => handleRemoveItem(item.id)}
currentTimeMs={currentTimeMs}
isPlaying={isPlaying && playbackStoryId === story.id}
/>
+49 -33
View File
@@ -1,14 +1,6 @@
import { Plus, BookOpen, MoreHorizontal, Pencil, Trash2 } from 'lucide-react';
import { useState } from 'react';
import { Button } from '@/components/ui/button';
import {
Dialog,
DialogContent,
DialogDescription,
DialogFooter,
DialogHeader,
DialogTitle,
} from '@/components/ui/dialog';
import { HugeiconsIcon } from '@hugeicons/react';
import { Add01Icon, Book01Icon, MoreHorizontalIcon, PencilIcon, Delete01Icon } from '@hugeicons/core-free-icons';
import { useState, useMemo } from 'react';
import {
AlertDialog,
AlertDialogAction,
@@ -19,6 +11,15 @@ import {
AlertDialogHeader,
AlertDialogTitle,
} from '@/components/ui/alert-dialog';
import { Button } from '@/components/ui/button';
import {
Dialog,
DialogContent,
DialogDescription,
DialogFooter,
DialogHeader,
DialogTitle,
} from '@/components/ui/dialog';
import {
DropdownMenu,
DropdownMenuContent,
@@ -26,35 +27,46 @@ import {
DropdownMenuTrigger,
} from '@/components/ui/dropdown-menu';
import { Input } from '@/components/ui/input';
import { Textarea } from '@/components/ui/textarea';
import { Label } from '@/components/ui/label';
import { Textarea } from '@/components/ui/textarea';
import { useToast } from '@/components/ui/use-toast';
import {
useStories,
useCreateStory,
useUpdateStory,
useDeleteStory,
} from '@/lib/hooks/useStories';
import { useStoryStore } from '@/stores/storyStore';
import { useStories, useCreateStory, useUpdateStory, useDeleteStory, useStory } from '@/lib/hooks/useStories';
import { cn } from '@/lib/utils/cn';
import { formatDate } from '@/lib/utils/format';
import { useStoryStore } from '@/stores/storyStore';
export function StoryList() {
const { data: stories, isLoading } = useStories();
const selectedStoryId = useStoryStore((state) => state.selectedStoryId);
const setSelectedStoryId = useStoryStore((state) => state.setSelectedStoryId);
const trackEditorHeight = useStoryStore((state) => state.trackEditorHeight);
const { data: currentStory } = useStory(selectedStoryId);
const createStory = useCreateStory();
const updateStory = useUpdateStory();
const deleteStory = useDeleteStory();
const [createDialogOpen, setCreateDialogOpen] = useState(false);
const [editDialogOpen, setEditDialogOpen] = useState(false);
const [deleteDialogOpen, setDeleteDialogOpen] = useState(false);
const [editingStory, setEditingStory] = useState<{ id: string; name: string; description?: string } | null>(null);
const [editingStory, setEditingStory] = useState<{
id: string;
name: string;
description?: string;
} | null>(null);
const [deletingStoryId, setDeletingStoryId] = useState<string | null>(null);
const [newStoryName, setNewStoryName] = useState('');
const [newStoryDescription, setNewStoryDescription] = useState('');
const { toast } = useToast();
// Calculate bottom padding to account for FloatingGenerateBox and StoryTrackEditor
const hasTrackEditor = currentStory && currentStory.items.length > 0;
const bottomPadding = useMemo(() => {
// FloatingGenerateBox height (~100px) + gap (24px)
const generateBoxHeight = 124;
// Track editor height when visible
const editorHeight = hasTrackEditor ? trackEditorHeight + 24 : 0;
return generateBoxHeight + editorHeight;
}, [hasTrackEditor, trackEditorHeight]);
const handleCreateStory = () => {
if (!newStoryName.trim()) {
toast({
@@ -178,16 +190,19 @@ export function StoryList() {
<div className="flex items-center justify-between mb-4 px-1">
<h2 className="text-2xl font-bold">Stories</h2>
<Button onClick={() => setCreateDialogOpen(true)} size="sm">
<Plus className="mr-2 h-4 w-4" />
<HugeiconsIcon icon={Add01Icon} size={16} className="mr-2 h-4 w-4" />
New Story
</Button>
</div>
{/* Story List */}
<div className="flex-1 min-h-0 overflow-y-auto space-y-2">
<div
className="flex-1 min-h-0 overflow-y-auto space-y-2"
style={{ paddingBottom: `${bottomPadding}px` }}
>
{storyList.length === 0 ? (
<div className="text-center py-12 px-5 border-2 border-dashed border-muted rounded-md text-muted-foreground">
<BookOpen className="h-12 w-12 mx-auto mb-4 opacity-50" />
<div className="text-center py-12 px-5 border-2 border-dashed border-muted rounded-2xl text-muted-foreground">
<HugeiconsIcon icon={Book01Icon} size={48} className="h-12 w-12 mx-auto mb-4 opacity-50" />
<p className="text-sm">No stories yet</p>
<p className="text-xs mt-2">Create your first story to get started</p>
</div>
@@ -196,7 +211,7 @@ export function StoryList() {
<div
key={story.id}
className={cn(
'h-24 p-4 border rounded-md transition-colors group flex items-center',
'h-24 p-4 border rounded-2xl transition-colors group flex items-center',
selectedStoryId === story.id && 'bg-muted border-primary',
)}
>
@@ -213,7 +228,9 @@ export function StoryList() {
</p>
)}
<div className="flex items-center gap-3 mt-2 text-xs text-muted-foreground">
<span>{story.item_count} {story.item_count === 1 ? 'item' : 'items'}</span>
<span>
{story.item_count} {story.item_count === 1 ? 'item' : 'items'}
</span>
<span>•</span>
<span>{formatDate(story.updated_at)}</span>
</div>
@@ -226,19 +243,19 @@ export function StoryList() {
className="h-8 w-8 opacity-0 group-hover:opacity-100 transition-opacity"
onClick={(e) => e.stopPropagation()}
>
<MoreHorizontal className="h-4 w-4" />
<HugeiconsIcon icon={MoreHorizontalIcon} size={16} className="h-4 w-4" />
</Button>
</DropdownMenuTrigger>
<DropdownMenuContent align="end">
<DropdownMenuItem onClick={() => handleEditClick(story)}>
<Pencil className="mr-2 h-4 w-4" />
<HugeiconsIcon icon={PencilIcon} size={16} className="mr-2 h-4 w-4" />
Edit
</DropdownMenuItem>
<DropdownMenuItem
onClick={() => handleDeleteClick(story.id)}
className="text-destructive focus:text-destructive"
>
<Trash2 className="mr-2 h-4 w-4" />
<HugeiconsIcon icon={Delete01Icon} size={16} className="mr-2 h-4 w-4" />
Delete
</DropdownMenuItem>
</DropdownMenuContent>
@@ -300,9 +317,7 @@ export function StoryList() {
<DialogContent>
<DialogHeader>
<DialogTitle>Edit Story</DialogTitle>
<DialogDescription>
Update the story name and description.
</DialogDescription>
<DialogDescription>Update the story name and description.</DialogDescription>
</DialogHeader>
<div className="space-y-4 py-4">
<div className="space-y-2">
@@ -347,7 +362,8 @@ export function StoryList() {
<AlertDialogHeader>
<AlertDialogTitle>Are you sure?</AlertDialogTitle>
<AlertDialogDescription>
This will permanently delete the story and all its items. This action cannot be undone.
This will permanently delete the story and all its items. This action cannot be
undone.
</AlertDialogDescription>
</AlertDialogHeader>
<AlertDialogFooter>
File diff suppressed because it is too large Load Diff
@@ -1,8 +1,32 @@
import { Mic, Pause, Play, Square } from 'lucide-react';
import { HugeiconsIcon } from '@hugeicons/react';
import { Mic01Icon, PauseIcon, PlayIcon, SquareIcon } from '@hugeicons/core-free-icons';
import { memo, useEffect, useState } from 'react';
import { Visualizer } from 'react-sound-visualizer';
import { Button } from '@/components/ui/button';
import { FormControl, FormItem, FormLabel, FormMessage } from '@/components/ui/form';
import { FormControl, FormItem, FormMessage } from '@/components/ui/form';
import { formatAudioDuration } from '@/lib/utils/audio';
const MemoizedWaveform = memo(function MemoizedWaveform({
audioStream,
}: {
audioStream: MediaStream;
}) {
return (
<div className="absolute inset-0 pointer-events-none flex items-center justify-center opacity-30">
<Visualizer audio={audioStream} autoStart strokeColor="#b39a3d">
{({ canvasRef }) => (
<canvas
ref={canvasRef}
width={500}
height={150}
className="w-full h-full"
/>
)}
</Visualizer>
</div>
);
});
interface AudioSampleRecordingProps {
file: File | null | undefined;
isRecording: boolean;
@@ -14,6 +38,7 @@ interface AudioSampleRecordingProps {
onPlayPause: () => void;
isPlaying: boolean;
isTranscribing?: boolean;
showWaveform?: boolean;
}
export function AudioSampleRecording({
@@ -27,29 +52,67 @@ export function AudioSampleRecording({
onPlayPause,
isPlaying,
isTranscribing = false,
showWaveform = true,
}: AudioSampleRecordingProps) {
const [audioStream, setAudioStream] = useState<MediaStream | null>(null);
// Request microphone access when component mounts
useEffect(() => {
if (!showWaveform) return;
let stream: MediaStream | null = null;
navigator.mediaDevices
.getUserMedia({ audio: true, video: false })
.then((s) => {
stream = s;
setAudioStream(s);
})
.catch((err) => {
console.warn('Could not access microphone for visualization:', err);
});
return () => {
if (stream) {
stream.getTracks().forEach((track) => {
track.stop();
});
}
};
}, [showWaveform]);
return (
<FormItem>
<FormLabel>Record Audio</FormLabel>
<FormControl>
<div className="space-y-4">
{!isRecording && !file && (
<div className="flex flex-col items-center justify-center gap-4 p-4 border-2 border-dashed rounded-lg min-h-[180px]">
<Button type="button" onClick={onStart} size="lg" className="flex items-center gap-2">
<Mic className="h-5 w-5" />
<div className="relative flex flex-col items-center justify-center gap-4 p-4 border-2 border-dashed rounded-lg min-h-[180px] overflow-hidden">
{showWaveform && audioStream && (
<MemoizedWaveform audioStream={audioStream} />
)}
<Button
type="button"
onClick={onStart}
size="lg"
className="relative z-10 flex items-center gap-2"
>
<HugeiconsIcon icon={Mic01Icon} size={20} className="h-5 w-5" />
Start Recording
</Button>
<p className="text-sm text-muted-foreground text-center">
<p className="relative z-10 text-sm text-muted-foreground text-center">
Click to start recording. Maximum duration: 30 seconds.
</p>
</div>
)}
{isRecording && (
<div className="flex flex-col items-center justify-center gap-4 p-4 border-2 border-destructive rounded-lg bg-destructive/5 min-h-[180px]">
<div className="flex items-center gap-4">
<div className="relative flex flex-col items-center justify-center gap-4 p-4 border-2 border-accent rounded-lg bg-accent/5 min-h-[180px] overflow-hidden">
{showWaveform && audioStream && (
<MemoizedWaveform audioStream={audioStream} />
)}
<div className="relative z-10 flex items-center gap-4">
<div className="flex items-center gap-2">
<div className="h-3 w-3 rounded-full bg-destructive animate-pulse" />
<div className="h-3 w-3 rounded-full bg-accent animate-pulse" />
<span className="text-lg font-mono font-semibold">
{formatAudioDuration(duration)}
</span>
@@ -58,13 +121,12 @@ export function AudioSampleRecording({
<Button
type="button"
onClick={onStop}
variant="destructive"
className="flex items-center gap-2"
className="relative z-10 flex items-center gap-2 bg-accent text-accent-foreground hover:bg-accent/90"
>
<Square className="h-4 w-4" />
<HugeiconsIcon icon={SquareIcon} size={16} className="h-4 w-4" />
Stop Recording
</Button>
<p className="text-sm text-muted-foreground text-center">
<p className="relative z-10 text-sm text-muted-foreground text-center">
{formatAudioDuration(30 - duration)} remaining
</p>
</div>
@@ -73,13 +135,13 @@ export function AudioSampleRecording({
{file && !isRecording && (
<div className="flex flex-col items-center justify-center gap-4 p-4 border-2 border-primary rounded-lg bg-primary/5 min-h-[180px]">
<div className="flex items-center gap-2">
<Mic className="h-5 w-5 text-primary" />
<HugeiconsIcon icon={Mic01Icon} size={20} className="h-5 w-5 text-primary" />
<span className="font-medium">Recording complete</span>
</div>
<p className="text-sm text-muted-foreground text-center">File: {file.name}</p>
<div className="flex gap-2">
<Button type="button" size="icon" variant="outline" onClick={onPlayPause}>
{isPlaying ? <Pause className="h-4 w-4" /> : <Play className="h-4 w-4" />}
{isPlaying ? <HugeiconsIcon icon={PauseIcon} size={16} className="h-4 w-4" /> : <HugeiconsIcon icon={PlayIcon} size={16} className="h-4 w-4" />}
</Button>
<Button
type="button"
@@ -88,7 +150,7 @@ export function AudioSampleRecording({
disabled={isTranscribing}
className="flex items-center gap-2"
>
<Mic className="h-4 w-4" />
<HugeiconsIcon icon={Mic01Icon} size={16} className="h-4 w-4" />
{isTranscribing ? 'Transcribing...' : 'Transcribe'}
</Button>
<Button
@@ -1,6 +1,7 @@
import { Mic, Monitor, Pause, Play, Square } from 'lucide-react';
import { HugeiconsIcon } from '@hugeicons/react';
import { Mic01Icon, DeskIcon, PauseIcon, PlayIcon, SquareIcon } from '@hugeicons/core-free-icons';
import { Button } from '@/components/ui/button';
import { FormControl, FormItem, FormLabel, FormMessage } from '@/components/ui/form';
import { FormControl, FormItem, FormMessage } from '@/components/ui/form';
import { formatAudioDuration } from '@/lib/utils/audio';
interface AudioSampleSystemProps {
@@ -30,13 +31,12 @@ export function AudioSampleSystem({
}: AudioSampleSystemProps) {
return (
<FormItem>
<FormLabel>Capture System Audio</FormLabel>
<FormControl>
<div className="space-y-4">
{!isRecording && !file && (
<div className="flex flex-col items-center justify-center gap-4 p-4 border-2 border-dashed rounded-lg min-h-[180px]">
<Button type="button" onClick={onStart} size="lg" className="flex items-center gap-2">
<Monitor className="h-5 w-5" />
<HugeiconsIcon icon={DeskIcon} size={20} className="h-5 w-5" />
Start Capture
</Button>
<p className="text-sm text-muted-foreground text-center">
@@ -61,7 +61,7 @@ export function AudioSampleSystem({
variant="destructive"
className="flex items-center gap-2"
>
<Square className="h-4 w-4" />
<HugeiconsIcon icon={SquareIcon} size={16} className="h-4 w-4" />
Stop Capture
</Button>
<p className="text-sm text-muted-foreground text-center">
@@ -73,13 +73,13 @@ export function AudioSampleSystem({
{file && !isRecording && (
<div className="flex flex-col items-center justify-center gap-4 p-4 border-2 border-primary rounded-lg bg-primary/5 min-h-[180px]">
<div className="flex items-center gap-2">
<Monitor className="h-5 w-5 text-primary" />
<HugeiconsIcon icon={DeskIcon} size={20} className="h-5 w-5 text-primary" />
<span className="font-medium">Capture complete</span>
</div>
<p className="text-sm text-muted-foreground text-center">File: {file.name}</p>
<div className="flex gap-2">
<Button type="button" size="icon" variant="outline" onClick={onPlayPause}>
{isPlaying ? <Pause className="h-4 w-4" /> : <Play className="h-4 w-4" />}
{isPlaying ? <HugeiconsIcon icon={PauseIcon} size={16} className="h-4 w-4" /> : <HugeiconsIcon icon={PlayIcon} size={16} className="h-4 w-4" />}
</Button>
<Button
type="button"
@@ -88,7 +88,7 @@ export function AudioSampleSystem({
disabled={isTranscribing}
className="flex items-center gap-2"
>
<Mic className="h-4 w-4" />
<HugeiconsIcon icon={Mic01Icon} size={16} className="h-4 w-4" />
{isTranscribing ? 'Transcribing...' : 'Transcribe'}
</Button>
<Button
@@ -1,7 +1,8 @@
import { Mic, Pause, Play, Upload } from 'lucide-react';
import { HugeiconsIcon } from '@hugeicons/react';
import { Mic01Icon, PauseIcon, PlayIcon, Upload01Icon } from '@hugeicons/core-free-icons';
import { useRef, useState } from 'react';
import { Button } from '@/components/ui/button';
import { FormControl, FormItem, FormLabel, FormMessage } from '@/components/ui/form';
import { FormControl, FormItem, FormMessage } from '@/components/ui/form';
interface AudioSampleUploadProps {
file: File | null | undefined;
@@ -31,7 +32,6 @@ export function AudioSampleUpload({
return (
<FormItem>
<FormLabel>Audio File</FormLabel>
<FormControl>
<div className="flex flex-col gap-2">
<input
@@ -90,7 +90,7 @@ export function AudioSampleUpload({
onClick={() => fileInputRef.current?.click()}
className="flex items-center gap-2"
>
<Upload className="h-5 w-5" />
<HugeiconsIcon icon={Upload01Icon} size={20} className="h-5 w-5" />
Choose File
</Button>
<p className="text-sm text-muted-foreground text-center">
@@ -100,7 +100,7 @@ export function AudioSampleUpload({
) : (
<>
<div className="flex items-center gap-2">
<Upload className="h-5 w-5 text-primary" />
<HugeiconsIcon icon={Upload01Icon} size={20} className="h-5 w-5 text-primary" />
<span className="font-medium">File uploaded</span>
</div>
<p className="text-sm text-muted-foreground text-center">File: {file.name}</p>
@@ -112,7 +112,7 @@ export function AudioSampleUpload({
onClick={onPlayPause}
disabled={isValidating}
>
{isPlaying ? <Pause className="h-4 w-4" /> : <Play className="h-4 w-4" />}
{isPlaying ? <HugeiconsIcon icon={PauseIcon} size={16} className="h-4 w-4" /> : <HugeiconsIcon icon={PlayIcon} size={16} className="h-4 w-4" />}
</Button>
<Button
type="button"
@@ -121,7 +121,7 @@ export function AudioSampleUpload({
disabled={isTranscribing || isValidating || isDisabled}
className="flex items-center gap-2"
>
<Mic className="h-4 w-4" />
<HugeiconsIcon icon={Mic01Icon} size={16} className="h-4 w-4" />
{isTranscribing ? 'Transcribing...' : 'Transcribe'}
</Button>
<Button
@@ -0,0 +1,80 @@
import { HugeiconsIcon } from '@hugeicons/react';
import { Mic01Icon } from '@hugeicons/core-free-icons';
import { useState } from 'react';
import { cn } from '@/lib/utils/cn';
import { useServerStore } from '@/stores/serverStore';
interface ProfileAvatarProps {
profileId: string;
avatarPath?: string | null;
size?: 'sm' | 'md' | 'lg' | 'xl';
grayscale?: boolean;
className?: string;
alt?: string;
}
const sizeClasses = {
sm: 'h-6 w-6',
md: 'h-8 w-8',
lg: 'h-10 w-10',
xl: 'h-24 w-24',
};
const iconSizes = {
sm: 14,
md: 16,
lg: 20,
xl: 40,
};
const iconClassNames = {
sm: 'h-3.5 w-3.5',
md: 'h-4 w-4',
lg: 'h-5 w-5',
xl: 'h-10 w-10',
};
export function ProfileAvatar({
profileId,
avatarPath,
size = 'md',
grayscale = false,
className,
alt = 'Profile avatar',
}: ProfileAvatarProps) {
const [avatarError, setAvatarError] = useState(false);
const serverUrl = useServerStore((state) => state.serverUrl);
// If avatarPath is explicitly null or empty string, don't try to load avatar
// Otherwise, always try to load (avatarPath might not be available in all contexts)
const avatarUrl =
avatarPath === null || avatarPath === '' ? null : `${serverUrl}/profiles/${profileId}/avatar`;
return (
<div
className={cn(
sizeClasses[size],
'rounded-full bg-muted flex items-center justify-center shrink-0 overflow-hidden',
className,
)}
>
{avatarUrl && !avatarError ? (
<img
src={avatarUrl}
alt={alt}
className={cn(
'h-full w-full object-cover transition-all duration-200',
grayscale && 'grayscale',
)}
onError={() => setAvatarError(true)}
/>
) : (
<HugeiconsIcon
icon={Mic01Icon}
size={iconSizes[size]}
className={cn(iconClassNames[size], 'text-muted-foreground')}
/>
)}
</div>
);
}
@@ -1,4 +1,5 @@
import { Download, Edit, Mic, Trash2 } from 'lucide-react';
import { HugeiconsIcon } from '@hugeicons/react';
import { Download01Icon, Edit01Icon, Delete01Icon } from '@hugeicons/core-free-icons';
import { useState } from 'react';
import { Badge } from '@/components/ui/badge';
import { Button } from '@/components/ui/button';
@@ -12,6 +13,7 @@ import {
DialogHeader,
DialogTitle,
} from '@/components/ui/dialog';
import { ProfileAvatar } from '@/components/VoiceProfiles/ProfileAvatar';
import type { VoiceProfileResponse } from '@/lib/api/types';
import { useDeleteProfile, useExportProfile } from '@/lib/hooks/useProfiles';
import { cn } from '@/lib/utils/cn';
@@ -67,9 +69,13 @@ export function ProfileCard({ profile }: ProfileCardProps) {
>
<CardHeader className="p-3 pb-2">
<CardTitle className="flex items-center gap-1.5 text-base font-medium">
<div className="h-6 w-6 rounded-full bg-muted flex items-center justify-center shrink-0">
<Mic className="h-3.5 w-3.5 text-muted-foreground" />
</div>
<ProfileAvatar
profileId={profile.id}
avatarPath={profile.avatar_path}
size="sm"
grayscale={!isSelected}
alt={`${profile.name} avatar`}
/>
<span className="break-words">{profile.name}</span>
</CardTitle>
</CardHeader>
@@ -84,13 +90,13 @@ export function ProfileCard({ profile }: ProfileCardProps) {
</div>
<div className="flex gap-0.5 justify-end items-end mt-auto">
<CircleButton
icon={Download}
icon={(props) => <HugeiconsIcon icon={Download01Icon} size={14} {...props} />}
onClick={handleExport}
disabled={exportProfile.isPending}
aria-label="Export profile"
/>
<CircleButton
icon={Edit}
icon={(props) => <HugeiconsIcon icon={Edit01Icon} size={14} {...props} />}
onClick={(e) => {
e.stopPropagation();
handleEdit();
@@ -98,7 +104,7 @@ export function ProfileCard({ profile }: ProfileCardProps) {
aria-label="Edit profile"
/>
<CircleButton
icon={Trash2}
icon={(props) => <HugeiconsIcon icon={Delete01Icon} size={14} {...props} />}
onClick={handleDeleteClick}
disabled={deleteProfile.isPending}
aria-label="Delete profile"
+485 -215
View File
@@ -1,6 +1,7 @@
import { zodResolver } from '@hookform/resolvers/zod';
import { Mic, Monitor, Upload } from 'lucide-react';
import { useEffect, useState } from 'react';
import { HugeiconsIcon } from '@hugeicons/react';
import { Edit02Icon, Mic01Icon, DeskIcon, Upload01Icon, Cancel01Icon } from '@hugeicons/core-free-icons';
import { useEffect, useRef, useState } from 'react';
import { useForm } from 'react-hook-form';
import * as z from 'zod';
import { Button } from '@/components/ui/button';
@@ -36,14 +37,17 @@ import { useAudioRecording } from '@/lib/hooks/useAudioRecording';
import {
useAddSample,
useCreateProfile,
useDeleteAvatar,
useProfile,
useUpdateProfile,
useUploadAvatar,
} from '@/lib/hooks/useProfiles';
import { useSystemAudioCapture } from '@/lib/hooks/useSystemAudioCapture';
import { useTranscription } from '@/lib/hooks/useTranscription';
import { isTauri } from '@/lib/tauri';
import { formatAudioDuration, getAudioDuration } from '@/lib/utils/audio';
import { useUIStore } from '@/stores/uiStore';
import { usePlatform } from '@/platform/PlatformContext';
import { useServerStore } from '@/stores/serverStore';
import { type ProfileFormDraft, useUIStore } from '@/stores/uiStore';
import { AudioSampleRecording } from './AudioSampleRecording';
import { AudioSampleSystem } from './AudioSampleSystem';
import { AudioSampleUpload } from './AudioSampleUpload';
@@ -57,6 +61,7 @@ const baseProfileSchema = z.object({
language: z.enum(LANGUAGE_CODES as [LanguageCode, ...LanguageCode[]]),
sampleFile: z.instanceof(File).optional(),
referenceText: z.string().max(1000).optional(),
avatarFile: z.instanceof(File).optional(),
});
const profileSchema = baseProfileSchema.refine(
@@ -75,22 +80,52 @@ const profileSchema = baseProfileSchema.refine(
type ProfileFormValues = z.infer<typeof profileSchema>;
// Helper to convert File to base64
async function fileToBase64(file: File): Promise<string> {
return new Promise((resolve, reject) => {
const reader = new FileReader();
reader.onload = () => resolve(reader.result as string);
reader.onerror = reject;
reader.readAsDataURL(file);
});
}
// Helper to convert base64 to File
function base64ToFile(base64: string, fileName: string, fileType: string): File {
const arr = base64.split(',');
const bstr = atob(arr[1]);
let n = bstr.length;
const u8arr = new Uint8Array(n);
while (n--) {
u8arr[n] = bstr.charCodeAt(n);
}
return new File([u8arr], fileName, { type: fileType });
}
export function ProfileForm() {
const platform = usePlatform();
const open = useUIStore((state) => state.profileDialogOpen);
const setOpen = useUIStore((state) => state.setProfileDialogOpen);
const editingProfileId = useUIStore((state) => state.editingProfileId);
const setEditingProfileId = useUIStore((state) => state.setEditingProfileId);
const profileFormDraft = useUIStore((state) => state.profileFormDraft);
const setProfileFormDraft = useUIStore((state) => state.setProfileFormDraft);
const { data: editingProfile } = useProfile(editingProfileId || '');
const createProfile = useCreateProfile();
const updateProfile = useUpdateProfile();
const addSample = useAddSample();
const uploadAvatar = useUploadAvatar();
const deleteAvatar = useDeleteAvatar();
const transcribe = useTranscription();
const { toast } = useToast();
const [sampleMode, setSampleMode] = useState<'upload' | 'record' | 'system'>('upload');
const [sampleMode, setSampleMode] = useState<'upload' | 'record' | 'system'>('record');
const [audioDuration, setAudioDuration] = useState<number | null>(null);
const [isValidatingAudio, setIsValidatingAudio] = useState(false);
const [avatarPreview, setAvatarPreview] = useState<string | null>(null);
const avatarInputRef = useRef<HTMLInputElement>(null);
const { isPlaying, playPause, cleanup: cleanupAudio } = useAudioPlayer();
const isCreating = !editingProfileId;
const serverUrl = useServerStore((state) => state.serverUrl);
const form = useForm<ProfileFormValues>({
resolver: zodResolver(profileSchema),
@@ -100,10 +135,12 @@ export function ProfileForm() {
language: 'en',
sampleFile: undefined,
referenceText: '',
avatarFile: undefined,
},
});
const selectedFile = form.watch('sampleFile');
const selectedAvatarFile = form.watch('avatarFile');
// Validate audio duration when file is selected
useEffect(() => {
@@ -220,6 +257,20 @@ export function ProfileForm() {
}
}, [systemRecordingError, toast]);
// Handle avatar preview
useEffect(() => {
if (selectedAvatarFile instanceof File) {
const url = URL.createObjectURL(selectedAvatarFile);
setAvatarPreview(url);
return () => URL.revokeObjectURL(url);
} else if (editingProfile?.avatar_path) {
setAvatarPreview(`${serverUrl}/profiles/${editingProfile.id}/avatar`);
} else {
setAvatarPreview(null);
}
}, [selectedAvatarFile, editingProfile, serverUrl]);
// Restore form state from draft or editing profile
useEffect(() => {
if (editingProfile) {
form.reset({
@@ -228,18 +279,46 @@ export function ProfileForm() {
language: editingProfile.language as LanguageCode,
sampleFile: undefined,
referenceText: undefined,
avatarFile: undefined,
});
} else {
} else if (profileFormDraft && open) {
// Restore from draft when opening in create mode
form.reset({
name: profileFormDraft.name,
description: profileFormDraft.description,
language: profileFormDraft.language as LanguageCode,
referenceText: profileFormDraft.referenceText,
sampleFile: undefined,
avatarFile: undefined,
});
setSampleMode(profileFormDraft.sampleMode);
// Restore the file if we have it saved
if (
profileFormDraft.sampleFileData &&
profileFormDraft.sampleFileName &&
profileFormDraft.sampleFileType
) {
const file = base64ToFile(
profileFormDraft.sampleFileData,
profileFormDraft.sampleFileName,
profileFormDraft.sampleFileType,
);
form.setValue('sampleFile', file);
}
} else if (!open) {
// Only reset to defaults when modal is closed and no draft
form.reset({
name: '',
description: '',
language: 'en',
sampleFile: undefined,
referenceText: undefined,
avatarFile: undefined,
});
setSampleMode('upload');
setSampleMode('record');
setAvatarPreview(null);
}
}, [editingProfile, form]);
}, [editingProfile, profileFormDraft, open, form]);
async function handleTranscribe() {
const file = form.getValues('sampleFile');
@@ -281,6 +360,52 @@ export function ProfileForm() {
playPause(file);
}
function handleAvatarFileChange(e: React.ChangeEvent<HTMLInputElement>) {
const file = e.target.files?.[0];
if (file) {
if (!file.type.startsWith('image/')) {
toast({
title: 'Invalid file type',
description: 'Please select an image file (PNG, JPG, or WebP)',
variant: 'destructive',
});
return;
}
if (file.size > 5 * 1024 * 1024) {
toast({
title: 'File too large',
description: 'Image must be less than 5MB',
variant: 'destructive',
});
return;
}
form.setValue('avatarFile', file);
}
}
async function handleRemoveAvatar() {
if (editingProfileId && editingProfile?.avatar_path) {
try {
await deleteAvatar.mutateAsync(editingProfileId);
toast({
title: 'Avatar removed',
description: 'Avatar image has been removed successfully.',
});
} catch (error) {
toast({
title: 'Failed to remove avatar',
description: error instanceof Error ? error.message : 'Unknown error',
variant: 'destructive',
});
}
}
form.setValue('avatarFile', undefined);
setAvatarPreview(null);
if (avatarInputRef.current) {
avatarInputRef.current.value = '';
}
}
async function onSubmit(data: ProfileFormValues) {
try {
if (editingProfileId) {
@@ -293,6 +418,24 @@ export function ProfileForm() {
language: data.language,
},
});
// Handle avatar upload/update if file changed
if (data.avatarFile) {
try {
await uploadAvatar.mutateAsync({
profileId: editingProfileId,
file: data.avatarFile,
});
} catch (avatarError) {
toast({
title: 'Avatar upload failed',
description:
avatarError instanceof Error ? avatarError.message : 'Failed to upload avatar',
variant: 'destructive',
});
}
}
toast({
title: 'Voice updated',
description: `"${data.name}" has been updated successfully.`,
@@ -369,6 +512,24 @@ export function ProfileForm() {
file: sampleFile,
referenceText: referenceText,
});
// Handle avatar upload if provided
if (data.avatarFile) {
try {
await uploadAvatar.mutateAsync({
profileId: profile.id,
file: data.avatarFile,
});
} catch (avatarError) {
toast({
title: 'Avatar upload failed',
description:
avatarError instanceof Error ? avatarError.message : 'Failed to upload avatar',
variant: 'destructive',
});
}
}
toast({
title: 'Profile created',
description: `"${data.name}" has been created with a sample.`,
@@ -383,6 +544,8 @@ export function ProfileForm() {
}
}
// Clear draft and reset form on success
setProfileFormDraft(null);
form.reset();
setEditingProfileId(null);
setOpen(false);
@@ -395,12 +558,41 @@ export function ProfileForm() {
}
}
function handleOpenChange(open: boolean) {
setOpen(open);
if (!open) {
async function handleOpenChange(newOpen: boolean) {
if (!newOpen && isCreating) {
// Save draft when closing the create modal
const values = form.getValues();
const hasContent =
values.name || values.description || values.referenceText || values.sampleFile;
if (hasContent) {
const draft: ProfileFormDraft = {
name: values.name || '',
description: values.description || '',
language: values.language || 'en',
referenceText: values.referenceText || '',
sampleMode,
};
// Save file as base64 if present
if (values.sampleFile) {
try {
draft.sampleFileName = values.sampleFile.name;
draft.sampleFileType = values.sampleFile.type;
draft.sampleFileData = await fileToBase64(values.sampleFile);
} catch {
// If file conversion fails, just don't save the file
}
}
setProfileFormDraft(draft);
}
}
setOpen(newOpen);
if (!newOpen) {
setEditingProfileId(null);
form.reset();
setSampleMode('upload');
// Don't reset form here - let the effect handle it based on draft state
if (isRecording) {
cancelRecording();
}
@@ -413,174 +605,119 @@ export function ProfileForm() {
return (
<Dialog open={open} onOpenChange={handleOpenChange}>
<DialogContent className="max-w-4xl">
<DialogHeader>
<DialogTitle>{editingProfileId ? 'Edit Voice' : 'Create Voice Profile'}</DialogTitle>
<DialogDescription>
{editingProfileId
? 'Update your voice profile details and manage samples.'
: 'Create a new voice profile with an audio sample to clone the voice.'}
</DialogDescription>
</DialogHeader>
<Form {...form}>
<form onSubmit={form.handleSubmit(onSubmit)}>
<div className="grid gap-6 grid-cols-2">
{/* Left column: Profile info */}
<div className="space-y-4">
<FormField
control={form.control}
name="name"
render={({ field }) => (
<FormItem>
<FormLabel>Name</FormLabel>
<FormControl>
<Input placeholder="My Voice" {...field} />
</FormControl>
<FormMessage />
</FormItem>
)}
/>
<FormField
control={form.control}
name="description"
render={({ field }) => (
<FormItem>
<FormLabel>Description (Optional)</FormLabel>
<FormControl>
<Textarea placeholder="Describe this voice..." {...field} />
</FormControl>
<FormMessage />
</FormItem>
)}
/>
<FormField
control={form.control}
name="language"
render={({ field }) => (
<FormItem>
<FormLabel>Language</FormLabel>
<Select onValueChange={field.onChange} defaultValue={field.value}>
<FormControl>
<SelectTrigger>
<SelectValue />
</SelectTrigger>
</FormControl>
<SelectContent>
{LANGUAGE_OPTIONS.map((lang) => (
<SelectItem key={lang.value} value={lang.value}>
{lang.label}
</SelectItem>
))}
</SelectContent>
</Select>
<FormMessage />
</FormItem>
)}
/>
<DialogContent className="max-w-none w-screen h-screen left-0 top-0 translate-x-0 translate-y-0 rounded-none p-6 overflow-y-auto">
<div className="max-w-5xl max-h-[85vh] mx-auto my-auto w-full flex flex-col">
<DialogHeader>
<DialogTitle className="text-2xl">
{editingProfileId ? 'Edit Voice' : 'Clone voice'}
</DialogTitle>
<DialogDescription>
{editingProfileId
? 'Update your voice profile details and manage samples.'
: 'Create a new voice profile with an audio sample to clone the voice.'}
</DialogDescription>
{isCreating && profileFormDraft && (
<div className="flex items-center gap-2 pt-2">
<span className="text-xs text-muted-foreground">Draft restored</span>
<Button
type="button"
variant="ghost"
size="sm"
className="h-6 px-2 text-xs text-muted-foreground"
onClick={() => {
setProfileFormDraft(null);
form.reset({
name: '',
description: '',
language: 'en',
sampleFile: undefined,
referenceText: '',
});
setSampleMode('record');
}}
>
<HugeiconsIcon icon={Cancel01Icon} size={12} className="h-3 w-3 mr-1" />
Discard
</Button>
</div>
)}
</DialogHeader>
{/* Right column: Sample management */}
<div className="space-y-4 border-l pl-6">
{isCreating ? (
<>
<div>
<h3 className="text-sm font-medium mb-2">Add Sample</h3>
<p className="text-sm text-muted-foreground mb-4">
Provide an audio sample to clone the voice. You can add more samples later.
</p>
</div>
<Tabs
value={sampleMode}
onValueChange={(v) => {
const newMode = v as 'upload' | 'record' | 'system';
// Cancel any active recordings when switching modes
if (isRecording && newMode !== 'record') {
cancelRecording();
}
if (isSystemRecording && newMode !== 'system') {
cancelSystemRecording();
}
setSampleMode(newMode);
}}
>
<TabsList
className={`grid w-full ${isTauri() && isSystemAudioSupported ? 'grid-cols-3' : 'grid-cols-2'}`}
<Form {...form}>
<form onSubmit={form.handleSubmit(onSubmit)} className="flex-1 min-h-0 flex flex-col">
<div className="grid gap-6 grid-cols-2 flex-1 overflow-y-auto min-h-0">
{/* Left column: Sample management */}
<div className="space-y-4 border-r pr-6">
{isCreating ? (
<>
<Tabs
className="pt-4"
value={sampleMode}
onValueChange={(v) => {
const newMode = v as 'upload' | 'record' | 'system';
// Cancel any active recordings when switching modes
if (isRecording && newMode !== 'record') {
cancelRecording();
}
if (isSystemRecording && newMode !== 'system') {
cancelSystemRecording();
}
setSampleMode(newMode);
}}
>
<TabsTrigger value="upload" className="flex items-center gap-2">
<Upload className="h-4 w-4 shrink-0" />
Upload
</TabsTrigger>
<TabsTrigger value="record" className="flex items-center gap-2">
<Mic className="h-4 w-4 shrink-0" />
Record
</TabsTrigger>
{isTauri() && isSystemAudioSupported && (
<TabsTrigger value="system" className="flex items-center gap-2">
<Monitor className="h-4 w-4 shrink-0" />
System Audio
<TabsList
className={`grid w-full ${platform.metadata.isTauri && isSystemAudioSupported ? 'grid-cols-3' : 'grid-cols-2'}`}
>
<TabsTrigger value="upload" className="flex items-center gap-2">
<HugeiconsIcon icon={Upload01Icon} size={16} className="h-4 w-4 shrink-0" />
Upload
</TabsTrigger>
)}
</TabsList>
<TabsContent value="upload" className="space-y-4">
<FormField
control={form.control}
name="sampleFile"
render={({ field: { onChange, name } }) => (
<AudioSampleUpload
file={selectedFile}
onFileChange={onChange}
onTranscribe={handleTranscribe}
onPlayPause={handlePlayPause}
isPlaying={isPlaying}
isValidating={isValidatingAudio}
isTranscribing={transcribe.isPending}
isDisabled={
audioDuration !== null && audioDuration > MAX_AUDIO_DURATION_SECONDS
}
fieldName={name}
/>
<TabsTrigger value="record" className="flex items-center gap-2">
<HugeiconsIcon icon={Mic01Icon} size={16} className="h-4 w-4 shrink-0" />
Record
</TabsTrigger>
{platform.metadata.isTauri && isSystemAudioSupported && (
<TabsTrigger value="system" className="flex items-center gap-2">
<HugeiconsIcon icon={DeskIcon} size={16} className="h-4 w-4 shrink-0" />
System Audio
</TabsTrigger>
)}
/>
</TabsContent>
</TabsList>
<TabsContent value="record" className="space-y-4">
<FormField
control={form.control}
name="sampleFile"
render={() => (
<AudioSampleRecording
file={selectedFile}
isRecording={isRecording}
duration={duration}
onStart={startRecording}
onStop={stopRecording}
onCancel={handleCancelRecording}
onTranscribe={handleTranscribe}
onPlayPause={handlePlayPause}
isPlaying={isPlaying}
isTranscribing={transcribe.isPending}
/>
)}
/>
</TabsContent>
<TabsContent value="upload" className="space-y-4">
<FormField
control={form.control}
name="sampleFile"
render={({ field: { onChange, name } }) => (
<AudioSampleUpload
file={selectedFile}
onFileChange={onChange}
onTranscribe={handleTranscribe}
onPlayPause={handlePlayPause}
isPlaying={isPlaying}
isValidating={isValidatingAudio}
isTranscribing={transcribe.isPending}
isDisabled={
audioDuration !== null &&
audioDuration > MAX_AUDIO_DURATION_SECONDS
}
fieldName={name}
/>
)}
/>
</TabsContent>
{isTauri() && isSystemAudioSupported && (
<TabsContent value="system" className="space-y-4">
<TabsContent value="record" className="space-y-4">
<FormField
control={form.control}
name="sampleFile"
render={() => (
<AudioSampleSystem
<AudioSampleRecording
file={selectedFile}
isRecording={isSystemRecording}
duration={systemDuration}
onStart={startSystemRecording}
onStop={stopSystemRecording}
isRecording={isRecording}
duration={duration}
onStart={startRecording}
onStop={stopRecording}
onCancel={handleCancelRecording}
onTranscribe={handleTranscribe}
onPlayPause={handlePlayPause}
@@ -590,55 +727,188 @@ export function ProfileForm() {
)}
/>
</TabsContent>
)}
</Tabs>
<FormField
control={form.control}
name="referenceText"
render={({ field }) => (
<FormItem>
<FormLabel>Reference Text</FormLabel>
<FormControl>
<Textarea
placeholder="Enter the exact text spoken in the audio..."
className="min-h-[100px]"
{...field}
{platform.metadata.isTauri && isSystemAudioSupported && (
<TabsContent value="system" className="space-y-4">
<FormField
control={form.control}
name="sampleFile"
render={() => (
<AudioSampleSystem
file={selectedFile}
isRecording={isSystemRecording}
duration={systemDuration}
onStart={startSystemRecording}
onStop={stopSystemRecording}
onCancel={handleCancelRecording}
onTranscribe={handleTranscribe}
onPlayPause={handlePlayPause}
isPlaying={isPlaying}
isTranscribing={transcribe.isPending}
/>
)}
/>
</FormControl>
<FormMessage />
</FormItem>
)}
/>
</>
) : (
// Show sample list when editing
editingProfileId && (
<div>
<SampleList profileId={editingProfileId} />
</div>
)
)}
</div>
</div>
</TabsContent>
)}
</Tabs>
<div className="flex gap-2 justify-end mt-6 pt-4 border-t">
<Button type="button" variant="outline" onClick={() => handleOpenChange(false)}>
Cancel
</Button>
<Button
type="submit"
disabled={createProfile.isPending || updateProfile.isPending || addSample.isPending}
>
{createProfile.isPending || updateProfile.isPending || addSample.isPending
? 'Saving...'
: editingProfileId
? 'Save Changes'
: 'Create Profile'}
</Button>
</div>
</form>
</Form>
<FormField
control={form.control}
name="referenceText"
render={({ field }) => (
<FormItem>
<FormLabel>Reference Text</FormLabel>
<FormControl>
<Textarea
placeholder="Enter the exact text spoken in the audio..."
className="min-h-[100px]"
{...field}
/>
</FormControl>
<FormMessage />
</FormItem>
)}
/>
</>
) : (
// Show sample list when editing
editingProfileId && (
<div>
<SampleList profileId={editingProfileId} />
</div>
)
)}
</div>
{/* Right column: Profile info */}
<div className="space-y-4">
{/* Avatar Upload */}
<FormField
control={form.control}
name="avatarFile"
render={() => (
<FormItem>
<FormControl>
<div className="flex justify-center pt-4 pb-2">
<div className="relative group">
<div className="h-24 w-24 rounded-full bg-muted flex items-center justify-center shrink-0 overflow-hidden border-2 border-border">
{avatarPreview ? (
<img
src={avatarPreview}
alt="Avatar preview"
className="h-full w-full object-cover"
/>
) : (
<HugeiconsIcon icon={Mic01Icon} size={40} className="h-10 w-10 text-muted-foreground" />
)}
</div>
<button
type="button"
onClick={() => avatarInputRef.current?.click()}
className="absolute inset-0 rounded-full bg-accent/60 opacity-0 group-hover:opacity-100 transition-opacity flex items-center justify-center cursor-pointer"
>
<HugeiconsIcon icon={Edit02Icon} size={24} className="h-6 w-6 text-accent-foreground" />
</button>
{(avatarPreview || editingProfile?.avatar_path) && (
<button
type="button"
onClick={handleRemoveAvatar}
disabled={deleteAvatar.isPending}
className="absolute bottom-0 right-0 h-6 w-6 rounded-full bg-background/60 backdrop-blur-sm text-muted-foreground flex items-center justify-center hover:bg-background/80 hover:text-foreground transition-colors shadow-sm border border-border/50"
>
<HugeiconsIcon icon={Cancel01Icon} size={14} className="h-3.5 w-3.5" />
</button>
)}
</div>
<input
ref={avatarInputRef}
type="file"
accept="image/png,image/jpeg,image/webp"
onChange={handleAvatarFileChange}
className="hidden"
/>
</div>
</FormControl>
<FormMessage />
</FormItem>
)}
/>
<FormField
control={form.control}
name="name"
render={({ field }) => (
<FormItem>
<FormLabel>Name</FormLabel>
<FormControl>
<Input placeholder="My Voice" {...field} />
</FormControl>
<FormMessage />
</FormItem>
)}
/>
<FormField
control={form.control}
name="description"
render={({ field }) => (
<FormItem>
<FormLabel>Description (Optional)</FormLabel>
<FormControl>
<Textarea placeholder="Describe this voice..." {...field} />
</FormControl>
<FormMessage />
</FormItem>
)}
/>
<FormField
control={form.control}
name="language"
render={({ field }) => (
<FormItem>
<FormLabel>Language</FormLabel>
<Select onValueChange={field.onChange} defaultValue={field.value}>
<FormControl>
<SelectTrigger>
<SelectValue />
</SelectTrigger>
</FormControl>
<SelectContent>
{LANGUAGE_OPTIONS.map((lang) => (
<SelectItem key={lang.value} value={lang.value}>
{lang.label}
</SelectItem>
))}
</SelectContent>
</Select>
<FormMessage />
</FormItem>
)}
/>
</div>
</div>
<div className="flex gap-2 justify-end mt-6 pt-4 border-t">
<Button type="button" variant="outline" onClick={() => handleOpenChange(false)}>
Cancel
</Button>
<Button
type="submit"
disabled={
createProfile.isPending || updateProfile.isPending || addSample.isPending
}
>
{createProfile.isPending || updateProfile.isPending || addSample.isPending
? 'Saving...'
: editingProfileId
? 'Save Changes'
: 'Create Profile'}
</Button>
</div>
</form>
</Form>
</div>
</DialogContent>
</Dialog>
);
@@ -1,4 +1,5 @@
import { Mic, Sparkles } from 'lucide-react';
import { HugeiconsIcon } from '@hugeicons/react';
import { Mic01Icon, SparklesIcon } from '@hugeicons/core-free-icons';
import { Button } from '@/components/ui/button';
import { Card, CardContent } from '@/components/ui/card';
import { useProfiles } from '@/lib/hooks/useProfiles';
@@ -30,12 +31,12 @@ export function ProfileList() {
{allProfiles.length === 0 ? (
<Card>
<CardContent className="flex flex-col items-center justify-center py-12">
<Mic className="h-12 w-12 text-muted-foreground mb-4" />
<HugeiconsIcon icon={Mic01Icon} size={48} className="h-12 w-12 text-muted-foreground mb-4" />
<p className="text-muted-foreground mb-4">
No voice profiles yet. Create your first profile to get started.
</p>
<Button onClick={() => setDialogOpen(true)}>
<Sparkles className="mr-2 h-4 w-4" />
<HugeiconsIcon icon={SparklesIcon} size={16} className="mr-2 h-4 w-4" />
Create Voice
</Button>
</CardContent>
+323 -55
View File
@@ -1,11 +1,142 @@
import { Plus, Trash2, Play } from 'lucide-react';
import { useState } from 'react';
import { HugeiconsIcon } from '@hugeicons/react';
import { CheckmarkCircle01Icon, Edit01Icon, PauseIcon, PlayIcon, Add01Icon, Delete01Icon, VolumeHighIcon, Cancel01Icon } from '@hugeicons/core-free-icons';
import { useEffect, useRef, useState } from 'react';
import { Button } from '@/components/ui/button';
import { useDeleteSample, useProfileSamples } from '@/lib/hooks/useProfiles';
import { usePlayerStore } from '@/stores/playerStore';
import { CircleButton } from '@/components/ui/circle-button';
import {
Dialog,
DialogContent,
DialogDescription,
DialogFooter,
DialogHeader,
DialogTitle,
} from '@/components/ui/dialog';
import { Slider } from '@/components/ui/slider';
import { Textarea } from '@/components/ui/textarea';
import { useToast } from '@/components/ui/use-toast';
import { apiClient } from '@/lib/api/client';
import { useDeleteSample, useProfileSamples, useUpdateSample } from '@/lib/hooks/useProfiles';
import { formatAudioDuration } from '@/lib/utils/audio';
import { cn } from '@/lib/utils/cn';
import { SampleUpload } from './SampleUpload';
interface MiniSamplePlayerProps {
audioUrl: string;
}
function MiniSamplePlayer({ audioUrl }: MiniSamplePlayerProps) {
const audioRef = useRef<HTMLAudioElement | null>(null);
const [isPlaying, setIsPlaying] = useState(false);
const [currentTime, setCurrentTime] = useState(0);
const [duration, setDuration] = useState(0);
const [isLoading, setIsLoading] = useState(true);
useEffect(() => {
const audio = new Audio(audioUrl);
audioRef.current = audio;
const handleLoadedMetadata = () => {
setDuration(audio.duration);
setIsLoading(false);
};
const handleTimeUpdate = () => {
setCurrentTime(audio.currentTime);
};
const handleEnded = () => {
setIsPlaying(false);
setCurrentTime(0);
};
const handlePlay = () => setIsPlaying(true);
const handlePause = () => setIsPlaying(false);
audio.addEventListener('loadedmetadata', handleLoadedMetadata);
audio.addEventListener('timeupdate', handleTimeUpdate);
audio.addEventListener('ended', handleEnded);
audio.addEventListener('play', handlePlay);
audio.addEventListener('pause', handlePause);
return () => {
audio.pause();
audio.removeEventListener('loadedmetadata', handleLoadedMetadata);
audio.removeEventListener('timeupdate', handleTimeUpdate);
audio.removeEventListener('ended', handleEnded);
audio.removeEventListener('play', handlePlay);
audio.removeEventListener('pause', handlePause);
audio.src = '';
};
}, [audioUrl]);
const handlePlayPause = () => {
if (!audioRef.current) return;
if (isPlaying) {
audioRef.current.pause();
} else {
audioRef.current.play();
}
};
const handleSeek = (value: number[]) => {
if (!audioRef.current || duration === 0) return;
const progress = value[0] / 100;
audioRef.current.currentTime = progress * duration;
};
const handleStop = () => {
if (audioRef.current) {
audioRef.current.pause();
audioRef.current.currentTime = 0;
}
setIsPlaying(false);
setCurrentTime(0);
};
return (
<div className="border-t bg-muted/30 px-3 py-2 mt-2">
<div className="flex items-center gap-2">
<Button
type="button"
variant="ghost"
size="icon"
className="h-7 w-7 shrink-0"
onClick={handlePlayPause}
disabled={isLoading}
>
{isPlaying ? <HugeiconsIcon icon={PauseIcon} size={14} className="h-3.5 w-3.5" /> : <HugeiconsIcon icon={PlayIcon} size={14} className="h-3.5 w-3.5 ml-0.5" />}
</Button>
<div className="flex-1 min-w-0 flex items-center gap-2">
<Slider
value={duration > 0 ? [(currentTime / duration) * 100] : [0]}
onValueChange={handleSeek}
max={100}
step={0.1}
className="flex-1"
/>
<div className="flex items-center gap-1 text-xs text-muted-foreground shrink-0 min-w-[70px]">
<span className="font-mono">{formatAudioDuration(currentTime)}</span>
<span>/</span>
<span className="font-mono">{formatAudioDuration(duration)}</span>
</div>
</div>
<Button
type="button"
variant="ghost"
size="icon"
className="h-7 w-7 shrink-0"
onClick={handleStop}
title="Stop"
>
<HugeiconsIcon icon={Cancel01Icon} size={14} className="h-3.5 w-3.5" />
</Button>
</div>
</div>
);
}
interface SampleListProps {
profileId: string;
}
@@ -13,20 +144,62 @@ interface SampleListProps {
export function SampleList({ profileId }: SampleListProps) {
const { data: samples, isLoading } = useProfileSamples(profileId);
const deleteSample = useDeleteSample();
const updateSample = useUpdateSample();
const { toast } = useToast();
const [uploadOpen, setUploadOpen] = useState(false);
const setAudio = usePlayerStore((state) => state.setAudio);
const currentAudioId = usePlayerStore((state) => state.audioId);
const isPlaying = usePlayerStore((state) => state.isPlaying);
const [editingSampleId, setEditingSampleId] = useState<string | null>(null);
const [editedText, setEditedText] = useState<string>('');
const [deleteDialogOpen, setDeleteDialogOpen] = useState(false);
const [sampleToDelete, setSampleToDelete] = useState<string | null>(null);
const handleDelete = (sampleId: string) => {
if (confirm('Are you sure you want to delete this sample?')) {
deleteSample.mutate(sampleId);
const handleDeleteClick = (sampleId: string) => {
setSampleToDelete(sampleId);
setDeleteDialogOpen(true);
};
const handleDeleteConfirm = () => {
if (sampleToDelete) {
deleteSample.mutate(sampleToDelete);
setDeleteDialogOpen(false);
setSampleToDelete(null);
}
};
const handlePlay = (referenceText: string, sampleId: string) => {
const audioUrl = apiClient.getSampleUrl(sampleId);
setAudio(audioUrl, sampleId, referenceText.substring(0, 50));
const handleStartEdit = (sampleId: string, currentText: string) => {
setEditingSampleId(sampleId);
setEditedText(currentText);
};
const handleCancelEdit = () => {
setEditingSampleId(null);
setEditedText('');
};
const handleSaveEdit = async (sampleId: string) => {
if (!editedText.trim()) {
toast({
title: 'Invalid text',
description: 'Reference text cannot be empty.',
variant: 'destructive',
});
return;
}
try {
await updateSample.mutateAsync({ sampleId, referenceText: editedText.trim() });
toast({
title: 'Sample updated',
description: 'Reference text has been updated successfully.',
});
setEditingSampleId(null);
setEditedText('');
} catch (error) {
toast({
title: 'Update failed',
description: error instanceof Error ? error.message : 'Failed to update sample',
variant: 'destructive',
});
}
};
if (isLoading) {
@@ -34,57 +207,152 @@ export function SampleList({ profileId }: SampleListProps) {
}
return (
<div className="space-y-4">
<div className="flex items-center justify-between">
<h3 className="text-lg font-semibold">Audio Samples</h3>
<Button type="button" size="sm" onClick={() => setUploadOpen(true)}>
<Plus className="mr-2 h-4 w-4" />
Add Sample
</Button>
</div>
<div className="space-y-4 pt-4">
{samples && samples.length === 0 ? (
<div className="text-sm text-muted-foreground py-4">
No samples yet. Add your first audio sample.
<div className="flex flex-col items-center justify-center py-8 text-center border border-dashed rounded-lg">
<HugeiconsIcon icon={VolumeHighIcon} size={32} className="h-8 w-8 text-muted-foreground/50 mb-2" />
<p className="text-sm text-muted-foreground">No samples yet</p>
<p className="text-xs text-muted-foreground/70 mt-1">
Add your first audio sample to get started
</p>
</div>
) : (
<div className="space-y-2">
{samples?.map((sample) => (
<div
key={sample.id}
className="flex items-center justify-between p-3 border rounded-lg"
>
<div className="flex-1">
<p className="text-sm font-medium">{sample.reference_text}</p>
<p className="text-xs text-muted-foreground mt-1">{sample.audio_path}</p>
{samples?.map((sample, index) => {
const isEditing = editingSampleId === sample.id;
return (
<div
key={sample.id}
className={cn(
'group relative rounded-lg border bg-card transition-all duration-200',
isEditing ? 'ring-2 ring-primary/20' : 'hover:border-primary/30',
)}
>
{isEditing ? (
/* Edit Mode */
<div className="p-4 space-y-3">
<div className="flex items-center gap-2 text-xs text-muted-foreground mb-2">
<HugeiconsIcon icon={Edit01Icon} size={12} className="h-3 w-3" />
<span>Editing transcription</span>
</div>
<Textarea
value={editedText}
onChange={(e) => setEditedText(e.target.value)}
className="min-h-[100px] text-sm resize-none"
placeholder="Enter reference text..."
autoFocus
/>
<div className="flex items-center justify-end gap-2 pt-1">
<Button
type="button"
size="sm"
variant="ghost"
onClick={handleCancelEdit}
disabled={updateSample.isPending}
>
<HugeiconsIcon icon={Cancel01Icon} size={16} className="h-4 w-4 mr-1" />
Cancel
</Button>
<Button
type="button"
size="sm"
onClick={() => handleSaveEdit(sample.id)}
disabled={updateSample.isPending}
>
<HugeiconsIcon icon={CheckmarkCircle01Icon} size={16} className="h-4 w-4 mr-1" />
{updateSample.isPending ? 'Saving...' : 'Save'}
</Button>
</div>
</div>
) : (
<>
{/* View Mode */}
<div className="flex items-center gap-3 p-3 h-[72px]">
{/* Text Content */}
<div className="flex-1 min-w-0 py-0.5">
<p className="text-sm font-medium line-clamp-2 leading-snug">
{sample.reference_text}
</p>
</div>
{/* Action Buttons */}
<div className="shrink-0 flex items-center gap-0.5 opacity-0 group-hover:opacity-100 transition-opacity">
<CircleButton
icon={(props) => <HugeiconsIcon icon={Edit01Icon} size={14} {...props} />}
title="Edit transcription"
onClick={() => handleStartEdit(sample.id, sample.reference_text)}
/>
<CircleButton
icon={(props) => <HugeiconsIcon icon={Delete01Icon} size={14} {...props} />}
title="Delete sample"
onClick={() => handleDeleteClick(sample.id)}
disabled={deleteSample.isPending}
/>
</div>
{/* Sample Number Badge */}
<div className="absolute top-1 right-2 text-[10px] text-muted-foreground/50 font-medium">
#{index + 1}
</div>
</div>
{/* Mini Player - Always visible */}
<MiniSamplePlayer audioUrl={apiClient.getSampleUrl(sample.id)} />
</>
)}
</div>
<div className="flex gap-2">
<Button
type="button"
variant="ghost"
size="sm"
onClick={() => handlePlay(sample.reference_text, sample.id)}
className={currentAudioId === sample.id && isPlaying ? 'text-primary' : ''}
>
<Play className="h-4 w-4 mr-1" />
Play
</Button>
<Button
type="button"
variant="ghost"
size="sm"
onClick={() => handleDelete(sample.id)}
disabled={deleteSample.isPending}
>
<Trash2 className="h-4 w-4 text-destructive" />
</Button>
</div>
</div>
))}
);
})}
</div>
)}
<Button
type="button"
variant="outline"
className="w-full"
onClick={() => setUploadOpen(true)}
>
<HugeiconsIcon icon={Add01Icon} size={16} className="mr-2 h-4 w-4" />
Add Sample
</Button>
<p className="text-xs text-muted-foreground text-center px-2">
Note: A single 30-second sample is the sweet spot. Quality may decrease with multiple
samples. In a future update samples might be interchangeable and tagged for varying styles
of the same voice.
</p>
<SampleUpload profileId={profileId} open={uploadOpen} onOpenChange={setUploadOpen} />
<Dialog open={deleteDialogOpen} onOpenChange={setDeleteDialogOpen}>
<DialogContent>
<DialogHeader>
<DialogTitle>Delete Sample</DialogTitle>
<DialogDescription>
Are you sure you want to delete this audio sample? This action cannot be undone.
</DialogDescription>
</DialogHeader>
<DialogFooter>
<Button
variant="outline"
onClick={() => {
setDeleteDialogOpen(false);
setSampleToDelete(null);
}}
>
Cancel
</Button>
<Button
variant="destructive"
onClick={handleDeleteConfirm}
disabled={deleteSample.isPending}
>
{deleteSample.isPending ? 'Deleting...' : 'Delete'}
</Button>
</DialogFooter>
</DialogContent>
</Dialog>
</div>
);
}
@@ -1,5 +1,6 @@
import { zodResolver } from '@hookform/resolvers/zod';
import { Mic, Monitor, Upload } from 'lucide-react';
import { HugeiconsIcon } from '@hugeicons/react';
import { Mic01Icon, DeskIcon, Upload01Icon } from '@hugeicons/core-free-icons';
import { useEffect, useState } from 'react';
import { useForm } from 'react-hook-form';
import * as z from 'zod';
@@ -27,7 +28,7 @@ import { useAudioRecording } from '@/lib/hooks/useAudioRecording';
import { useAddSample, useProfile } from '@/lib/hooks/useProfiles';
import { useSystemAudioCapture } from '@/lib/hooks/useSystemAudioCapture';
import { useTranscription } from '@/lib/hooks/useTranscription';
import { isTauri } from '@/lib/tauri';
import { usePlatform } from '@/platform/PlatformContext';
import { AudioSampleRecording } from './AudioSampleRecording';
import { AudioSampleSystem } from './AudioSampleSystem';
import { AudioSampleUpload } from './AudioSampleUpload';
@@ -49,6 +50,7 @@ interface SampleUploadProps {
}
export function SampleUpload({ profileId, open, onOpenChange }: SampleUploadProps) {
const platform = usePlatform();
const addSample = useAddSample();
const transcribe = useTranscription();
const { data: profile } = useProfile(profileId);
@@ -232,19 +234,19 @@ export function SampleUpload({ profileId, open, onOpenChange }: SampleUploadProp
<form onSubmit={form.handleSubmit(onSubmit)} className="space-y-4">
<Tabs value={mode} onValueChange={(v) => setMode(v as 'upload' | 'record' | 'system')}>
<TabsList
className={`grid w-full ${isTauri() && isSystemAudioSupported ? 'grid-cols-3' : 'grid-cols-2'}`}
className={`grid w-full ${platform.metadata.isTauri && isSystemAudioSupported ? 'grid-cols-3' : 'grid-cols-2'}`}
>
<TabsTrigger value="upload" className="flex items-center gap-2">
<Upload className="h-4 w-4 shrink-0" />
<HugeiconsIcon icon={Upload01Icon} size={16} className="h-4 w-4 shrink-0" />
Upload
</TabsTrigger>
<TabsTrigger value="record" className="flex items-center gap-2">
<Mic className="h-4 w-4 shrink-0" />
<HugeiconsIcon icon={Mic01Icon} size={16} className="h-4 w-4 shrink-0" />
Record
</TabsTrigger>
{isTauri() && isSystemAudioSupported && (
{platform.metadata.isTauri && isSystemAudioSupported && (
<TabsTrigger value="system" className="flex items-center gap-2">
<Monitor className="h-4 w-4 shrink-0" />
<HugeiconsIcon icon={DeskIcon} size={16} className="h-4 w-4 shrink-0" />
System Audio
</TabsTrigger>
)}
@@ -289,7 +291,7 @@ export function SampleUpload({ profileId, open, onOpenChange }: SampleUploadProp
/>
</TabsContent>
{isTauri() && isSystemAudioSupported && (
{platform.metadata.isTauri && isSystemAudioSupported && (
<TabsContent value="system" className="space-y-4">
<FormField
control={form.control}
+13 -8
View File
@@ -1,5 +1,6 @@
import { useQuery, useQueryClient } from '@tanstack/react-query';
import { Edit, MoreHorizontal, Plus, Trash2, Mic } from 'lucide-react';
import { HugeiconsIcon } from '@hugeicons/react';
import { Edit01Icon, MoreHorizontalIcon, Add01Icon, Delete01Icon } from '@hugeicons/core-free-icons';
import { useMemo, useRef } from 'react';
import { Button } from '@/components/ui/button';
import {
@@ -18,6 +19,7 @@ import {
TableRow,
} from '@/components/ui/table';
import { ProfileForm } from '@/components/VoiceProfiles/ProfileForm';
import { ProfileAvatar } from '@/components/VoiceProfiles/ProfileAvatar';
import { apiClient } from '@/lib/api/client';
import type { VoiceProfileResponse } from '@/lib/api/types';
import { BOTTOM_SAFE_AREA_PADDING } from '@/lib/constants/ui';
@@ -112,7 +114,7 @@ export function VoicesTab() {
<div className="flex items-center justify-between mb-6">
<h1 className="text-2xl font-bold">Voices</h1>
<Button onClick={() => setDialogOpen(true)}>
<Plus className="h-4 w-4 mr-2" />
<HugeiconsIcon icon={Add01Icon} size={16} className="h-4 w-4 mr-2" />
New Voice
</Button>
</div>
@@ -184,9 +186,12 @@ function VoiceRow({
<TableRow className="cursor-pointer" onClick={onEdit}>
<TableCell>
<div className="flex items-center gap-2">
<div className="h-8 w-8 rounded-lg bg-muted flex items-center justify-center shrink-0">
<Mic className="h-4 w-4 text-muted-foreground" />
</div>
<ProfileAvatar
profileId={profile.id}
avatarPath={profile.avatar_path}
size="md"
alt={`${profile.name} avatar`}
/>
<div>
<div className="font-medium">{profile.name}</div>
{profile.description && (
@@ -214,16 +219,16 @@ function VoiceRow({
<DropdownMenu>
<DropdownMenuTrigger asChild>
<Button variant="ghost" size="icon">
<MoreHorizontal className="h-4 w-4" />
<HugeiconsIcon icon={MoreHorizontalIcon} size={16} className="h-4 w-4" />
</Button>
</DropdownMenuTrigger>
<DropdownMenuContent>
<DropdownMenuItem onClick={onEdit}>
<Edit className="h-4 w-4 mr-2" />
<HugeiconsIcon icon={Edit01Icon} size={16} className="h-4 w-4 mr-2" />
Edit
</DropdownMenuItem>
<DropdownMenuItem onClick={onDelete} className="text-destructive">
<Trash2 className="h-4 w-4 mr-2" />
<HugeiconsIcon icon={Delete01Icon} size={16} className="h-4 w-4 mr-2" />
Delete
</DropdownMenuItem>
</DropdownMenuContent>
+3 -2
View File
@@ -1,5 +1,6 @@
import * as React from 'react';
import { Check } from 'lucide-react';
import { HugeiconsIcon } from '@hugeicons/react';
import { CheckmarkCircle01Icon } from '@hugeicons/core-free-icons';
import { cn } from '@/lib/utils/cn';
export interface CheckboxProps {
@@ -34,7 +35,7 @@ const Checkbox = React.forwardRef<HTMLButtonElement, CheckboxProps>(
)}
{...props}
>
{checked && <Check className="h-3 w-3 text-accent-foreground" />}
{checked && <HugeiconsIcon icon={CheckmarkCircle01Icon} size={12} className="h-3 w-3 text-accent-foreground" />}
</button>
);
},
+2 -1
View File
@@ -6,10 +6,11 @@ export interface CircleButtonProps extends React.ButtonHTMLAttributes<HTMLButton
}
const CircleButton = React.forwardRef<HTMLButtonElement, CircleButtonProps>(
({ className, icon: Icon, ...props }, ref) => {
({ className, icon: Icon, type = 'button', ...props }, ref) => {
return (
<button
ref={ref}
type={type}
className={cn(
'h-7 w-7 rounded-full flex items-center justify-center flex-shrink-0',
'hover:bg-muted transition-colors',
+3 -2
View File
@@ -1,5 +1,6 @@
import * as DialogPrimitive from '@radix-ui/react-dialog';
import { X } from 'lucide-react';
import { HugeiconsIcon } from '@hugeicons/react';
import { Cancel01Icon } from '@hugeicons/core-free-icons';
import * as React from 'react';
import { cn } from '@/lib/utils/cn';
@@ -42,7 +43,7 @@ const DialogContent = React.forwardRef<
>
{children}
<DialogPrimitive.Close className="absolute right-4 top-4 rounded-sm opacity-70 ring-offset-background transition-opacity hover:opacity-100 focus:outline-none focus:ring-2 focus:ring-ring focus:ring-offset-2 disabled:pointer-events-none data-[state=open]:bg-accent data-[state=open]:text-muted-foreground">
<X className="h-4 w-4" />
<HugeiconsIcon icon={Cancel01Icon} size={16} className="h-4 w-4" />
<span className="sr-only">Close</span>
</DialogPrimitive.Close>
</DialogPrimitive.Content>
+10 -7
View File
@@ -1,6 +1,7 @@
import * as React from 'react';
import { MoreHorizontalIcon } from '@hugeicons/core-free-icons';
import { HugeiconsIcon } from '@hugeicons/react';
import * as DropdownMenuPrimitive from '@radix-ui/react-dropdown-menu';
import { MoreHorizontal } from 'lucide-react';
import * as React from 'react';
import { cn } from '@/lib/utils/cn';
const DropdownMenu = DropdownMenuPrimitive.Root;
@@ -26,7 +27,7 @@ const DropdownMenuSubTrigger = React.forwardRef<
{...props}
>
{children}
<MoreHorizontal className="ml-auto h-4 w-4" />
<HugeiconsIcon icon={MoreHorizontalIcon} size={16} className="ml-auto h-4 w-4" />
</DropdownMenuPrimitive.SubTrigger>
));
DropdownMenuSubTrigger.displayName = DropdownMenuPrimitive.SubTrigger.displayName;
@@ -73,7 +74,7 @@ const DropdownMenuItem = React.forwardRef<
<DropdownMenuPrimitive.Item
ref={ref}
className={cn(
'relative flex cursor-default select-none items-center rounded-sm px-2 py-1.5 text-sm outline-none transition-colors focus:bg-accent focus:text-accent-foreground data-[disabled]:pointer-events-none data-[disabled]:opacity-50',
'relative flex cursor-default select-none items-center rounded-sm px-2 py-1.5 text-sm outline-none focus:bg-accent focus:text-accent-foreground data-[disabled]:pointer-events-none data-[disabled]:opacity-50',
inset && 'pl-8',
className,
)}
@@ -97,7 +98,7 @@ const DropdownMenuCheckboxItem = React.forwardRef<
>
<span className="absolute left-2 flex h-3.5 w-3.5 items-center justify-center">
<DropdownMenuPrimitive.ItemIndicator>
<MoreHorizontal className="h-4 w-4" />
<HugeiconsIcon icon={MoreHorizontalIcon} size={16} className="h-4 w-4" />
</DropdownMenuPrimitive.ItemIndicator>
</span>
{children}
@@ -119,7 +120,7 @@ const DropdownMenuRadioItem = React.forwardRef<
>
<span className="absolute left-2 flex h-3.5 w-3.5 items-center justify-center">
<DropdownMenuPrimitive.ItemIndicator>
<MoreHorizontal className="h-2 w-2 fill-current" />
<HugeiconsIcon icon={MoreHorizontalIcon} size={8} className="h-2 w-2 fill-current" />
</DropdownMenuPrimitive.ItemIndicator>
</span>
{children}
@@ -154,7 +155,9 @@ const DropdownMenuSeparator = React.forwardRef<
DropdownMenuSeparator.displayName = DropdownMenuPrimitive.Separator.displayName;
const DropdownMenuShortcut = ({ className, ...props }: React.HTMLAttributes<HTMLSpanElement>) => {
return <span className={cn('ml-auto text-xs tracking-widest opacity-60', className)} {...props} />;
return (
<span className={cn('ml-auto text-xs tracking-widest opacity-60', className)} {...props} />
);
};
DropdownMenuShortcut.displayName = 'DropdownMenuShortcut';
+4 -3
View File
@@ -1,5 +1,6 @@
import * as React from 'react';
import { ChevronDown, Check } from 'lucide-react';
import { HugeiconsIcon } from '@hugeicons/react';
import { ArrowDown01Icon, CheckmarkCircle01Icon } from '@hugeicons/core-free-icons';
import { cn } from '@/lib/utils/cn';
import {
DropdownMenu,
@@ -36,7 +37,7 @@ const MultiSelectCheckboxItem = React.forwardRef<
>
<span className="absolute left-2 flex h-3.5 w-3.5 items-center justify-center">
<DropdownMenuPrimitive.ItemIndicator>
<Check className="h-4 w-4" />
<HugeiconsIcon icon={CheckmarkCircle01Icon} size={16} className="h-4 w-4" />
</DropdownMenuPrimitive.ItemIndicator>
</span>
{children}
@@ -78,7 +79,7 @@ export function MultiSelect({
)}
>
<span className="line-clamp-1">{displayText}</span>
<ChevronDown className="h-4 w-4 opacity-50" />
<HugeiconsIcon icon={ArrowDown01Icon} size={16} className="h-4 w-4 opacity-50" />
</button>
</DropdownMenuTrigger>
<DropdownMenuContent
+43
View File
@@ -0,0 +1,43 @@
'use client';
import { CircleIcon } from '@hugeicons/core-free-icons';
import { HugeiconsIcon } from '@hugeicons/react';
import * as RadioGroupPrimitive from '@radix-ui/react-radio-group';
import * as React from 'react';
import { cn } from '@/lib/utils/cn';
const RadioGroup = React.forwardRef<
React.ElementRef<typeof RadioGroupPrimitive.Root>,
React.ComponentPropsWithoutRef<typeof RadioGroupPrimitive.Root>
>(({ className, ...props }, ref) => {
return <RadioGroupPrimitive.Root className={cn('grid gap-2', className)} {...props} ref={ref} />;
});
RadioGroup.displayName = RadioGroupPrimitive.Root.displayName;
const RadioGroupItem = React.forwardRef<
React.ElementRef<typeof RadioGroupPrimitive.Item>,
React.ComponentPropsWithoutRef<typeof RadioGroupPrimitive.Item>
>(({ className, ...props }, ref) => {
return (
<RadioGroupPrimitive.Item
ref={ref}
className={cn(
'aspect-square h-4 w-4 rounded-full border border-accent text-accent ring-offset-background focus:outline-none focus-visible:ring-2 focus-visible:ring-ring focus-visible:ring-offset-2 disabled:cursor-not-allowed disabled:opacity-50',
className,
)}
{...props}
>
<RadioGroupPrimitive.Indicator className="flex items-center justify-center">
<HugeiconsIcon
icon={CircleIcon}
size={10}
className="h-2.5 w-2.5 fill-current text-current"
/>
</RadioGroupPrimitive.Indicator>
</RadioGroupPrimitive.Item>
);
});
RadioGroupItem.displayName = RadioGroupPrimitive.Item.displayName;
export { RadioGroup, RadioGroupItem };
+6 -5
View File
@@ -1,5 +1,6 @@
import * as SelectPrimitive from '@radix-ui/react-select';
import { Check, ChevronDown, ChevronUp } from 'lucide-react';
import { HugeiconsIcon } from '@hugeicons/react';
import { CheckmarkCircle01Icon, ArrowDown01Icon, ArrowUp01Icon } from '@hugeicons/core-free-icons';
import * as React from 'react';
import { cn } from '@/lib/utils/cn';
@@ -23,7 +24,7 @@ const SelectTrigger = React.forwardRef<
>
{children}
<SelectPrimitive.Icon asChild>
<ChevronDown className="h-4 w-4 opacity-50" />
<HugeiconsIcon icon={ArrowDown01Icon} size={16} className="h-4 w-4 opacity-50" />
</SelectPrimitive.Icon>
</SelectPrimitive.Trigger>
));
@@ -38,7 +39,7 @@ const SelectScrollUpButton = React.forwardRef<
className={cn('flex cursor-default items-center justify-center py-1', className)}
{...props}
>
<ChevronUp className="h-4 w-4" />
<HugeiconsIcon icon={ArrowUp01Icon} size={16} className="h-4 w-4" />
</SelectPrimitive.ScrollUpButton>
));
SelectScrollUpButton.displayName = SelectPrimitive.ScrollUpButton.displayName;
@@ -52,7 +53,7 @@ const SelectScrollDownButton = React.forwardRef<
className={cn('flex cursor-default items-center justify-center py-1', className)}
{...props}
>
<ChevronDown className="h-4 w-4" />
<HugeiconsIcon icon={ArrowDown01Icon} size={16} className="h-4 w-4" />
</SelectPrimitive.ScrollDownButton>
));
SelectScrollDownButton.displayName = SelectPrimitive.ScrollDownButton.displayName;
@@ -115,7 +116,7 @@ const SelectItem = React.forwardRef<
>
<span className="absolute left-2 flex h-3.5 w-3.5 items-center justify-center">
<SelectPrimitive.ItemIndicator>
<Check className="h-4 w-4" />
<HugeiconsIcon icon={CheckmarkCircle01Icon} size={16} className="h-4 w-4" />
</SelectPrimitive.ItemIndicator>
</span>
+3 -2
View File
@@ -1,6 +1,7 @@
import * as ToastPrimitives from '@radix-ui/react-toast';
import { cva, type VariantProps } from 'class-variance-authority';
import { X } from 'lucide-react';
import { HugeiconsIcon } from '@hugeicons/react';
import { Cancel01Icon } from '@hugeicons/core-free-icons';
import * as React from 'react';
import { cn } from '@/lib/utils/cn';
@@ -79,7 +80,7 @@ const ToastClose = React.forwardRef<
toast-close=""
{...props}
>
<X className="h-4 w-4" />
<HugeiconsIcon icon={Cancel01Icon} size={16} className="h-4 w-4" />
</ToastPrimitives.Close>
));
ToastClose.displayName = ToastPrimitives.Close.displayName;
+210
View File
@@ -0,0 +1,210 @@
import { Download01Icon, Refresh01Icon } from '@hugeicons/core-free-icons';
import { HugeiconsIcon } from '@hugeicons/react';
import { useCallback, useEffect, useRef, useState } from 'react';
import { Progress } from '@/components/ui/progress';
import { ToastAction } from '@/components/ui/toast';
import { useToast } from '@/components/ui/use-toast';
import { usePlatform } from '@/platform/PlatformContext';
import type { UpdateStatus } from '@/platform/types';
// Re-export UpdateStatus for backwards compatibility
export type { UpdateStatus };
interface UseAutoUpdaterOptions {
checkOnMount?: boolean;
showToast?: boolean;
}
export function useAutoUpdater(options: boolean | UseAutoUpdaterOptions = false) {
// Support both old boolean API and new options object
const { checkOnMount, showToast } =
typeof options === 'boolean'
? { checkOnMount: options, showToast: false }
: { checkOnMount: options.checkOnMount ?? false, showToast: options.showToast ?? false };
const platform = usePlatform();
const { toast } = useToast();
const [status, setStatus] = useState<UpdateStatus>(platform.updater.getStatus());
const hasCheckedRef = useRef(false);
const toastIdRef = useRef<string | null>(null);
const toastUpdateRef = useRef<
| ((props: {
title?: React.ReactNode;
description?: React.ReactNode;
duration?: number;
variant?: 'default' | 'destructive';
open?: boolean;
action?: React.ReactElement<typeof ToastAction>;
}) => void)
| null
>(null);
// Subscribe to updater status changes
useEffect(() => {
const unsubscribe = platform.updater.subscribe((newStatus) => {
setStatus(newStatus);
});
return unsubscribe;
// Empty dependency array - platform is stable from context
// eslint-disable-next-line react-hooks/exhaustive-deps
}, [platform.updater.subscribe]);
const checkForUpdates = useCallback(async () => {
await platform.updater.checkForUpdates();
// eslint-disable-next-line react-hooks/exhaustive-deps
}, [platform.updater.checkForUpdates]);
const downloadAndInstall = useCallback(async () => {
await platform.updater.downloadAndInstall();
// eslint-disable-next-line react-hooks/exhaustive-deps
}, [platform.updater.downloadAndInstall]);
const restartAndInstall = useCallback(async () => {
await platform.updater.restartAndInstall();
// eslint-disable-next-line react-hooks/exhaustive-deps
}, [platform.updater.restartAndInstall]);
// Check for updates on mount
useEffect(() => {
if (checkOnMount && platform.metadata.isTauri && !hasCheckedRef.current) {
hasCheckedRef.current = true;
checkForUpdates().catch((error) => {
console.error('Auto update check failed:', error);
});
}
// Empty dependency array - only run once on mount
// eslint-disable-next-line react-hooks/exhaustive-deps
}, [platform.metadata.isTauri, checkOnMount, checkForUpdates]);
// Show toast when update is available
useEffect(() => {
if (
!showToast ||
!status.available ||
status.downloading ||
status.readyToInstall ||
toastIdRef.current
) {
return;
}
const handleUpdateNow = async () => {
await downloadAndInstall();
};
const toastResult = toast({
title: 'Update Available',
description: `Version ${status.version} is ready to download.`,
duration: Infinity,
action: (
<ToastAction altText="Update now" onClick={handleUpdateNow}>
Update Now
</ToastAction>
),
});
toastIdRef.current = toastResult.id;
// Type assertion needed because update function has broader type than our ref
toastUpdateRef.current = toastResult.update as typeof toastUpdateRef.current;
}, [
showToast,
status.available,
status.downloading,
status.readyToInstall,
status.version,
downloadAndInstall,
toast,
]);
// Update toast when downloading
useEffect(() => {
if (!showToast || !status.downloading || !toastIdRef.current || !toastUpdateRef.current) {
return;
}
const progressPercent = status.downloadProgress || 0;
const progressText =
status.downloadedBytes !== undefined &&
status.totalBytes !== undefined &&
status.totalBytes > 0
? `${(status.downloadedBytes / 1024 / 1024).toFixed(1)} MB / ${(status.totalBytes / 1024 / 1024).toFixed(1)} MB`
: '';
toastUpdateRef.current({
title: (
<div className="flex items-center gap-2">
<HugeiconsIcon icon={Download01Icon} size={16} className="h-4 w-4 animate-pulse" />
<span>Downloading Update</span>
</div>
),
description: (
<div className="space-y-2">
<div className="text-sm">Version {status.version}</div>
{progressPercent > 0 && (
<>
<Progress value={progressPercent} className="h-2" />
{progressText && <div className="text-xs text-muted-foreground">{progressText}</div>}
</>
)}
</div>
),
duration: Infinity,
});
}, [
showToast,
status.downloading,
status.downloadProgress,
status.downloadedBytes,
status.totalBytes,
status.version,
]);
// Update toast when ready to install
useEffect(() => {
if (!showToast || !status.readyToInstall || !toastIdRef.current || !toastUpdateRef.current) {
return;
}
const handleRestartNow = async () => {
await restartAndInstall();
};
toastUpdateRef.current({
title: 'Update Ready',
description: `Version ${status.version} has been downloaded and is ready to install.`,
duration: Infinity,
action: (
<ToastAction altText="Restart now" onClick={handleRestartNow}>
<HugeiconsIcon icon={Refresh01Icon} size={12} className="h-3 w-3 mr-1" />
Restart Now
</ToastAction>
),
});
}, [showToast, status.readyToInstall, status.version, restartAndInstall]);
// Handle errors in toast
useEffect(() => {
if (!showToast || !status.error || !toastIdRef.current || !toastUpdateRef.current) {
return;
}
toastUpdateRef.current({
title: 'Update Failed',
description: status.error,
variant: 'destructive',
duration: 5000,
});
setTimeout(() => {
toastIdRef.current = null;
toastUpdateRef.current = null;
}, 5000);
}, [showToast, status.error]);
return {
status,
checkForUpdates,
downloadAndInstall,
restartAndInstall,
};
}
+186 -31
View File
@@ -1,26 +1,30 @@
import type { LanguageCode } from '@/lib/constants/languages';
import { useServerStore } from '@/stores/serverStore';
import type {
VoiceProfileCreate,
VoiceProfileResponse,
ProfileSampleResponse,
ActiveTasksResponse,
FolderPathsResponse,
GenerationRequest,
GenerationResponse,
HistoryQuery,
HistoryListResponse,
HistoryResponse,
TranscriptionResponse,
HealthResponse,
ModelStatusListResponse,
HistoryListResponse,
HistoryQuery,
HistoryResponse,
ModelDownloadRequest,
ActiveTasksResponse,
ModelStatusListResponse,
ProfileSampleResponse,
StoryCreate,
StoryResponse,
StoryDetailResponse,
StoryItemBatchUpdate,
StoryItemCreate,
StoryItemDetail,
StoryItemBatchUpdate,
StoryItemReorder,
StoryItemMove,
StoryItemReorder,
StoryItemSplit,
StoryItemTrim,
StoryResponse,
TranscriptionResponse,
VoiceProfileCreate,
VoiceProfileResponse,
} from './types';
class ApiClient {
@@ -54,6 +58,11 @@ class ApiClient {
return this.request<HealthResponse>('/health');
}
// System
async getSystemFolders(): Promise<FolderPathsResponse> {
return this.request<FolderPathsResponse>('/system/folders');
}
// Profiles
async createProfile(data: VoiceProfileCreate): Promise<VoiceProfileResponse> {
return this.request<VoiceProfileResponse>('/profiles', {
@@ -118,6 +127,16 @@ class ApiClient {
});
}
async updateProfileSample(
sampleId: string,
referenceText: string,
): Promise<ProfileSampleResponse> {
return this.request<ProfileSampleResponse>(`/profiles/samples/${sampleId}`, {
method: 'PUT',
body: JSON.stringify({ reference_text: referenceText }),
});
}
async exportProfile(profileId: string): Promise<Blob> {
const url = `${this.getBaseUrl()}/profiles/${profileId}/export`;
const response = await fetch(url);
@@ -152,6 +171,32 @@ class ApiClient {
return response.json();
}
async uploadAvatar(profileId: string, file: File): Promise<VoiceProfileResponse> {
const url = `${this.getBaseUrl()}/profiles/${profileId}/avatar`;
const formData = new FormData();
formData.append('file', file);
const response = await fetch(url, {
method: 'POST',
body: formData,
});
if (!response.ok) {
const error = await response.json().catch(() => ({
detail: response.statusText,
}));
throw new Error(error.detail || `HTTP error! status: ${response.status}`);
}
return response.json();
}
async deleteAvatar(profileId: string): Promise<void> {
await this.request<void>(`/profiles/${profileId}/avatar`, {
method: 'DELETE',
});
}
// Generation
async generateSpeech(data: GenerationRequest): Promise<GenerationResponse> {
return this.request<GenerationResponse>('/generate', {
@@ -160,6 +205,77 @@ class ApiClient {
});
}
// Providers
async listProviders(): Promise<{
providers: Array<{
type: string;
name: string;
installed: boolean;
size_mb: number | null;
}>;
installed: string[];
}> {
return this.request('/providers');
}
async getActiveProvider(): Promise<{
provider: string;
health: {
status: string;
provider: string;
version: string | null;
model: string | null;
device: string | null;
};
status: {
model_loaded: boolean;
model_size: string | null;
available_sizes: string[];
gpu_available: boolean | null;
vram_used_mb: number | null;
};
}> {
return this.request('/providers/active');
}
async startProvider(providerType: string): Promise<{
message: string;
provider: {
status: string;
provider: string;
version: string | null;
model: string | null;
device: string | null;
};
}> {
return this.request('/providers/start', {
method: 'POST',
body: JSON.stringify({ provider_type: providerType }),
});
}
async stopProvider(): Promise<{ message: string }> {
return this.request('/providers/stop', {
method: 'POST',
});
}
async downloadProvider(providerType: string): Promise<{
message: string;
provider_type: string;
}> {
return this.request('/providers/download', {
method: 'POST',
body: JSON.stringify({ provider_type: providerType }),
});
}
async deleteProvider(providerType: string): Promise<{ message: string }> {
return this.request(`/providers/${providerType}`, {
method: 'DELETE',
});
}
// History
async listHistory(query?: HistoryQuery): Promise<HistoryListResponse> {
const params = new URLSearchParams();
@@ -212,7 +328,15 @@ class ApiClient {
return response.blob();
}
async importGeneration(file: File): Promise<{ id: string; profile_id: string; profile_name: string; text: string; message: string }> {
async importGeneration(
file: File,
): Promise<{
id: string;
profile_id: string;
profile_name: string;
text: string;
message: string;
}> {
const url = `${this.getBaseUrl()}/history/import`;
const formData = new FormData();
formData.append('file', file);
@@ -242,7 +366,7 @@ class ApiClient {
}
// Transcription
async transcribeAudio(file: File, language?: 'en' | 'zh'): Promise<TranscriptionResponse> {
async transcribeAudio(file: File, language?: LanguageCode): Promise<TranscriptionResponse> {
const formData = new FormData();
formData.append('file', file);
if (language) {
@@ -271,10 +395,18 @@ class ApiClient {
}
async triggerModelDownload(modelName: string): Promise<{ message: string }> {
return this.request<{ message: string }>('/models/download', {
console.log(
'[API] triggerModelDownload called for:',
modelName,
'at',
new Date().toISOString(),
);
const result = await this.request<{ message: string }>('/models/download', {
method: 'POST',
body: JSON.stringify({ model_name: modelName } as ModelDownloadRequest),
});
console.log('[API] triggerModelDownload response:', result);
return result;
}
async deleteModel(modelName: string): Promise<{ message: string }> {
@@ -301,10 +433,7 @@ class ApiClient {
return this.request('/channels');
}
async createChannel(data: {
name: string;
device_ids: string[];
}): Promise<{
async createChannel(data: { name: string; device_ids: string[] }): Promise<{
id: string;
name: string;
is_default: boolean;
@@ -346,10 +475,7 @@ class ApiClient {
return this.request(`/channels/${channelId}/voices`);
}
async setChannelVoices(
channelId: string,
profileIds: string[],
): Promise<{ message: string }> {
async setChannelVoices(channelId: string, profileIds: string[]): Promise<{ message: string }> {
return this.request(`/channels/${channelId}/voices`, {
method: 'PUT',
body: JSON.stringify({ profile_ids: profileIds }),
@@ -360,10 +486,7 @@ class ApiClient {
return this.request(`/profiles/${profileId}/channels`);
}
async setProfileChannels(
profileId: string,
channelIds: string[],
): Promise<{ message: string }> {
async setProfileChannels(profileId: string, channelIds: string[]): Promise<{ message: string }> {
return this.request(`/profiles/${profileId}/channels`, {
method: 'PUT',
body: JSON.stringify({ channel_ids: channelIds }),
@@ -406,8 +529,8 @@ class ApiClient {
});
}
async removeStoryItem(storyId: string, generationId: string): Promise<void> {
await this.request<void>(`/stories/${storyId}/items/${generationId}`, {
async removeStoryItem(storyId: string, itemId: string): Promise<void> {
await this.request<void>(`/stories/${storyId}/items/${itemId}`, {
method: 'DELETE',
});
}
@@ -426,13 +549,45 @@ class ApiClient {
});
}
async moveStoryItem(storyId: string, generationId: string, data: StoryItemMove): Promise<StoryItemDetail> {
return this.request<StoryItemDetail>(`/stories/${storyId}/items/${generationId}/move`, {
async moveStoryItem(
storyId: string,
itemId: string,
data: StoryItemMove,
): Promise<StoryItemDetail> {
return this.request<StoryItemDetail>(`/stories/${storyId}/items/${itemId}/move`, {
method: 'PUT',
body: JSON.stringify(data),
});
}
async trimStoryItem(
storyId: string,
itemId: string,
data: StoryItemTrim,
): Promise<StoryItemDetail> {
return this.request<StoryItemDetail>(`/stories/${storyId}/items/${itemId}/trim`, {
method: 'PUT',
body: JSON.stringify(data),
});
}
async splitStoryItem(
storyId: string,
itemId: string,
data: StoryItemSplit,
): Promise<StoryItemDetail[]> {
return this.request<StoryItemDetail[]>(`/stories/${storyId}/items/${itemId}/split`, {
method: 'POST',
body: JSON.stringify(data),
});
}
async duplicateStoryItem(storyId: string, itemId: string): Promise<StoryItemDetail> {
return this.request<StoryItemDetail>(`/stories/${storyId}/items/${itemId}/duplicate`, {
method: 'POST',
});
}
async exportStoryAudio(storyId: string): Promise<Blob> {
const url = `${this.getBaseUrl()}/stories/${storyId}/export-audio`;
const response = await fetch(url);
+1
View File
@@ -9,6 +9,7 @@ export type ModelStatus = {
model_name: string;
display_name: string;
downloaded: boolean;
downloading?: boolean; // True if download is in progress
size_mb?: number | null;
loaded?: boolean;
};
+24 -3
View File
@@ -1,9 +1,10 @@
// API Types matching backend Pydantic models
import type { LanguageCode } from '@/lib/constants/languages';
export interface VoiceProfileCreate {
name: string;
description?: string;
language: 'en' | 'zh';
language: LanguageCode;
}
export interface VoiceProfileResponse {
@@ -11,6 +12,7 @@ export interface VoiceProfileResponse {
name: string;
description?: string;
language: string;
avatar_path?: string;
created_at: string;
updated_at: string;
}
@@ -29,9 +31,10 @@ export interface ProfileSampleResponse {
export interface GenerationRequest {
profile_id: string;
text: string;
language: 'en' | 'zh';
language: LanguageCode;
seed?: number;
model_size?: '1.7B' | '0.6B';
instruct?: string;
}
export interface GenerationResponse {
@@ -62,7 +65,7 @@ export interface HistoryListResponse {
}
export interface TranscriptionRequest {
language?: 'en' | 'zh';
language?: LanguageCode;
}
export interface TranscriptionResponse {
@@ -94,6 +97,7 @@ export interface ModelStatus {
model_name: string;
display_name: string;
downloaded: boolean;
downloading: boolean; // True if download is in progress
size_mb?: number;
loaded: boolean;
}
@@ -124,6 +128,12 @@ export interface ActiveTasksResponse {
generations: ActiveGenerationTask[];
}
export interface FolderPathsResponse {
data_dir: string;
models_dir: string;
providers_dir: string;
}
export interface StoryCreate {
name: string;
description?: string;
@@ -144,6 +154,8 @@ export interface StoryItemDetail {
generation_id: string;
start_time_ms: number;
track: number;
trim_start_ms: number;
trim_end_ms: number;
created_at: string;
profile_id: string;
profile_name: string;
@@ -188,3 +200,12 @@ export interface StoryItemMove {
start_time_ms: number;
track: number;
}
export interface StoryItemTrim {
trim_start_ms: number;
trim_end_ms: number;
}
export interface StoryItemSplit {
split_time_ms: number;
}
+4 -4
View File
@@ -1,5 +1,5 @@
import { useCallback, useEffect, useRef, useState } from 'react';
import { isTauri } from '@/lib/tauri';
import { usePlatform } from '@/platform/PlatformContext';
import { convertToWav } from '@/lib/utils/audio';
interface UseAudioRecordingOptions {
@@ -11,6 +11,7 @@ export function useAudioRecording({
maxDurationSeconds = 29,
onRecordingComplete,
}: UseAudioRecordingOptions = {}) {
const platform = usePlatform();
const [isRecording, setIsRecording] = useState(false);
const [duration, setDuration] = useState(0);
const [error, setError] = useState<string | null>(null);
@@ -40,15 +41,14 @@ export function useAudioRecording({
await new Promise((resolve) => setTimeout(resolve, 100));
if (!navigator.mediaDevices || !navigator.mediaDevices.getUserMedia) {
const isTauriEnv = isTauri();
console.error('MediaDevices check:', {
hasNavigator: typeof navigator !== 'undefined',
hasMediaDevices: !!navigator?.mediaDevices,
hasGetUserMedia: !!navigator?.mediaDevices?.getUserMedia,
isTauri: isTauriEnv,
isTauri: platform.metadata.isTauri,
});
const errorMsg = isTauriEnv
const errorMsg = platform.metadata.isTauri
? 'Microphone access is not available. Please ensure:\n1. The app has microphone permissions in System Settings (macOS: System Settings > Privacy & Security > Microphone)\n2. You restart the app after granting permissions\n3. You are using Tauri v2 with a webview that supports getUserMedia'
: 'Microphone access is not available. Please ensure you are using a secure context (HTTPS or localhost) and that your browser has microphone permissions enabled.';
setError(errorMsg);
+2 -7
View File
@@ -28,7 +28,7 @@ interface UseGenerationFormOptions {
export function useGenerationForm(options: UseGenerationFormOptions = {}) {
const { toast } = useToast();
const generation = useGeneration();
const setAudio = usePlayerStore((state) => state.setAudio);
const setAudioWithAutoPlay = usePlayerStore((state) => state.setAudioWithAutoPlay);
const setIsGenerating = useGenerationStore((state) => state.setIsGenerating);
const [downloadingModelName, setDownloadingModelName] = useState<string | null>(null);
const [downloadingDisplayName, setDownloadingDisplayName] = useState<string | null>(null);
@@ -91,13 +91,8 @@ export function useGenerationForm(options: UseGenerationFormOptions = {}) {
instruct: data.instruct || undefined,
});
toast({
title: 'Generation complete!',
description: `Audio generated (${result.duration.toFixed(2)}s)`,
});
const audioUrl = apiClient.getAudioUrl(result.id);
setAudio(audioUrl, result.id, selectedProfileId, data.text.substring(0, 50));
setAudioWithAutoPlay(audioUrl, result.id, selectedProfileId, data.text.substring(0, 50));
form.reset();
options.onSuccess?.(result.id);
+31 -95
View File
@@ -1,7 +1,7 @@
import { useMutation, useQuery, useQueryClient } from '@tanstack/react-query';
import { apiClient } from '@/lib/api/client';
import type { HistoryQuery } from '@/lib/api/types';
import { isTauri } from '@/lib/tauri';
import { usePlatform } from '@/platform/PlatformContext';
export function useHistory(query?: HistoryQuery) {
return useQuery({
@@ -30,116 +30,52 @@ export function useDeleteGeneration() {
}
export function useExportGeneration() {
const platform = usePlatform();
return useMutation({
mutationFn: async ({ generationId, text }: { generationId: string; text: string }) => {
const blob = await apiClient.exportGeneration(generationId);
// Create safe filename from text
const safeText = text.substring(0, 30).replace(/[^a-z0-9]/gi, '-').toLowerCase();
const safeText = text
.substring(0, 30)
.replace(/[^a-z0-9]/gi, '-')
.toLowerCase();
const filename = `generation-${safeText}.voicebox.zip`;
if (isTauri()) {
// Use Tauri's native save dialog
try {
const { save } = await import('@tauri-apps/plugin-dialog');
const filePath = await save({
defaultPath: filename,
filters: [
{
name: 'Voicebox Generation',
extensions: ['voicebox.zip', 'zip'],
},
],
});
if (filePath) {
// Write file using Tauri's filesystem API
const { writeBinaryFile } = await import('@tauri-apps/plugin-fs');
const arrayBuffer = await blob.arrayBuffer();
await writeBinaryFile(filePath, new Uint8Array(arrayBuffer));
}
} catch (error) {
console.error('Failed to use Tauri dialog, falling back to browser download:', error);
// Fall back to browser download if Tauri dialog fails
const url = window.URL.createObjectURL(blob);
const a = document.createElement('a');
a.href = url;
a.download = filename;
document.body.appendChild(a);
a.click();
window.URL.revokeObjectURL(url);
document.body.removeChild(a);
}
} else {
// Browser: trigger download
const url = window.URL.createObjectURL(blob);
const a = document.createElement('a');
a.href = url;
a.download = filename;
document.body.appendChild(a);
a.click();
window.URL.revokeObjectURL(url);
document.body.removeChild(a);
}
await platform.filesystem.saveFile(filename, blob, [
{
name: 'Voicebox Generation',
extensions: ['zip'],
},
]);
return blob;
},
});
}
export function useExportGenerationAudio() {
const platform = usePlatform();
return useMutation({
mutationFn: async ({ generationId, text }: { generationId: string; text: string }) => {
const blob = await apiClient.exportGenerationAudio(generationId);
// Create safe filename from text
const safeText = text.substring(0, 30).replace(/[^a-z0-9]/gi, '-').toLowerCase();
const safeText = text
.substring(0, 30)
.replace(/[^a-z0-9]/gi, '-')
.toLowerCase();
const filename = `${safeText}.wav`;
if (isTauri()) {
// Use Tauri's native save dialog
try {
const { save } = await import('@tauri-apps/plugin-dialog');
const filePath = await save({
defaultPath: filename,
filters: [
{
name: 'Audio File',
extensions: ['wav'],
},
],
});
if (filePath) {
// Write file using Tauri's filesystem API
const { writeBinaryFile } = await import('@tauri-apps/plugin-fs');
const arrayBuffer = await blob.arrayBuffer();
await writeBinaryFile(filePath, new Uint8Array(arrayBuffer));
}
} catch (error) {
console.error('Failed to use Tauri dialog, falling back to browser download:', error);
// Fall back to browser download if Tauri dialog fails
const url = window.URL.createObjectURL(blob);
const a = document.createElement('a');
a.href = url;
a.download = filename;
document.body.appendChild(a);
a.click();
window.URL.revokeObjectURL(url);
document.body.removeChild(a);
}
} else {
// Browser: trigger download
const url = window.URL.createObjectURL(blob);
const a = document.createElement('a');
a.href = url;
a.download = filename;
document.body.appendChild(a);
a.click();
window.URL.revokeObjectURL(url);
document.body.removeChild(a);
}
await platform.filesystem.saveFile(filename, blob, [
{
name: 'Audio File',
extensions: ['wav'],
},
]);
return blob;
},
});
+101 -39
View File
@@ -1,14 +1,18 @@
import { useEffect, useRef } from 'react';
import { useToast } from '@/components/ui/use-toast';
import { useServerStore } from '@/stores/serverStore';
import { CancelCircleIcon, CheckmarkCircle02Icon } from '@hugeicons/core-free-icons';
import { HugeiconsIcon } from '@hugeicons/react';
import { Icon } from '@iconify/react';
import { useCallback, useEffect, useRef } from 'react';
import { Progress } from '@/components/ui/progress';
import { Loader2, CheckCircle2, XCircle } from 'lucide-react';
import { useToast } from '@/components/ui/use-toast';
import type { ModelProgress } from '@/lib/api/types';
import { useServerStore } from '@/stores/serverStore';
interface UseModelDownloadToastOptions {
modelName: string;
displayName: string;
enabled?: boolean;
onComplete?: () => void;
onError?: () => void;
}
/**
@@ -19,47 +23,64 @@ export function useModelDownloadToast({
modelName,
displayName,
enabled = false,
onComplete,
onError,
}: UseModelDownloadToastOptions) {
const { toast } = useToast();
const serverUrl = useServerStore((state) => state.serverUrl);
const toastIdRef = useRef<string | null>(null);
const toastUpdateRef = useRef<
((props: {
title?: React.ReactNode;
description?: React.ReactNode;
duration?: number;
variant?: 'default' | 'destructive';
open?: boolean;
}) => void) | null
>(null);
// biome-ignore lint: Using any for toast update ref to handle complex toast types
const toastUpdateRef = useRef<any>(null);
const eventSourceRef = useRef<EventSource | null>(null);
const formatBytes = (bytes: number): string => {
const formatBytes = useCallback((bytes: number): string => {
if (bytes === 0) return '0 B';
const k = 1024;
const sizes = ['B', 'KB', 'MB', 'GB'];
const i = Math.floor(Math.log(bytes) / Math.log(k));
return `${(bytes / Math.pow(k, i)).toFixed(1)} ${sizes[i]}`;
};
return `${(bytes / k ** i).toFixed(1)} ${sizes[i]}`;
}, []);
useEffect(() => {
console.log('[useModelDownloadToast] useEffect triggered', {
enabled,
serverUrl,
modelName,
displayName,
});
if (!enabled || !serverUrl || !modelName) {
console.log('[useModelDownloadToast] Not enabled, skipping');
return;
}
console.log('[useModelDownloadToast] Creating toast and EventSource for:', modelName);
// Create initial toast
const toastResult = toast({
title: displayName,
description: 'Starting download...',
description: (
<div className="flex items-center gap-2">
<Icon icon="svg-spinners:ring-resize" className="h-4 w-4 animate-spin" />
<span>Connecting to download...</span>
</div>
),
duration: Infinity, // Don't auto-dismiss, we'll handle it manually
});
toastIdRef.current = toastResult.id;
toastUpdateRef.current = toastResult.update;
// Subscribe to progress updates via Server-Sent Events
const eventSource = new EventSource(`${serverUrl}/models/progress/${modelName}`);
const eventSourceUrl = `${serverUrl}/models/progress/${modelName}`;
console.log('[useModelDownloadToast] Creating EventSource to:', eventSourceUrl);
const eventSource = new EventSource(eventSourceUrl);
eventSource.onopen = () => {
console.log('[useModelDownloadToast] EventSource connection opened for:', modelName);
};
eventSource.onmessage = (event) => {
console.log('[useModelDownloadToast] Received SSE message:', event.data);
try {
const progress = JSON.parse(event.data) as ModelProgress;
@@ -77,19 +98,35 @@ export function useModelDownloadToast({
switch (progress.status) {
case 'complete':
statusIcon = <CheckCircle2 className="h-4 w-4 text-green-500" />;
statusIcon = (
<HugeiconsIcon
icon={CheckmarkCircle02Icon}
size={16}
className="h-4 w-4 text-green-500"
/>
);
statusText = 'Download complete';
break;
case 'error':
statusIcon = <XCircle className="h-4 w-4 text-destructive" />;
statusIcon = (
<HugeiconsIcon
icon={CancelCircleIcon}
size={16}
className="h-4 w-4 text-destructive"
/>
);
statusText = `Error: ${progress.error || 'Unknown error'}`;
break;
case 'downloading':
statusIcon = <Loader2 className="h-4 w-4 animate-spin" />;
statusText = progress.filename ? `Downloading ${progress.filename}...` : 'Downloading...';
statusIcon = (
<Icon icon="svg-spinners:ring-resize" className="h-4 w-4 animate-spin" />
);
statusText = progress.filename || 'Downloading...';
break;
case 'extracting':
statusIcon = <Loader2 className="h-4 w-4 animate-spin" />;
statusIcon = (
<Icon icon="svg-spinners:ring-resize" className="h-4 w-4 animate-spin" />
);
statusText = 'Extracting...';
break;
}
@@ -117,21 +154,44 @@ export function useModelDownloadToast({
});
// Close connection and dismiss toast on completion or error
if (progress.status === 'complete' || progress.status === 'error') {
// Also treat progress >= 100% as complete
const isComplete = progress.status === 'complete' || progress.progress >= 100;
const isError = progress.status === 'error';
if (isComplete || isError) {
console.log('[useModelDownloadToast] Download finished:', {
isComplete,
isError,
progress: progress.progress,
});
eventSource.close();
eventSourceRef.current = null;
// Auto-dismiss on completion after delay
if (progress.status === 'complete') {
setTimeout(() => {
if (toastIdRef.current && toastUpdateRef.current) {
toastUpdateRef.current({
open: false,
});
toastIdRef.current = null;
toastUpdateRef.current = null;
}
}, 5000);
// Update toast to show completion state before callbacks
if (isComplete && toastUpdateRef.current) {
toastUpdateRef.current({
title: (
<div className="flex items-center gap-2">
<HugeiconsIcon
icon={CheckmarkCircle02Icon}
size={16}
className="h-4 w-4 text-green-500"
/>
<span>{displayName}</span>
</div>
),
description: 'Download complete',
duration: 3000,
});
}
// Call callbacks
if (isComplete && onComplete) {
console.log('[useModelDownloadToast] Download complete, calling onComplete callback');
onComplete();
} else if (isError && onError) {
console.log('[useModelDownloadToast] Download error, calling onError callback');
onError();
}
}
}
@@ -140,8 +200,9 @@ export function useModelDownloadToast({
}
};
eventSource.onerror = () => {
console.error('SSE error');
eventSource.onerror = (error) => {
console.error('[useModelDownloadToast] SSE error for:', modelName, error);
console.log('[useModelDownloadToast] EventSource readyState:', eventSource.readyState);
eventSource.close();
eventSourceRef.current = null;
@@ -162,15 +223,16 @@ export function useModelDownloadToast({
// Cleanup on unmount or when disabled
return () => {
console.log('[useModelDownloadToast] Cleanup - closing EventSource for:', modelName);
if (eventSourceRef.current) {
eventSourceRef.current.close();
eventSourceRef.current = null;
}
// Note: We don't dismiss the toast here as it might still be showing completion state
};
}, [enabled, serverUrl, modelName, displayName, toast]);
}, [enabled, serverUrl, modelName, displayName, toast, formatBytes, onComplete, onError]);
return {
isTracking: enabled && eventSourceRef.current !== null,
};
}
}
+59 -47
View File
@@ -1,7 +1,7 @@
import { useMutation, useQuery, useQueryClient } from '@tanstack/react-query';
import { apiClient } from '@/lib/api/client';
import type { VoiceProfileCreate } from '@/lib/api/types';
import { isTauri } from '@/lib/tauri';
import { usePlatform } from '@/platform/PlatformContext';
export function useProfiles() {
return useQuery({
@@ -98,60 +98,43 @@ export function useDeleteSample() {
});
}
export function useUpdateSample() {
const queryClient = useQueryClient();
return useMutation({
mutationFn: ({ sampleId, referenceText }: { sampleId: string; referenceText: string }) =>
apiClient.updateProfileSample(sampleId, referenceText),
onSuccess: (data) => {
queryClient.invalidateQueries({
queryKey: ['profiles', data.profile_id, 'samples'],
});
queryClient.invalidateQueries({
queryKey: ['profiles', data.profile_id],
});
queryClient.invalidateQueries({ queryKey: ['profiles'] });
},
});
}
export function useExportProfile() {
const platform = usePlatform();
return useMutation({
mutationFn: async (profileId: string) => {
const blob = await apiClient.exportProfile(profileId);
// Get profile name for filename
const profile = await apiClient.getProfile(profileId);
const safeName = profile.name.replace(/[^a-z0-9]/gi, '-').toLowerCase();
const filename = `profile-${safeName}.voicebox.zip`;
if (isTauri()) {
// Use Tauri's native save dialog
try {
const { save } = await import('@tauri-apps/plugin-dialog');
const filePath = await save({
defaultPath: filename,
filters: [
{
name: 'Voicebox Profile',
extensions: ['voicebox.zip', 'zip'],
},
],
});
if (filePath) {
// Write file using Tauri's filesystem API
const { writeBinaryFile } = await import('@tauri-apps/plugin-fs');
const arrayBuffer = await blob.arrayBuffer();
await writeBinaryFile(filePath, new Uint8Array(arrayBuffer));
}
} catch (error) {
console.error('Failed to use Tauri dialog, falling back to browser download:', error);
// Fall back to browser download if Tauri dialog fails
const url = window.URL.createObjectURL(blob);
const a = document.createElement('a');
a.href = url;
a.download = filename;
document.body.appendChild(a);
a.click();
window.URL.revokeObjectURL(url);
document.body.removeChild(a);
}
} else {
// Browser: trigger download
const url = window.URL.createObjectURL(blob);
const a = document.createElement('a');
a.href = url;
a.download = filename;
document.body.appendChild(a);
a.click();
window.URL.revokeObjectURL(url);
document.body.removeChild(a);
}
await platform.filesystem.saveFile(filename, blob, [
{
name: 'Voicebox Profile',
extensions: ['zip'],
},
]);
return blob;
},
});
@@ -167,3 +150,32 @@ export function useImportProfile() {
},
});
}
export function useUploadAvatar() {
const queryClient = useQueryClient();
return useMutation({
mutationFn: ({ profileId, file }: { profileId: string; file: File }) =>
apiClient.uploadAvatar(profileId, file),
onSuccess: (_, variables) => {
queryClient.invalidateQueries({ queryKey: ['profiles'] });
queryClient.invalidateQueries({
queryKey: ['profiles', variables.profileId],
});
},
});
}
export function useDeleteAvatar() {
const queryClient = useQueryClient();
return useMutation({
mutationFn: (profileId: string) => apiClient.deleteAvatar(profileId),
onSuccess: (_, profileId) => {
queryClient.invalidateQueries({ queryKey: ['profiles'] });
queryClient.invalidateQueries({
queryKey: ['profiles', profileId],
});
},
});
}
+53 -49
View File
@@ -1,7 +1,7 @@
import { useMutation, useQuery, useQueryClient } from '@tanstack/react-query';
import { apiClient } from '@/lib/api/client';
import type { StoryCreate, StoryItemCreate, StoryItemBatchUpdate, StoryItemReorder, StoryItemMove } from '@/lib/api/types';
import { isTauri } from '@/lib/tauri';
import type { StoryCreate, StoryItemCreate, StoryItemBatchUpdate, StoryItemReorder, StoryItemMove, StoryItemTrim, StoryItemSplit } from '@/lib/api/types';
import { usePlatform } from '@/platform/PlatformContext';
export function useStories() {
return useQuery({
@@ -70,8 +70,8 @@ export function useRemoveStoryItem() {
const queryClient = useQueryClient();
return useMutation({
mutationFn: ({ storyId, generationId }: { storyId: string; generationId: string }) =>
apiClient.removeStoryItem(storyId, generationId),
mutationFn: ({ storyId, itemId }: { storyId: string; itemId: string }) =>
apiClient.removeStoryItem(storyId, itemId),
onSuccess: (_, variables) => {
queryClient.invalidateQueries({ queryKey: ['stories'] });
queryClient.invalidateQueries({ queryKey: ['stories', variables.storyId] });
@@ -109,8 +109,47 @@ export function useMoveStoryItem() {
const queryClient = useQueryClient();
return useMutation({
mutationFn: ({ storyId, generationId, data }: { storyId: string; generationId: string; data: StoryItemMove }) =>
apiClient.moveStoryItem(storyId, generationId, data),
mutationFn: ({ storyId, itemId, data }: { storyId: string; itemId: string; data: StoryItemMove }) =>
apiClient.moveStoryItem(storyId, itemId, data),
onSuccess: (_, variables) => {
queryClient.invalidateQueries({ queryKey: ['stories'] });
queryClient.invalidateQueries({ queryKey: ['stories', variables.storyId] });
},
});
}
export function useTrimStoryItem() {
const queryClient = useQueryClient();
return useMutation({
mutationFn: ({ storyId, itemId, data }: { storyId: string; itemId: string; data: StoryItemTrim }) =>
apiClient.trimStoryItem(storyId, itemId, data),
onSuccess: (_, variables) => {
queryClient.invalidateQueries({ queryKey: ['stories'] });
queryClient.invalidateQueries({ queryKey: ['stories', variables.storyId] });
},
});
}
export function useSplitStoryItem() {
const queryClient = useQueryClient();
return useMutation({
mutationFn: ({ storyId, itemId, data }: { storyId: string; itemId: string; data: StoryItemSplit }) =>
apiClient.splitStoryItem(storyId, itemId, data),
onSuccess: (_, variables) => {
queryClient.invalidateQueries({ queryKey: ['stories'] });
queryClient.invalidateQueries({ queryKey: ['stories', variables.storyId] });
},
});
}
export function useDuplicateStoryItem() {
const queryClient = useQueryClient();
return useMutation({
mutationFn: ({ storyId, itemId }: { storyId: string; itemId: string }) =>
apiClient.duplicateStoryItem(storyId, itemId),
onSuccess: (_, variables) => {
queryClient.invalidateQueries({ queryKey: ['stories'] });
queryClient.invalidateQueries({ queryKey: ['stories', variables.storyId] });
@@ -119,6 +158,8 @@ export function useMoveStoryItem() {
}
export function useExportStoryAudio() {
const platform = usePlatform();
return useMutation({
mutationFn: async ({ storyId, storyName }: { storyId: string; storyName: string }) => {
const blob = await apiClient.exportStoryAudio(storyId);
@@ -127,49 +168,12 @@ export function useExportStoryAudio() {
const safeName = storyName.substring(0, 50).replace(/[^a-z0-9]/gi, '-').toLowerCase();
const filename = `${safeName || 'story'}.wav`;
if (isTauri()) {
// Use Tauri's native save dialog
try {
const { save } = await import('@tauri-apps/plugin-dialog');
const filePath = await save({
defaultPath: filename,
filters: [
{
name: 'Audio File',
extensions: ['wav'],
},
],
});
if (filePath) {
// Write file using Tauri's filesystem API
const { writeBinaryFile } = await import('@tauri-apps/plugin-fs');
const arrayBuffer = await blob.arrayBuffer();
await writeBinaryFile(filePath, new Uint8Array(arrayBuffer));
}
} catch (error) {
console.error('Failed to use Tauri dialog, falling back to browser download:', error);
// Fall back to browser download if Tauri dialog fails
const url = window.URL.createObjectURL(blob);
const a = document.createElement('a');
a.href = url;
a.download = filename;
document.body.appendChild(a);
a.click();
window.URL.revokeObjectURL(url);
document.body.removeChild(a);
}
} else {
// Browser: trigger download
const url = window.URL.createObjectURL(blob);
const a = document.createElement('a');
a.href = url;
a.download = filename;
document.body.appendChild(a);
a.click();
window.URL.revokeObjectURL(url);
document.body.removeChild(a);
}
await platform.filesystem.saveFile(filename, blob, [
{
name: 'Audio File',
extensions: ['wav'],
},
]);
return blob;
},
+36 -22
View File
@@ -5,6 +5,7 @@ import { useStoryStore } from '@/stores/storyStore';
interface ActiveSource {
source: AudioBufferSourceNode;
itemId: string;
generationId: string;
startTimeMs: number;
endTimeMs: number;
@@ -26,9 +27,9 @@ export function useStoryPlayback(items: StoryItemDetail[] | undefined) {
const audioContextRef = useRef<AudioContext | null>(null);
// Master gain for volume control
const masterGainRef = useRef<GainNode | null>(null);
// Preloaded AudioBuffers by generation_id
// Preloaded AudioBuffers by generation_id (audio file is shared between split clips)
const audioBuffersRef = useRef<Map<string, AudioBuffer>>(new Map());
// Currently playing AudioBufferSourceNodes by generation_id
// Currently playing AudioBufferSourceNodes by item.id (unique per clip)
const activeSourcesRef = useRef<Map<string, ActiveSource>>(new Map());
// Animation frame for syncing visual playhead
const animationFrameRef = useRef<number | null>(null);
@@ -56,16 +57,16 @@ export function useStoryPlayback(items: StoryItemDetail[] | undefined) {
return audioContextRef.current;
}, []);
// Stop a source
const stopSource = useCallback((generationId: string) => {
const activeSource = activeSourcesRef.current.get(generationId);
// Stop a source by item id
const stopSource = useCallback((itemId: string) => {
const activeSource = activeSourcesRef.current.get(itemId);
if (activeSource) {
try {
activeSource.source.stop();
} catch {
// Source may have already stopped
}
activeSourcesRef.current.delete(generationId);
activeSourcesRef.current.delete(itemId);
}
}, []);
@@ -123,8 +124,8 @@ export function useStoryPlayback(items: StoryItemDetail[] | undefined) {
useEffect(() => {
return () => {
// Stop all sources
for (const [generationId] of activeSourcesRef.current) {
stopSource(generationId);
for (const [itemId] of activeSourcesRef.current) {
stopSource(itemId);
}
activeSourcesRef.current.clear();
@@ -151,7 +152,11 @@ export function useStoryPlayback(items: StoryItemDetail[] | undefined) {
(storyTimeMs: number, itemList: StoryItemDetail[]): StoryItemDetail[] => {
return itemList.filter((item) => {
const itemStart = item.start_time_ms;
const itemEnd = item.start_time_ms + item.duration * 1000;
// Use effective duration (accounting for trims)
const trimStartMs = item.trim_start_ms || 0;
const trimEndMs = item.trim_end_ms || 0;
const effectiveDurationMs = item.duration * 1000 - trimStartMs - trimEndMs;
const itemEnd = item.start_time_ms + effectiveDurationMs;
return storyTimeMs >= itemStart && storyTimeMs < itemEnd;
});
},
@@ -185,8 +190,8 @@ export function useStoryPlayback(items: StoryItemDetail[] | undefined) {
// Stop all sources
const stopAllSources = useCallback(() => {
console.log('[StoryPlayback] Stopping all sources');
for (const [generationId] of activeSourcesRef.current) {
stopSource(generationId);
for (const [itemId] of activeSourcesRef.current) {
stopSource(itemId);
}
activeSourcesRef.current.clear();
}, [stopSource]);
@@ -199,18 +204,18 @@ export function useStoryPlayback(items: StoryItemDetail[] | undefined) {
// Find all items that should be playing
const shouldBePlaying = findActiveItems(storyTimeMs, itemList);
const shouldBePlayingIds = new Set(shouldBePlaying.map((item) => item.generation_id));
const shouldBePlayingIds = new Set(shouldBePlaying.map((item) => item.id));
// Stop sources that shouldn't be playing anymore
for (const [generationId] of activeSourcesRef.current) {
if (!shouldBePlayingIds.has(generationId)) {
stopSource(generationId);
for (const [itemId] of activeSourcesRef.current) {
if (!shouldBePlayingIds.has(itemId)) {
stopSource(itemId);
}
}
// Schedule new sources for items that should be playing
for (const item of shouldBePlaying) {
if (!activeSourcesRef.current.has(item.generation_id)) {
if (!activeSourcesRef.current.has(item.id)) {
const buffer = audioBuffersRef.current.get(item.generation_id);
if (!buffer) {
console.warn('[StoryPlayback] Buffer not loaded for:', item.generation_id);
@@ -219,16 +224,24 @@ export function useStoryPlayback(items: StoryItemDetail[] | undefined) {
// Calculate when this item should start in AudioContext time
const itemStartContextTime = storyTimeToContextTime(item.start_time_ms);
const itemEndStoryTime = item.start_time_ms + item.duration * 1000;
// Calculate effective duration and trim offsets
const trimStartSec = (item.trim_start_ms || 0) / 1000;
const trimEndSec = (item.trim_end_ms || 0) / 1000;
const effectiveDuration = item.duration - trimStartSec - trimEndSec;
const itemEndStoryTime = item.start_time_ms + effectiveDuration * 1000;
// Calculate offset into the buffer (if seeking mid-way)
const offsetIntoBuffer = Math.max(0, (storyTimeMs - item.start_time_ms) / 1000);
const duration = item.duration - offsetIntoBuffer;
// Offset is relative to the trimmed start of the clip
const offsetIntoEffectiveClip = Math.max(0, (storyTimeMs - item.start_time_ms) / 1000);
const offsetIntoBuffer = trimStartSec + offsetIntoEffectiveClip;
const duration = effectiveDuration - offsetIntoEffectiveClip;
// If the item should have already started, schedule it to start immediately
const startAtContextTime = Math.max(currentContextTime, itemStartContextTime);
console.log('[StoryPlayback] Scheduling source:', {
itemId: item.id,
generationId: item.generation_id,
storyTimeMs,
itemStart: item.start_time_ms,
@@ -243,20 +256,21 @@ export function useStoryPlayback(items: StoryItemDetail[] | undefined) {
const activeSource: ActiveSource = {
source,
itemId: item.id,
generationId: item.generation_id,
startTimeMs: item.start_time_ms,
endTimeMs: itemEndStoryTime,
};
activeSourcesRef.current.set(item.generation_id, activeSource);
activeSourcesRef.current.set(item.id, activeSource);
// Schedule playback
source.start(startAtContextTime, offsetIntoBuffer, duration);
// Clean up when source ends
source.onended = () => {
console.log('[StoryPlayback] Source ended:', item.generation_id);
activeSourcesRef.current.delete(item.generation_id);
console.log('[StoryPlayback] Source ended:', item.id);
activeSourcesRef.current.delete(item.id);
};
}
}
+15 -35
View File
@@ -1,6 +1,5 @@
import { useState, useRef, useCallback, useEffect } from 'react';
import { invoke } from '@tauri-apps/api/core';
import { isTauri } from '@/lib/tauri';
import { usePlatform } from '@/platform/PlatformContext';
interface UseSystemAudioCaptureOptions {
maxDurationSeconds?: number;
@@ -15,6 +14,7 @@ export function useSystemAudioCapture({
maxDurationSeconds = 29,
onRecordingComplete,
}: UseSystemAudioCaptureOptions = {}) {
const platform = usePlatform();
const [isRecording, setIsRecording] = useState(false);
const [duration, setDuration] = useState(0);
const [error, setError] = useState<string | null>(null);
@@ -26,22 +26,12 @@ export function useSystemAudioCapture({
// Check if system audio capture is supported
useEffect(() => {
if (!isTauri()) {
setIsSupported(false);
return;
}
invoke<boolean>('is_system_audio_supported')
.then((supported) => {
setIsSupported(supported);
})
.catch(() => {
setIsSupported(false);
});
}, []);
const supported = platform.audio.isSystemAudioSupported();
setIsSupported(supported);
}, [platform]);
const startRecording = useCallback(async () => {
if (!isTauri()) {
if (!platform.metadata.isTauri) {
const errorMsg = 'System audio capture is only available in the desktop app.';
setError(errorMsg);
return;
@@ -58,9 +48,7 @@ export function useSystemAudioCapture({
setDuration(0);
// Start native capture
await invoke('start_system_audio_capture', {
maxDurationSecs: maxDurationSeconds,
});
await platform.audio.startSystemAudioCapture(maxDurationSeconds);
setIsRecording(true);
isRecordingRef.current = true;
@@ -86,10 +74,10 @@ export function useSystemAudioCapture({
setError(errorMessage);
setIsRecording(false);
}
}, [maxDurationSeconds, isSupported]);
}, [maxDurationSeconds, isSupported, platform]);
const stopRecording = useCallback(async () => {
if (!isRecording || !isTauri()) {
if (!isRecording || !platform.metadata.isTauri) {
return;
}
@@ -102,17 +90,9 @@ export function useSystemAudioCapture({
timerRef.current = null;
}
// Stop capture and get base64 WAV data
const base64Data = await invoke<string>('stop_system_audio_capture');
// Stop capture and get Blob
const blob = await platform.audio.stopSystemAudioCapture();
// Convert base64 to Blob
const binaryString = atob(base64Data);
const bytes = new Uint8Array(binaryString.length);
for (let i = 0; i < binaryString.length; i++) {
bytes[i] = binaryString.charCodeAt(i);
}
const blob = new Blob([bytes], { type: 'audio/wav' });
// Pass the actual recorded duration
const recordedDuration = startTimeRef.current
? (Date.now() - startTimeRef.current) / 1000
@@ -125,7 +105,7 @@ export function useSystemAudioCapture({
: 'Failed to stop system audio capture.';
setError(errorMessage);
}
}, [isRecording, onRecordingComplete]);
}, [isRecording, onRecordingComplete, platform]);
// Store stopRecording in ref for use in timer
useEffect(() => {
@@ -155,15 +135,15 @@ export function useSystemAudioCapture({
timerRef.current = null;
}
// Cancel recording on unmount if still recording
if (isRecordingRef.current && isTauri()) {
if (isRecordingRef.current && platform.metadata.isTauri) {
// Call stop directly without the callback to avoid stale closure
invoke('stop_system_audio_capture').catch((err) => {
platform.audio.stopSystemAudioCapture().catch((err) => {
console.error('Error stopping audio capture on unmount:', err);
});
}
};
// biome-ignore lint/correctness/useExhaustiveDependencies: Only run on unmount
}, []);
}, [platform]);
return {
isRecording,
+14
View File
@@ -0,0 +1,14 @@
import { useQuery } from '@tanstack/react-query';
import { apiClient } from '@/lib/api/client';
import { useServerStore } from '@/stores/serverStore';
export function useSystemFolders() {
const serverUrl = useServerStore((state) => state.serverUrl);
return useQuery({
queryKey: ['system', 'folders', serverUrl],
queryFn: () => apiClient.getSystemFolders(),
staleTime: 60000, // Cache for 1 minute - folder paths don't change often
retry: 1,
});
}
+2 -1
View File
@@ -1,9 +1,10 @@
import { useMutation } from '@tanstack/react-query';
import { apiClient } from '@/lib/api/client';
import type { LanguageCode } from '@/lib/constants/languages';
export function useTranscription() {
return useMutation({
mutationFn: ({ file, language }: { file: File; language?: 'en' | 'zh' }) =>
mutationFn: ({ file, language }: { file: File; language?: LanguageCode }) =>
apiClient.transcribeAudio(file, language),
});
}
-108
View File
@@ -1,108 +0,0 @@
/**
* Tauri integration utilities
*/
import { invoke } from '@tauri-apps/api/core';
import { listen, emit } from '@tauri-apps/api/event';
/**
* Check if running in Tauri environment
*/
export function isTauri(): boolean {
return '__TAURI_INTERNALS__' in window;
}
/**
* Check if running on macOS
*/
export function isMacOS(): boolean {
return navigator.platform.toLowerCase().includes('mac');
}
/**
* Start the bundled Python server (Tauri only)
*/
export async function startServer(remote = false): Promise<string> {
if (!isTauri()) {
throw new Error('Not running in Tauri environment');
}
try {
const result = await invoke<string>('start_server', { remote });
console.log('Server started:', result);
return result;
} catch (error) {
console.error('Failed to start server:', error);
throw error;
}
}
/**
* Stop the bundled Python server (Tauri only)
*/
export async function stopServer(): Promise<void> {
if (!isTauri()) {
throw new Error('Not running in Tauri environment');
}
try {
await invoke('stop_server');
console.log('Server stopped');
} catch (error) {
console.error('Failed to stop server:', error);
throw error;
}
}
/**
* Set whether the server should keep running when the app closes (Tauri only)
*/
export async function setKeepServerRunning(keepRunning: boolean): Promise<void> {
if (!isTauri()) {
return;
}
try {
await invoke('set_keep_server_running', { keepRunning });
} catch (error) {
console.error('Failed to set keep server running setting:', error);
}
}
/**
* Setup window close handler to check setting and stop server if needed
*/
export async function setupWindowCloseHandler(): Promise<void> {
if (!isTauri()) {
return;
}
try {
// Listen for window close request from Rust
await listen<null>('window-close-requested', async () => {
// Import store here to avoid circular dependency
const { useServerStore } = await import('@/stores/serverStore');
const keepRunning = useServerStore.getState().keepServerRunningOnClose;
// Check if server was started by this app instance
// In dev mode, serverStartedByApp will be false, so we won't try to stop a separately-run server
// We need to access the module-level variable - this is a bit hacky but works
// @ts-expect-error - accessing module-level variable from another module
const serverStartedByApp = window.__voiceboxServerStartedByApp ?? false;
if (!keepRunning && serverStartedByApp) {
// Stop server before closing (only if we started it)
try {
await stopServer();
} catch (error) {
console.error('Failed to stop server on close:', error);
}
}
// Emit event back to Rust to allow close
await emit('window-close-allowed');
});
} catch (error) {
console.error('Failed to setup window close handler:', error);
}
}
+25
View File
@@ -0,0 +1,25 @@
import { createContext, useContext, type ReactNode } from 'react';
import type { Platform } from './types';
const PlatformContext = createContext<Platform | null>(null);
export interface PlatformProviderProps {
platform: Platform;
children: ReactNode;
}
export function PlatformProvider({ platform, children }: PlatformProviderProps) {
return (
<PlatformContext.Provider value={platform}>
{children}
</PlatformContext.Provider>
);
}
export function usePlatform(): Platform {
const platform = useContext(PlatformContext);
if (!platform) {
throw new Error('usePlatform must be used within PlatformProvider');
}
return platform;
}
+77
View File
@@ -0,0 +1,77 @@
/**
* Platform abstraction types
* These interfaces define the contract that platform implementations must fulfill
*/
export interface FileFilter {
name: string;
extensions: string[];
}
export interface PlatformFilesystem {
saveFile(filename: string, blob: Blob, filters?: FileFilter[]): Promise<void>;
/**
* Open a folder in the native file explorer.
* On web, this is a no-op since browsers cannot open folders.
* @param path - The absolute path to the folder to open
* @returns true if the folder was opened, false if not supported
*/
openFolder(path: string): Promise<boolean>;
}
export interface UpdateStatus {
checking: boolean;
available: boolean;
version?: string;
downloading: boolean;
installing: boolean;
readyToInstall: boolean;
error?: string;
downloadProgress?: number; // 0-100 percentage
downloadedBytes?: number;
totalBytes?: number;
}
export interface PlatformUpdater {
checkForUpdates(): Promise<void>;
downloadAndInstall(): Promise<void>;
restartAndInstall(): Promise<void>;
getStatus(): UpdateStatus;
subscribe(callback: (status: UpdateStatus) => void): () => void;
}
export interface AudioDevice {
id: string;
name: string;
is_default: boolean;
}
export interface PlatformAudio {
isSystemAudioSupported(): boolean;
startSystemAudioCapture(maxDurationSecs: number): Promise<void>;
stopSystemAudioCapture(): Promise<Blob>;
listOutputDevices(): Promise<AudioDevice[]>;
playToDevices(audioData: Uint8Array, deviceIds: string[]): Promise<void>;
stopPlayback(): void;
}
export interface PlatformLifecycle {
startServer(remote?: boolean): Promise<string>;
stopServer(): Promise<void>;
setKeepServerRunning(keep: boolean): Promise<void>;
setupWindowCloseHandler(): Promise<void>;
onServerReady?: () => void;
}
export interface PlatformMetadata {
getVersion(): Promise<string>;
isTauri: boolean;
}
export interface Platform {
filesystem: PlatformFilesystem;
updater: PlatformUpdater;
audio: PlatformAudio;
lifecycle: PlatformLifecycle;
metadata: PlatformMetadata;
}
+2 -1
View File
@@ -10,7 +10,8 @@ import { Toaster } from '@/components/ui/toaster';
import { VoicesTab } from '@/components/VoicesTab/VoicesTab';
import { useModelDownloadToast } from '@/lib/hooks/useModelDownloadToast';
import { MODEL_DISPLAY_NAMES, useRestoreActiveTasks } from '@/lib/hooks/useRestoreActiveTasks';
import { isMacOS } from '@/lib/tauri';
// Simple platform check that works in both web and Tauri
const isMacOS = () => navigator.platform.toLowerCase().includes('mac');
// Root layout component
function RootLayout() {
+31 -8
View File
@@ -5,6 +5,8 @@ interface StoryPlaybackState {
// Selection
selectedStoryId: string | null;
setSelectedStoryId: (id: string | null) => void;
selectedClipId: string | null;
setSelectedClipId: (id: string | null) => void;
// Track editor UI state
trackEditorHeight: number;
@@ -26,6 +28,7 @@ interface StoryPlaybackState {
stop: () => void;
seek: (timeMs: number) => void;
setPlaybackTiming: (contextTime: number, storyTime: number) => void; // Set timing anchors for Web Audio API
setActiveStory: (storyId: string, items: StoryItemDetail[], totalDurationMs: number) => void; // Activate story for seeking without playing
}
const DEFAULT_TRACK_EDITOR_HEIGHT = 250;
@@ -34,6 +37,8 @@ export const useStoryStore = create<StoryPlaybackState>((set, get) => ({
// Selection
selectedStoryId: null,
setSelectedStoryId: (id) => set({ selectedStoryId: id }),
selectedClipId: null,
setSelectedClipId: (id) => set({ selectedClipId: id }),
// Track editor UI state
trackEditorHeight: DEFAULT_TRACK_EDITOR_HEIGHT,
@@ -53,14 +58,11 @@ export const useStoryStore = create<StoryPlaybackState>((set, get) => ({
// Calculate total duration from items
const maxEndTimeMs = Math.max(
...items.map((item) => item.start_time_ms + item.duration * 1000),
0
0,
);
// Find the minimum start time (first item)
const minStartTimeMs = Math.min(
...items.map((item) => item.start_time_ms),
0
);
const minStartTimeMs = Math.min(...items.map((item) => item.start_time_ms), 0);
// If resuming the same story, keep position; otherwise start at first item
const currentState = get();
@@ -70,7 +72,11 @@ export const useStoryStore = create<StoryPlaybackState>((set, get) => ({
console.log('[StoryStore] Play called:', {
storyId,
itemCount: items.length,
items: items.map(i => ({ id: i.generation_id, start: i.start_time_ms, duration: i.duration })),
items: items.map((i) => ({
id: i.generation_id,
start: i.start_time_ms,
duration: i.duration,
})),
maxEndTimeMs,
minStartTimeMs,
startTimeMs,
@@ -83,11 +89,14 @@ export const useStoryStore = create<StoryPlaybackState>((set, get) => ({
playbackItems: items,
totalDurationMs: maxEndTimeMs,
currentTimeMs: startTimeMs,
// Reset timing anchors - will be set fresh by the playback hook
playbackStartContextTime: null,
playbackStartStoryTime: null,
});
},
pause: () => {
set({
set({
isPlaying: false,
// Keep timing anchors so we can resume from same position
});
@@ -108,7 +117,7 @@ export const useStoryStore = create<StoryPlaybackState>((set, get) => ({
seek: (timeMs) => {
const state = get();
const clampedTime = Math.max(0, Math.min(timeMs, state.totalDurationMs));
set({
set({
currentTimeMs: clampedTime,
// Reset timing anchors - will be set by hook when playback resumes
playbackStartContextTime: null,
@@ -122,4 +131,18 @@ export const useStoryStore = create<StoryPlaybackState>((set, get) => ({
playbackStartStoryTime: storyTime,
});
},
setActiveStory: (storyId, items, totalDurationMs) => {
const currentState = get();
// Only update if switching to a different story
if (currentState.playbackStoryId !== storyId) {
set({
playbackStoryId: storyId,
playbackItems: items,
totalDurationMs,
currentTimeMs: 0,
isPlaying: false,
});
}
},
}));
+20
View File
@@ -1,5 +1,18 @@
import { create } from 'zustand';
// Draft state for the create voice profile form
export interface ProfileFormDraft {
name: string;
description: string;
language: string;
referenceText: string;
sampleMode: 'upload' | 'record' | 'system';
// Note: File objects can't be persisted, so we store metadata
sampleFileName?: string;
sampleFileType?: string;
sampleFileData?: string; // Base64 encoded
}
interface UIStore {
// Sidebar
sidebarOpen: boolean;
@@ -18,6 +31,10 @@ interface UIStore {
selectedProfileId: string | null;
setSelectedProfileId: (id: string | null) => void;
// Profile form draft (for persisting create voice modal state)
profileFormDraft: ProfileFormDraft | null;
setProfileFormDraft: (draft: ProfileFormDraft | null) => void;
// Theme
theme: 'light' | 'dark';
setTheme: (theme: 'light' | 'dark') => void;
@@ -38,6 +55,9 @@ export const useUIStore = create<UIStore>((set) => ({
selectedProfileId: null,
setSelectedProfileId: (id) => set({ selectedProfileId: id }),
profileFormDraft: null,
setProfileFormDraft: (draft) => set({ profileFormDraft: draft }),
theme: 'light',
setTheme: (theme) => {
set({ theme });
+28 -7
View File
@@ -19,8 +19,13 @@ Production-quality FastAPI backend for Qwen3-TTS voice cloning.
backend/
├── main.py # FastAPI app with all routes
├── models.py # Pydantic request/response models
├── tts.py # Qwen3-TTS inference
├── transcribe.py # Whisper ASR
├── platform_detect.py # Platform detection for backend selection
├── tts.py # TTS backend abstraction (delegates to MLX or PyTorch)
├── transcribe.py # STT backend abstraction (delegates to MLX or PyTorch)
├── backends/ # Backend implementations
│ ├── __init__.py # Backend factory and protocols
│ ├── mlx_backend.py # MLX backend (Apple Silicon)
│ └── pytorch_backend.py # PyTorch backend (Windows/Linux/Intel)
├── profiles.py # Voice profile CRUD
├── history.py # Generation history
├── studio.py # Audio editing (TODO)
@@ -31,6 +36,15 @@ backend/
└── validation.py # Input validation
```
### Backend Selection
Voicebox automatically selects the best backend based on platform:
- **Apple Silicon (M1/M2/M3)**: Uses MLX backend with native Metal acceleration (4-5x faster)
- **Windows/Linux/Intel Mac**: Uses PyTorch backend (CUDA GPU if available, CPU fallback)
The backend is detected at runtime via `platform_detect.py`. Both backends implement the same interface, so the API remains consistent across platforms.
## API Endpoints
### Health & Info
@@ -47,12 +61,20 @@ Health check with model status.
"status": "healthy",
"model_loaded": true,
"gpu_available": true,
"vram_used_mb": 1024.5
"gpu_type": "Metal (Apple Silicon via MLX)",
"backend_type": "mlx",
"vram_used_mb": null
}
```
**Backend Types:**
- `"mlx"` - MLX backend (Apple Silicon with Metal acceleration)
- `"pytorch"` - PyTorch backend (Windows/Linux/Intel Mac)
### Voice Profiles
**Note:** The database is automatically initialized when the server starts. No manual setup required.
#### `POST /profiles`
Create a new voice profile.
@@ -266,13 +288,12 @@ data/
pip install -r requirements.txt
```
### 2. Initialize Database
**Note:** On Apple Silicon, also install MLX dependencies for faster inference:
```bash
python -c "from database import init_db; init_db()"
pip install -r requirements-mlx.txt
```
### 3. Download Models (Automatic)
### 2. Download Models (Automatic)
The Qwen3-TTS models are automatically downloaded from HuggingFace Hub on first use, similar to how Whisper models work.
+2
View File
@@ -1 +1,3 @@
# Backend package
__version__ = "0.1.13"
+181
View File
@@ -0,0 +1,181 @@
"""
Backend abstraction layer for TTS and STT.
Provides a unified interface for MLX and PyTorch backends.
"""
from typing import Protocol, Optional, Tuple, List
from typing_extensions import runtime_checkable
import numpy as np
from ..platform_detect import get_backend_type
@runtime_checkable
class TTSBackend(Protocol):
"""Protocol for TTS backend implementations."""
async def load_model(self, model_size: str) -> None:
"""Load TTS model."""
...
async def create_voice_prompt(
self,
audio_path: str,
reference_text: str,
use_cache: bool = True,
) -> Tuple[dict, bool]:
"""
Create voice prompt from reference audio.
Returns:
Tuple of (voice_prompt_dict, was_cached)
"""
...
async def combine_voice_prompts(
self,
audio_paths: List[str],
reference_texts: List[str],
) -> Tuple[np.ndarray, str]:
"""
Combine multiple voice prompts.
Returns:
Tuple of (combined_audio_array, combined_text)
"""
...
async def generate(
self,
text: str,
voice_prompt: dict,
language: str = "en",
seed: Optional[int] = None,
instruct: Optional[str] = None,
) -> Tuple[np.ndarray, int]:
"""
Generate audio from text.
Returns:
Tuple of (audio_array, sample_rate)
"""
...
def unload_model(self) -> None:
"""Unload model to free memory."""
...
def is_loaded(self) -> bool:
"""Check if model is loaded."""
...
def _get_model_path(self, model_size: str) -> str:
"""
Get model path for a given size.
Returns:
Model path or HuggingFace Hub ID
"""
...
@runtime_checkable
class STTBackend(Protocol):
"""Protocol for STT (Speech-to-Text) backend implementations."""
async def load_model(self, model_size: str) -> None:
"""Load STT model."""
...
async def transcribe(
self,
audio_path: str,
language: Optional[str] = None,
) -> str:
"""
Transcribe audio to text.
Returns:
Transcribed text
"""
...
def unload_model(self) -> None:
"""Unload model to free memory."""
...
def is_loaded(self) -> bool:
"""Check if model is loaded."""
...
# Global backend instances
_tts_backend: Optional[TTSBackend] = None
_stt_backend: Optional[STTBackend] = None
def get_tts_backend() -> TTSBackend:
"""
Get or create TTS backend instance based on platform.
Returns:
TTS backend instance (MLX or PyTorch)
Raises:
ImportError: If required dependencies (mlx or torch) are not available
"""
global _tts_backend
if _tts_backend is None:
backend_type = get_backend_type()
if backend_type == "mlx":
try:
from .mlx_backend import MLXTTSBackend
_tts_backend = MLXTTSBackend()
except ImportError as e:
raise ImportError(
f"MLX backend dependencies not available. "
f"Please install mlx and mlx_audio or download a provider. Error: {e}"
)
else:
try:
from .pytorch_backend import PyTorchTTSBackend
_tts_backend = PyTorchTTSBackend()
except ImportError as e:
raise ImportError(
f"PyTorch backend dependencies not available. "
f"Please download a TTS provider (pytorch-cpu or pytorch-cuda) from the Downloads page. Error: {e}"
)
return _tts_backend
def get_stt_backend() -> STTBackend:
"""
Get or create STT backend instance based on platform.
Returns:
STT backend instance (MLX or PyTorch)
"""
global _stt_backend
if _stt_backend is None:
backend_type = get_backend_type()
if backend_type == "mlx":
from .mlx_backend import MLXSTTBackend
_stt_backend = MLXSTTBackend()
else:
from .pytorch_backend import PyTorchSTTBackend
_stt_backend = PyTorchSTTBackend()
return _stt_backend
def reset_backends():
"""Reset backend instances (useful for testing)."""
global _tts_backend, _stt_backend
_tts_backend = None
_stt_backend = None
+573
View File
@@ -0,0 +1,573 @@
"""
MLX backend implementation for TTS and STT using mlx-audio.
"""
from typing import Optional, List, Tuple
import asyncio
import numpy as np
from pathlib import Path
from . import TTSBackend, STTBackend
from ..utils.cache import get_cache_key, get_cached_voice_prompt, cache_voice_prompt
from ..utils.audio import normalize_audio, load_audio
from ..utils.progress import get_progress_manager
from ..utils.hf_progress import HFProgressTracker, create_hf_progress_callback
from ..utils.tasks import get_task_manager
class MLXTTSBackend:
"""MLX-based TTS backend using mlx-audio."""
def __init__(self, model_size: str = "1.7B"):
self.model = None
self.model_size = model_size
self._current_model_size = None
def is_loaded(self) -> bool:
"""Check if model is loaded."""
return self.model is not None
def _get_model_path(self, model_size: str) -> str:
"""
Get the MLX model path.
Args:
model_size: Model size (1.7B or 0.6B)
Returns:
HuggingFace Hub model ID for MLX
"""
# MLX model mapping
mlx_model_map = {
"1.7B": "mlx-community/Qwen3-TTS-12Hz-1.7B-Base-bf16",
# 0.6B not yet converted to MLX format
"0.6B": "mlx-community/Qwen3-TTS-12Hz-1.7B-Base-bf16", # Fallback to 1.7B
}
if model_size not in mlx_model_map:
raise ValueError(f"Unknown model size: {model_size}")
hf_model_id = mlx_model_map[model_size]
print(f"Will download MLX model from HuggingFace Hub: {hf_model_id}")
return hf_model_id
def _is_model_cached(self, model_size: str) -> bool:
"""
Check if the model is already cached locally AND fully downloaded.
Args:
model_size: Model size to check
Returns:
True if model is fully cached, False if missing or incomplete
"""
try:
from huggingface_hub import constants as hf_constants
model_path = self._get_model_path(model_size)
repo_cache = Path(hf_constants.HF_HUB_CACHE) / ("models--" + model_path.replace("/", "--"))
if not repo_cache.exists():
return False
# Check for .incomplete files - if any exist, download is still in progress
blobs_dir = repo_cache / "blobs"
if blobs_dir.exists() and any(blobs_dir.glob("*.incomplete")):
print(f"[_is_model_cached] Found .incomplete files for {model_size}, treating as not cached")
return False
# Check that actual model weight files exist in snapshots
snapshots_dir = repo_cache / "snapshots"
if snapshots_dir.exists():
has_weights = (
any(snapshots_dir.rglob("*.safetensors")) or
any(snapshots_dir.rglob("*.bin")) or
any(snapshots_dir.rglob("*.npz"))
)
if not has_weights:
print(f"[_is_model_cached] No model weights found for {model_size}, treating as not cached")
return False
return True
except Exception as e:
print(f"[_is_model_cached] Error checking cache for {model_size}: {e}")
return False
async def load_model_async(self, model_size: Optional[str] = None):
"""
Lazy load the MLX TTS model.
Args:
model_size: Model size to load (1.7B or 0.6B)
"""
if model_size is None:
model_size = self.model_size
# If already loaded with correct size, return
if self.model is not None and self._current_model_size == model_size:
return
# Unload existing model if different size requested
if self.model is not None and self._current_model_size != model_size:
self.unload_model()
# Run blocking load in thread pool
await asyncio.to_thread(self._load_model_sync, model_size)
# Alias for compatibility
load_model = load_model_async
def _load_model_sync(self, model_size: str):
"""Synchronous model loading."""
try:
# Get model path BEFORE importing mlx_audio
model_path = self._get_model_path(model_size)
# Set up progress tracking
progress_manager = get_progress_manager()
task_manager = get_task_manager()
model_name = f"qwen-tts-{model_size}"
# Check if model is already cached
is_cached = self._is_model_cached(model_size)
# Set up progress callback
# If cached: filter out non-download progress
# If not cached: report all progress (we're actually downloading)
progress_callback = create_hf_progress_callback(model_name, progress_manager)
tracker = HFProgressTracker(progress_callback, filter_non_downloads=is_cached)
print(f"Loading MLX TTS model {model_size}...")
# Only track download progress if model is NOT cached
if not is_cached:
# Start tracking download task
task_manager.start_download(model_name)
# Initialize progress state so SSE endpoint has initial data to send
# This provides immediate feedback while HuggingFace fetches metadata
progress_manager.update_progress(
model_name=model_name,
current=0,
total=0, # Will be updated once actual total is known
filename="Connecting to HuggingFace...",
status="downloading",
)
# IMPORTANT: Patch tqdm BEFORE importing mlx_audio
# Otherwise mlx_audio caches reference to original tqdm
tracker_context = tracker.patch_download()
tracker_context.__enter__()
# Import mlx_audio AFTER patching tqdm
from mlx_audio.tts import load
# Load MLX model (downloads automatically)
try:
self.model = load(model_path)
finally:
# Exit the patch context
tracker_context.__exit__(None, None, None)
# Only mark download as complete if we were tracking it
if not is_cached:
progress_manager.mark_complete(model_name)
task_manager.complete_download(model_name)
self._current_model_size = model_size
self.model_size = model_size
print(f"MLX TTS model {model_size} loaded successfully")
except ImportError as e:
print(f"Error: mlx_audio package not found. Install with: pip install mlx-audio")
progress_manager = get_progress_manager()
task_manager = get_task_manager()
model_name = f"qwen-tts-{model_size}"
progress_manager.mark_error(model_name, str(e))
task_manager.error_download(model_name, str(e))
raise
except Exception as e:
print(f"Error loading MLX TTS model: {e}")
progress_manager = get_progress_manager()
task_manager = get_task_manager()
model_name = f"qwen-tts-{model_size}"
progress_manager.mark_error(model_name, str(e))
task_manager.error_download(model_name, str(e))
raise
def unload_model(self):
"""Unload the model to free memory."""
if self.model is not None:
del self.model
self.model = None
self._current_model_size = None
print("MLX TTS model unloaded")
async def create_voice_prompt(
self,
audio_path: str,
reference_text: str,
use_cache: bool = True,
) -> Tuple[dict, bool]:
"""
Create voice prompt from reference audio.
MLX backend stores voice prompt as a dict with audio path and text.
The actual voice prompt processing happens during generation.
Args:
audio_path: Path to reference audio file
reference_text: Transcript of reference audio
use_cache: Whether to use cached prompt if available
Returns:
Tuple of (voice_prompt_dict, was_cached)
"""
await self.load_model_async(None)
# Check cache if enabled
if use_cache:
cache_key = get_cache_key(audio_path, reference_text)
cached_prompt = get_cached_voice_prompt(cache_key)
if cached_prompt is not None:
# Return cached prompt (should be dict format)
if isinstance(cached_prompt, dict):
# Validate that the cached audio file still exists
cached_audio_path = cached_prompt.get("ref_audio") or cached_prompt.get("ref_audio_path")
if cached_audio_path and Path(cached_audio_path).exists():
return cached_prompt, True
else:
# Cached file no longer exists, invalidate cache
print(f"Cached audio file not found: {cached_audio_path}, regenerating prompt")
# MLX voice prompt format - store audio path and text
# The model will process this during generation
voice_prompt_items = {
"ref_audio": str(audio_path),
"ref_text": reference_text,
}
# Cache if enabled
if use_cache:
cache_key = get_cache_key(audio_path, reference_text)
cache_voice_prompt(cache_key, voice_prompt_items)
return voice_prompt_items, False
async def combine_voice_prompts(
self,
audio_paths: List[str],
reference_texts: List[str],
) -> Tuple[np.ndarray, str]:
"""
Combine multiple reference samples for better quality.
Args:
audio_paths: List of audio file paths
reference_texts: List of reference texts
Returns:
Tuple of (combined_audio, combined_text)
"""
combined_audio = []
for audio_path in audio_paths:
audio, sr = load_audio(audio_path)
audio = normalize_audio(audio)
combined_audio.append(audio)
# Concatenate audio
mixed = np.concatenate(combined_audio)
mixed = normalize_audio(mixed)
# Combine texts
combined_text = " ".join(reference_texts)
return mixed, combined_text
async def generate(
self,
text: str,
voice_prompt: dict,
language: str = "en",
seed: Optional[int] = None,
instruct: Optional[str] = None,
) -> Tuple[np.ndarray, int]:
"""
Generate audio from text using voice prompt.
Args:
text: Text to synthesize
voice_prompt: Voice prompt dictionary with ref_audio and ref_text
language: Language code (en or zh) - may not be fully supported by MLX
seed: Random seed for reproducibility
instruct: Natural language instruction (may not be supported by MLX)
Returns:
Tuple of (audio_array, sample_rate)
"""
await self.load_model_async(None)
print(f"Generating audio for text: {text}")
def _generate_sync():
"""Run synchronous generation in thread pool."""
# MLX generate() returns a generator yielding GenerationResult objects
audio_chunks = []
sample_rate = 24000
# Set seed if provided (MLX uses numpy random)
if seed is not None:
import mlx.core as mx
np.random.seed(seed)
mx.random.seed(seed)
# Extract voice prompt info
ref_audio = voice_prompt.get("ref_audio") or voice_prompt.get("ref_audio_path")
ref_text = voice_prompt.get("ref_text", "")
# Validate that the audio file exists
if ref_audio and not Path(ref_audio).exists():
print(f"Warning: Audio file not found: {ref_audio}")
print("This may be due to a cached voice prompt referencing a deleted temp file.")
print("Regenerating without voice prompt.")
ref_audio = None
# Check if model supports voice cloning via generate method
# MLX API may support ref_audio parameter directly
try:
# Try with voice cloning parameters if supported
if ref_audio:
# Check if generate accepts ref_audio parameter
import inspect
sig = inspect.signature(self.model.generate)
if "ref_audio" in sig.parameters:
# Generate with voice cloning
for result in self.model.generate(text, ref_audio=ref_audio, ref_text=ref_text):
audio_chunks.append(np.array(result.audio))
sample_rate = result.sample_rate
else:
# Fallback: generate without voice cloning
for result in self.model.generate(text):
audio_chunks.append(np.array(result.audio))
sample_rate = result.sample_rate
else:
# No voice prompt, generate normally
for result in self.model.generate(text):
audio_chunks.append(np.array(result.audio))
sample_rate = result.sample_rate
except Exception as e:
# If voice cloning fails, try without it
print(f"Warning: Voice cloning failed, generating without voice prompt: {e}")
for result in self.model.generate(text):
audio_chunks.append(np.array(result.audio))
sample_rate = result.sample_rate
# Concatenate all chunks
if audio_chunks:
audio = np.concatenate([np.asarray(chunk, dtype=np.float32) for chunk in audio_chunks])
else:
# Fallback: empty audio
audio = np.array([], dtype=np.float32)
return audio, sample_rate
# Run blocking inference in thread pool
audio, sample_rate = await asyncio.to_thread(_generate_sync)
return audio, sample_rate
class MLXSTTBackend:
"""MLX-based STT backend using mlx-audio Whisper."""
def __init__(self, model_size: str = "base"):
self.model = None
self.model_size = model_size
def is_loaded(self) -> bool:
"""Check if model is loaded."""
return self.model is not None
def _is_model_cached(self, model_size: str) -> bool:
"""
Check if the Whisper model is already cached locally AND fully downloaded.
Args:
model_size: Model size to check
Returns:
True if model is fully cached, False if missing or incomplete
"""
try:
from huggingface_hub import constants as hf_constants
model_name = f"openai/whisper-{model_size}"
repo_cache = Path(hf_constants.HF_HUB_CACHE) / ("models--" + model_name.replace("/", "--"))
if not repo_cache.exists():
return False
# Check for .incomplete files - if any exist, download is still in progress
blobs_dir = repo_cache / "blobs"
if blobs_dir.exists() and any(blobs_dir.glob("*.incomplete")):
print(f"[_is_model_cached] Found .incomplete files for whisper-{model_size}, treating as not cached")
return False
# Check that actual model weight files exist in snapshots
snapshots_dir = repo_cache / "snapshots"
if snapshots_dir.exists():
has_weights = (
any(snapshots_dir.rglob("*.safetensors")) or
any(snapshots_dir.rglob("*.bin")) or
any(snapshots_dir.rglob("*.npz"))
)
if not has_weights:
print(f"[_is_model_cached] No model weights found for whisper-{model_size}, treating as not cached")
return False
return True
except Exception as e:
print(f"[_is_model_cached] Error checking cache for whisper-{model_size}: {e}")
return False
async def load_model_async(self, model_size: Optional[str] = None):
"""
Lazy load the MLX Whisper model.
Args:
model_size: Model size (tiny, base, small, medium, large)
"""
if model_size is None:
model_size = self.model_size
if self.model is not None and self.model_size == model_size:
return
# Run blocking load in thread pool
await asyncio.to_thread(self._load_model_sync, model_size)
# Alias for compatibility
load_model = load_model_async
def _load_model_sync(self, model_size: str):
"""Synchronous model loading."""
try:
progress_manager = get_progress_manager()
task_manager = get_task_manager()
progress_model_name = f"whisper-{model_size}"
# Check if model is already cached
is_cached = self._is_model_cached(model_size)
# Set up progress callback and tracker
# If cached: filter out non-download progress
# If not cached: report all progress (we're actually downloading)
progress_callback = create_hf_progress_callback(progress_model_name, progress_manager)
tracker = HFProgressTracker(progress_callback, filter_non_downloads=is_cached)
# Patch tqdm BEFORE importing mlx_audio
tracker_context = tracker.patch_download()
tracker_context.__enter__()
# Import mlx_audio
from mlx_audio.stt import load
# MLX Whisper uses the standard OpenAI models
model_name = f"openai/whisper-{model_size}"
print(f"Loading MLX Whisper model {model_size}...")
# Only track download progress if model is NOT cached
if not is_cached:
# Start tracking download task
task_manager.start_download(progress_model_name)
# Initialize progress state so SSE endpoint has initial data to send
progress_manager.update_progress(
model_name=progress_model_name,
current=0,
total=0,
filename="Connecting to HuggingFace...",
status="downloading",
)
# Load the model (tqdm is patched, but filters out non-download progress)
try:
self.model = load(model_name)
finally:
# Exit the patch context
tracker_context.__exit__(None, None, None)
# Only mark download as complete if we were tracking it
if not is_cached:
progress_manager.mark_complete(progress_model_name)
task_manager.complete_download(progress_model_name)
self.model_size = model_size
print(f"MLX Whisper model {model_size} loaded successfully")
except ImportError as e:
print(f"Error: mlx_audio package not found. Install with: pip install mlx-audio")
progress_manager = get_progress_manager()
task_manager = get_task_manager()
progress_model_name = f"whisper-{model_size}"
progress_manager.mark_error(progress_model_name, str(e))
task_manager.error_download(progress_model_name, str(e))
raise
except Exception as e:
print(f"Error loading MLX Whisper model: {e}")
progress_manager = get_progress_manager()
task_manager = get_task_manager()
progress_model_name = f"whisper-{model_size}"
progress_manager.mark_error(progress_model_name, str(e))
task_manager.error_download(progress_model_name, str(e))
raise
def unload_model(self):
"""Unload the model to free memory."""
if self.model is not None:
del self.model
self.model = None
print("MLX Whisper model unloaded")
async def transcribe(
self,
audio_path: str,
language: Optional[str] = None,
) -> str:
"""
Transcribe audio to text.
Args:
audio_path: Path to audio file
language: Optional language hint (en or zh)
Returns:
Transcribed text
"""
await self.load_model_async(None)
def _transcribe_sync():
"""Run synchronous transcription in thread pool."""
# MLX Whisper transcription using generate method
# The generate method accepts audio path directly
decode_options = {}
if language:
decode_options["language"] = language
result = self.model.generate(str(audio_path), **decode_options)
# Extract text from result
if isinstance(result, str):
return result.strip()
elif isinstance(result, dict):
return result.get("text", "").strip()
elif hasattr(result, "text"):
return result.text.strip()
else:
return str(result).strip()
# Run blocking transcription in thread pool
return await asyncio.to_thread(_transcribe_sync)
+575
View File
@@ -0,0 +1,575 @@
"""
PyTorch backend implementation for TTS and STT.
"""
from typing import Optional, List, Tuple
import asyncio
import torch
import numpy as np
from pathlib import Path
from . import TTSBackend, STTBackend
from ..utils.cache import get_cache_key, get_cached_voice_prompt, cache_voice_prompt
from ..utils.audio import normalize_audio, load_audio
from ..utils.progress import get_progress_manager
from ..utils.hf_progress import HFProgressTracker, create_hf_progress_callback
from ..utils.tasks import get_task_manager
class PyTorchTTSBackend:
"""PyTorch-based TTS backend using Qwen3-TTS."""
def __init__(self, model_size: str = "1.7B"):
self.model = None
self.model_size = model_size
self.device = self._get_device()
self._current_model_size = None
def _get_device(self) -> str:
"""Get the best available device."""
if torch.cuda.is_available():
return "cuda"
elif hasattr(torch.backends, 'mps') and torch.backends.mps.is_available():
# MPS can have issues, use CPU for stability
return "cpu"
return "cpu"
def is_loaded(self) -> bool:
"""Check if model is loaded."""
return self.model is not None
def _get_model_path(self, model_size: str) -> str:
"""
Get the HuggingFace Hub model ID.
Args:
model_size: Model size (1.7B or 0.6B)
Returns:
HuggingFace Hub model ID
"""
hf_model_map = {
"1.7B": "Qwen/Qwen3-TTS-12Hz-1.7B-Base",
"0.6B": "Qwen/Qwen3-TTS-12Hz-0.6B-Base",
}
if model_size not in hf_model_map:
raise ValueError(f"Unknown model size: {model_size}")
return hf_model_map[model_size]
def _is_model_cached(self, model_size: str) -> bool:
"""
Check if the model is already cached locally AND fully downloaded.
Args:
model_size: Model size to check
Returns:
True if model is fully cached, False if missing or incomplete
"""
try:
from huggingface_hub import constants as hf_constants
model_path = self._get_model_path(model_size)
repo_cache = Path(hf_constants.HF_HUB_CACHE) / ("models--" + model_path.replace("/", "--"))
if not repo_cache.exists():
return False
# Check for .incomplete files - if any exist, download is still in progress
blobs_dir = repo_cache / "blobs"
if blobs_dir.exists() and any(blobs_dir.glob("*.incomplete")):
print(f"[_is_model_cached] Found .incomplete files for {model_size}, treating as not cached")
return False
# Check that actual model weight files exist in snapshots
snapshots_dir = repo_cache / "snapshots"
if snapshots_dir.exists():
has_weights = (
any(snapshots_dir.rglob("*.safetensors")) or
any(snapshots_dir.rglob("*.bin"))
)
if not has_weights:
print(f"[_is_model_cached] No model weights found for {model_size}, treating as not cached")
return False
return True
except Exception as e:
print(f"[_is_model_cached] Error checking cache for {model_size}: {e}")
return False
async def load_model_async(self, model_size: Optional[str] = None):
"""
Lazy load the TTS model with automatic downloading from HuggingFace Hub.
Args:
model_size: Model size to load (1.7B or 0.6B)
"""
if model_size is None:
model_size = self.model_size
# If already loaded with correct size, return
if self.model is not None and self._current_model_size == model_size:
return
# Unload existing model if different size requested
if self.model is not None and self._current_model_size != model_size:
self.unload_model()
# Run blocking load in thread pool
await asyncio.to_thread(self._load_model_sync, model_size)
# Alias for compatibility
load_model = load_model_async
def _load_model_sync(self, model_size: str):
"""Synchronous model loading."""
try:
progress_manager = get_progress_manager()
task_manager = get_task_manager()
model_name = f"qwen-tts-{model_size}"
# Check if model is already cached
is_cached = self._is_model_cached(model_size)
# Set up progress callback and tracker
# If cached: filter out non-download progress (like "Segment 1/1" during generation)
# If not cached: report all progress (we're actually downloading)
progress_callback = create_hf_progress_callback(model_name, progress_manager)
tracker = HFProgressTracker(progress_callback, filter_non_downloads=is_cached)
# Patch tqdm BEFORE importing qwen_tts
tracker_context = tracker.patch_download()
tracker_context.__enter__()
# Import qwen_tts
from qwen_tts import Qwen3TTSModel
# Get model path (local or HuggingFace Hub ID)
model_path = self._get_model_path(model_size)
print(f"Loading TTS model {model_size} on {self.device}...")
# Only track download progress if model is NOT cached
if not is_cached:
# Start tracking download task
task_manager.start_download(model_name)
# Initialize progress state so SSE endpoint has initial data to send
progress_manager.update_progress(
model_name=model_name,
current=0,
total=0, # Will be updated once actual total is known
filename="Connecting to HuggingFace...",
status="downloading",
)
# Load the model (tqdm is patched, but filters out non-download progress)
try:
self.model = Qwen3TTSModel.from_pretrained(
model_path,
device_map=self.device,
torch_dtype=torch.float32 if self.device == "cpu" else torch.bfloat16,
)
finally:
# Exit the patch context
tracker_context.__exit__(None, None, None)
# Only mark download as complete if we were tracking it
if not is_cached:
progress_manager.mark_complete(model_name)
task_manager.complete_download(model_name)
self._current_model_size = model_size
self.model_size = model_size
print(f"TTS model {model_size} loaded successfully")
except ImportError as e:
print(f"Error: qwen_tts package not found. Install with: pip install git+https://github.com/QwenLM/Qwen3-TTS.git")
progress_manager = get_progress_manager()
task_manager = get_task_manager()
model_name = f"qwen-tts-{model_size}"
progress_manager.mark_error(model_name, str(e))
task_manager.error_download(model_name, str(e))
raise
except Exception as e:
print(f"Error loading TTS model: {e}")
print(f"Tip: The model will be automatically downloaded from HuggingFace Hub on first use.")
progress_manager = get_progress_manager()
task_manager = get_task_manager()
model_name = f"qwen-tts-{model_size}"
progress_manager.mark_error(model_name, str(e))
task_manager.error_download(model_name, str(e))
raise
def unload_model(self):
"""Unload the model to free memory."""
if self.model is not None:
del self.model
self.model = None
self._current_model_size = None
if torch.cuda.is_available():
torch.cuda.empty_cache()
print("TTS model unloaded")
async def create_voice_prompt(
self,
audio_path: str,
reference_text: str,
use_cache: bool = True,
) -> Tuple[dict, bool]:
"""
Create voice prompt from reference audio.
Args:
audio_path: Path to reference audio file
reference_text: Transcript of reference audio
use_cache: Whether to use cached prompt if available
Returns:
Tuple of (voice_prompt_dict, was_cached)
"""
await self.load_model_async(None)
# Check cache if enabled
if use_cache:
cache_key = get_cache_key(audio_path, reference_text)
cached_prompt = get_cached_voice_prompt(cache_key)
if cached_prompt is not None:
# Cache stores as torch.Tensor but actual prompt is dict
# Convert if needed
if isinstance(cached_prompt, dict):
# For PyTorch backend, the dict should contain tensors, not file paths
# So we can safely return it
return cached_prompt, True
elif isinstance(cached_prompt, torch.Tensor):
# Legacy cache format - convert to dict
# This shouldn't happen in practice, but handle it
return {"prompt": cached_prompt}, True
def _create_prompt_sync():
"""Run synchronous voice prompt creation in thread pool."""
return self.model.create_voice_clone_prompt(
ref_audio=str(audio_path),
ref_text=reference_text,
x_vector_only_mode=False,
)
# Run blocking operation in thread pool
voice_prompt_items = await asyncio.to_thread(_create_prompt_sync)
# Cache if enabled
if use_cache:
cache_key = get_cache_key(audio_path, reference_text)
cache_voice_prompt(cache_key, voice_prompt_items)
return voice_prompt_items, False
async def combine_voice_prompts(
self,
audio_paths: List[str],
reference_texts: List[str],
) -> Tuple[np.ndarray, str]:
"""
Combine multiple reference samples for better quality.
Args:
audio_paths: List of audio file paths
reference_texts: List of reference texts
Returns:
Tuple of (combined_audio, combined_text)
"""
combined_audio = []
for audio_path in audio_paths:
audio, sr = load_audio(audio_path)
audio = normalize_audio(audio)
combined_audio.append(audio)
# Concatenate audio
mixed = np.concatenate(combined_audio)
mixed = normalize_audio(mixed)
# Combine texts
combined_text = " ".join(reference_texts)
return mixed, combined_text
async def generate(
self,
text: str,
voice_prompt: dict,
language: str = "en",
seed: Optional[int] = None,
instruct: Optional[str] = None,
) -> Tuple[np.ndarray, int]:
"""
Generate audio from text using voice prompt.
Args:
text: Text to synthesize
voice_prompt: Voice prompt dictionary from create_voice_prompt
language: Language code (en or zh)
seed: Random seed for reproducibility
instruct: Natural language instruction for speech delivery control
Returns:
Tuple of (audio_array, sample_rate)
"""
# Load model
await self.load_model_async(None)
def _generate_sync():
"""Run synchronous generation in thread pool."""
# Set seed if provided
if seed is not None:
torch.manual_seed(seed)
if torch.cuda.is_available():
torch.cuda.manual_seed(seed)
# Generate audio - this is the blocking operation
wavs, sample_rate = self.model.generate_voice_clone(
text=text,
voice_clone_prompt=voice_prompt,
instruct=instruct,
)
return wavs[0], sample_rate
# Run blocking inference in thread pool to avoid blocking event loop
audio, sample_rate = await asyncio.to_thread(_generate_sync)
return audio, sample_rate
class PyTorchSTTBackend:
"""PyTorch-based STT backend using Whisper."""
def __init__(self, model_size: str = "base"):
self.model = None
self.processor = None
self.model_size = model_size
self.device = self._get_device()
def _get_device(self) -> str:
"""Get the best available device."""
if torch.cuda.is_available():
return "cuda"
elif hasattr(torch.backends, 'mps') and torch.backends.mps.is_available():
# MPS support for Whisper
return "cpu" # Use CPU for stability
return "cpu"
def is_loaded(self) -> bool:
"""Check if model is loaded."""
return self.model is not None
def _is_model_cached(self, model_size: str) -> bool:
"""
Check if the Whisper model is already cached locally AND fully downloaded.
Args:
model_size: Model size to check
Returns:
True if model is fully cached, False if missing or incomplete
"""
try:
from huggingface_hub import constants as hf_constants
model_name = f"openai/whisper-{model_size}"
repo_cache = Path(hf_constants.HF_HUB_CACHE) / ("models--" + model_name.replace("/", "--"))
if not repo_cache.exists():
return False
# Check for .incomplete files - if any exist, download is still in progress
blobs_dir = repo_cache / "blobs"
if blobs_dir.exists() and any(blobs_dir.glob("*.incomplete")):
print(f"[_is_model_cached] Found .incomplete files for whisper-{model_size}, treating as not cached")
return False
# Check that actual model weight files exist in snapshots
snapshots_dir = repo_cache / "snapshots"
if snapshots_dir.exists():
has_weights = (
any(snapshots_dir.rglob("*.safetensors")) or
any(snapshots_dir.rglob("*.bin"))
)
if not has_weights:
print(f"[_is_model_cached] No model weights found for whisper-{model_size}, treating as not cached")
return False
return True
except Exception as e:
print(f"[_is_model_cached] Error checking cache for whisper-{model_size}: {e}")
return False
async def load_model_async(self, model_size: Optional[str] = None):
"""
Lazy load the Whisper model.
Args:
model_size: Model size (tiny, base, small, medium, large)
"""
print(f"[DEBUG] load_model_async called with size: {model_size}")
if model_size is None:
model_size = self.model_size
print(f"[DEBUG] Model already loaded? {self.model is not None}, current size: {self.model_size}, requested: {model_size}")
if self.model is not None and self.model_size == model_size:
print(f"[DEBUG] Early return - model already loaded")
return
print(f"[DEBUG] Calling asyncio.to_thread for _load_model_sync")
# Run blocking load in thread pool
await asyncio.to_thread(self._load_model_sync, model_size)
print(f"[DEBUG] asyncio.to_thread completed")
# Alias for compatibility
load_model = load_model_async
def _load_model_sync(self, model_size: str):
"""Synchronous model loading."""
print(f"[DEBUG] _load_model_sync called for Whisper {model_size}")
try:
progress_manager = get_progress_manager()
task_manager = get_task_manager()
progress_model_name = f"whisper-{model_size}"
# Check if model is already cached
is_cached = self._is_model_cached(model_size)
# Set up progress callback and tracker
# If cached: filter out non-download progress
# If not cached: report all progress (we're actually downloading)
progress_callback = create_hf_progress_callback(progress_model_name, progress_manager)
tracker = HFProgressTracker(progress_callback, filter_non_downloads=is_cached)
# Patch tqdm BEFORE importing transformers
print("[DEBUG] Starting tqdm patch BEFORE transformers import")
tracker_context = tracker.patch_download()
tracker_context.__enter__()
print("[DEBUG] tqdm patched, now importing transformers")
# Import transformers
from transformers import WhisperProcessor, WhisperForConditionalGeneration
model_name = f"openai/whisper-{model_size}"
print(f"[DEBUG] Model name: {model_name}")
print(f"Loading Whisper model {model_size} on {self.device}...")
# Only track download progress if model is NOT cached
if not is_cached:
# Start tracking download task
task_manager.start_download(progress_model_name)
# Initialize progress state so SSE endpoint has initial data to send
progress_manager.update_progress(
model_name=progress_model_name,
current=0,
total=0, # Will be updated once actual total is known
filename="Connecting to HuggingFace...",
status="downloading",
)
# Load models (tqdm is patched, but filters out non-download progress)
try:
self.processor = WhisperProcessor.from_pretrained(model_name)
self.model = WhisperForConditionalGeneration.from_pretrained(model_name)
finally:
# Exit the patch context
tracker_context.__exit__(None, None, None)
# Only mark download as complete if we were tracking it
if not is_cached:
progress_manager.mark_complete(progress_model_name)
task_manager.complete_download(progress_model_name)
self.model.to(self.device)
self.model_size = model_size
print(f"Whisper model {model_size} loaded successfully")
except Exception as e:
print(f"Error loading Whisper model: {e}")
progress_manager = get_progress_manager()
task_manager = get_task_manager()
progress_model_name = f"whisper-{model_size}"
progress_manager.mark_error(progress_model_name, str(e))
task_manager.error_download(progress_model_name, str(e))
raise
def unload_model(self):
"""Unload the model to free memory."""
if self.model is not None:
del self.model
del self.processor
self.model = None
self.processor = None
if torch.cuda.is_available():
torch.cuda.empty_cache()
print("Whisper model unloaded")
async def transcribe(
self,
audio_path: str,
language: Optional[str] = None,
) -> str:
"""
Transcribe audio to text.
Args:
audio_path: Path to audio file
language: Optional language hint (en or zh)
Returns:
Transcribed text
"""
await self.load_model_async(None)
def _transcribe_sync():
"""Run synchronous transcription in thread pool."""
# Load audio
audio, sr = load_audio(audio_path, sample_rate=16000)
# Process audio
inputs = self.processor(
audio,
sampling_rate=16000,
return_tensors="pt",
)
inputs = inputs.to(self.device)
# Set language if provided
forced_decoder_ids = None
if language:
# Support all languages from frontend: en, zh, ja, ko, de, fr, ru, pt, es, it
# Whisper supports these and many more
forced_decoder_ids = self.processor.get_decoder_prompt_ids(
language=language,
task="transcribe",
)
# Generate transcription
with torch.no_grad():
predicted_ids = self.model.generate(
inputs["input_features"],
forced_decoder_ids=forced_decoder_ids,
)
# Decode
transcription = self.processor.batch_decode(
predicted_ids,
skip_special_tokens=True,
)[0]
return transcription.strip()
# Run blocking transcription in thread pool
return await asyncio.to_thread(_transcribe_sync)
+76 -19
View File
@@ -4,16 +4,19 @@ PyInstaller build script for creating standalone Python server binary.
import PyInstaller.__main__
import os
import platform
from pathlib import Path
def is_apple_silicon():
"""Check if running on Apple Silicon."""
return platform.system() == "Darwin" and platform.machine() == "arm64"
def build_server():
"""Build Python server as standalone binary."""
backend_dir = Path(__file__).parent
# Check for local editable qwen_tts install
local_qwen_path = Path.home() / 'Projects' / 'voice' / 'Qwen3-TTS'
# PyInstaller arguments
args = [
'server.py', # Use server.py as entry point instead of main.py
@@ -21,12 +24,13 @@ def build_server():
'--name', 'voicebox-server',
]
# Add local qwen_tts path if it exists (for editable installs)
if local_qwen_path.exists():
args.extend(['--paths', str(local_qwen_path)])
print(f"Using local qwen_tts source from: {local_qwen_path}")
# Add local qwen_tts path if specified (for editable installs)
qwen_tts_path = os.getenv('QWEN_TTS_PATH')
if qwen_tts_path and Path(qwen_tts_path).exists():
args.extend(['--paths', str(qwen_tts_path)])
print(f"Using local qwen_tts source from: {qwen_tts_path}")
# Add hidden imports
# Add common hidden imports (always included)
args.extend([
'--hidden-import', 'backend',
'--hidden-import', 'backend.main',
@@ -37,30 +41,83 @@ def build_server():
'--hidden-import', 'backend.history',
'--hidden-import', 'backend.tts',
'--hidden-import', 'backend.transcribe',
'--hidden-import', 'backend.platform_detect',
'--hidden-import', 'backend.providers',
'--hidden-import', 'backend.providers.base',
'--hidden-import', 'backend.providers.bundled',
'--hidden-import', 'backend.providers.types',
'--hidden-import', 'backend.utils.audio',
'--hidden-import', 'backend.utils.cache',
'--hidden-import', 'backend.utils.progress',
'--hidden-import', 'backend.utils.hf_progress',
'--hidden-import', 'backend.utils.validation',
'--hidden-import', 'torch',
'--hidden-import', 'transformers',
'--hidden-import', 'numpy',
'--hidden-import', 'numpy.core',
'--hidden-import', 'numpy.core._multiarray_umath',
'--hidden-import', 'scipy',
'--hidden-import', 'scipy.signal',
'--hidden-import', 'fastapi',
'--hidden-import', 'uvicorn',
'--hidden-import', 'sqlalchemy',
'--hidden-import', 'librosa',
'--hidden-import', 'soundfile',
'--hidden-import', 'qwen_tts',
'--hidden-import', 'qwen_tts.inference',
'--hidden-import', 'qwen_tts.inference.qwen3_tts_model',
'--hidden-import', 'qwen_tts.inference.qwen3_tts_tokenizer',
'--hidden-import', 'qwen_tts.core',
'--hidden-import', 'qwen_tts.cli',
'--copy-metadata', 'qwen-tts',
'--collect-submodules', 'qwen_tts',
'--collect-data', 'qwen_tts',
# Fix for pkg_resources and jaraco namespace packages
'--hidden-import', 'pkg_resources.extern',
'--collect-submodules', 'jaraco',
# Asyncio and threading support for PyInstaller
'--hidden-import', 'asyncio',
'--hidden-import', 'asyncio.subprocess',
'--hidden-import', 'concurrent.futures',
'--hidden-import', 'concurrent.futures.thread',
])
# Platform-specific TTS backend handling
system = platform.system()
if is_apple_silicon():
print("Building for Apple Silicon - including MLX dependencies (bundled)")
args.extend([
'--hidden-import', 'backend.backends',
'--hidden-import', 'backend.backends.mlx_backend',
'--hidden-import', 'mlx',
'--hidden-import', 'mlx.core',
'--hidden-import', 'mlx.nn',
'--hidden-import', 'mlx_audio',
'--hidden-import', 'mlx_audio.tts',
'--hidden-import', 'mlx_audio.stt',
'--collect-submodules', 'mlx',
'--collect-submodules', 'mlx_audio',
# Collect MLX data files including Metal shader libraries (.metallib)
'--collect-data', 'mlx',
'--collect-data', 'mlx_audio',
])
elif system == "Windows" or (system == "Darwin" and not is_apple_silicon()):
# Windows and Intel macOS: Bundle PyTorch CPU provider
print(f"Building for {system} - including PyTorch CPU provider (bundled)")
args.extend([
'--hidden-import', 'backend.backends',
'--hidden-import', 'backend.backends.pytorch_backend',
'--hidden-import', 'torch',
'--hidden-import', 'transformers',
'--hidden-import', 'qwen_tts',
'--hidden-import', 'qwen_tts.inference',
'--hidden-import', 'qwen_tts.inference.qwen3_tts_model',
'--hidden-import', 'qwen_tts.inference.qwen3_tts_tokenizer',
'--hidden-import', 'qwen_tts.core',
'--hidden-import', 'qwen_tts.cli',
'--copy-metadata', 'qwen-tts',
'--collect-submodules', 'qwen_tts',
'--collect-data', 'qwen_tts',
])
else:
# Linux: No bundled provider - users download providers separately
print("Building for Linux - no bundled provider (users download separately)")
args.extend([
'--hidden-import', 'backend.backends',
'--hidden-import', 'backend.backends.pytorch_backend',
])
args.extend([
'--noconfirm',
'--clean',
])
+32 -1
View File
@@ -17,11 +17,12 @@ Base = declarative_base()
class VoiceProfile(Base):
"""Voice profile database model."""
__tablename__ = "profiles"
id = Column(String, primary_key=True, default=lambda: str(uuid.uuid4()))
name = Column(String, unique=True, nullable=False)
description = Column(Text)
language = Column(String, default="en")
avatar_path = Column(String, nullable=True)
created_at = Column(DateTime, default=datetime.utcnow)
updated_at = Column(DateTime, default=datetime.utcnow, onupdate=datetime.utcnow)
@@ -71,6 +72,8 @@ class StoryItem(Base):
generation_id = Column(String, ForeignKey("generations.id"), nullable=False)
start_time_ms = Column(Integer, nullable=False, default=0) # Milliseconds from story start
track = Column(Integer, nullable=False, default=0) # Track number (0 = main track)
trim_start_ms = Column(Integer, nullable=False, default=0) # Milliseconds trimmed from start
trim_end_ms = Column(Integer, nullable=False, default=0) # Milliseconds trimmed from end
created_at = Column(DateTime, default=datetime.utcnow)
@@ -256,6 +259,34 @@ def _run_migrations(engine):
conn.execute(text("ALTER TABLE story_items ADD COLUMN track INTEGER NOT NULL DEFAULT 0"))
conn.commit()
print("Added track column to story_items")
# Migration: Add trim columns if they don't exist
# Re-check columns after potential track migration
columns = {col['name'] for col in inspector.get_columns('story_items')}
if 'trim_start_ms' not in columns:
print("Migrating story_items: adding trim_start_ms column")
with engine.connect() as conn:
conn.execute(text("ALTER TABLE story_items ADD COLUMN trim_start_ms INTEGER NOT NULL DEFAULT 0"))
conn.commit()
print("Added trim_start_ms column to story_items")
columns = {col['name'] for col in inspector.get_columns('story_items')}
if 'trim_end_ms' not in columns:
print("Migrating story_items: adding trim_end_ms column")
with engine.connect() as conn:
conn.execute(text("ALTER TABLE story_items ADD COLUMN trim_end_ms INTEGER NOT NULL DEFAULT 0"))
conn.commit()
print("Added trim_end_ms column to story_items")
# Migration: Add avatar_path to profiles table
if 'profiles' in inspector.get_table_names():
columns = {col['name'] for col in inspector.get_columns('profiles')}
if 'avatar_path' not in columns:
print("Migrating profiles: adding avatar_path column")
with engine.connect() as conn:
conn.execute(text("ALTER TABLE profiles ADD COLUMN avatar_path VARCHAR"))
conn.commit()
print("Added avatar_path column to profiles")
def get_db():
+40 -9
View File
@@ -75,6 +75,16 @@ def export_profile_to_zip(profile_id: str, db: Session) -> bytes:
zip_buffer = io.BytesIO()
with zipfile.ZipFile(zip_buffer, 'w', zipfile.ZIP_DEFLATED) as zip_file:
# Check if profile has avatar
has_avatar = False
if profile.avatar_path:
avatar_path = Path(profile.avatar_path)
if avatar_path.exists():
has_avatar = True
# Add avatar to ZIP root with original extension
avatar_ext = avatar_path.suffix
zip_file.write(avatar_path, f"avatar{avatar_ext}")
# Create manifest.json
manifest = {
"version": "1.0",
@@ -82,30 +92,31 @@ def export_profile_to_zip(profile_id: str, db: Session) -> bytes:
"name": profile.name,
"description": profile.description,
"language": profile.language,
}
},
"has_avatar": has_avatar,
}
zip_file.writestr("manifest.json", json.dumps(manifest, indent=2))
# Create samples.json mapping
samples_data = {}
profile_dir = _get_profiles_dir() / profile_id
for sample in samples:
# Get filename from audio_path (should be {sample_id}.wav)
audio_path = Path(sample.audio_path)
filename = audio_path.name
# Read audio file
if not audio_path.exists():
raise ValueError(f"Audio file not found: {audio_path}")
# Add to samples directory in ZIP
zip_path = f"samples/{filename}"
zip_file.write(audio_path, zip_path)
# Map filename to reference text
samples_data[filename] = sample.reference_text
zip_file.writestr("samples.json", json.dumps(samples_data, indent=2))
zip_buffer.seek(0)
@@ -168,11 +179,31 @@ async def import_profile_from_zip(file_bytes: bytes, db: Session) -> VoiceProfil
)
profile = await create_profile(profile_create, db)
# Extract and add samples
profile_dir = _get_profiles_dir() / profile.id
profile_dir.mkdir(parents=True, exist_ok=True)
# Handle avatar if present
avatar_files = [f for f in namelist if f.startswith("avatar.")]
if avatar_files:
try:
avatar_file = avatar_files[0]
# Extract to temporary file
import tempfile
with tempfile.NamedTemporaryFile(suffix=Path(avatar_file).suffix, delete=False) as tmp:
tmp.write(zip_file.read(avatar_file))
tmp_path = tmp.name
try:
from .profiles import upload_avatar
await upload_avatar(profile.id, tmp_path, db)
finally:
Path(tmp_path).unlink(missing_ok=True)
except Exception as e:
# Avatar import is optional - continue even if it fails
pass
for filename, reference_text in samples_data.items():
# Validate filename
if not filename.endswith('.wav'):
+690 -124
View File
File diff suppressed because it is too large Load Diff
+29
View File
@@ -20,6 +20,7 @@ class VoiceProfileResponse(BaseModel):
name: str
description: Optional[str]
language: str
avatar_path: Optional[str] = None
created_at: datetime
updated_at: datetime
@@ -32,6 +33,11 @@ class ProfileSampleCreate(BaseModel):
reference_text: str = Field(..., min_length=1, max_length=1000)
class ProfileSampleUpdate(BaseModel):
"""Request model for updating a profile sample."""
reference_text: str = Field(..., min_length=1, max_length=1000)
class ProfileSampleResponse(BaseModel):
"""Response model for profile sample."""
id: str
@@ -118,7 +124,9 @@ class HealthResponse(BaseModel):
model_downloaded: Optional[bool] = None # Whether model is cached/downloaded
model_size: Optional[str] = None # Current model size if loaded
gpu_available: bool
gpu_type: Optional[str] = None # GPU type (CUDA, MPS, or None)
vram_used_mb: Optional[float] = None
backend_type: Optional[str] = None # Backend type (mlx or pytorch)
class ModelStatus(BaseModel):
@@ -126,6 +134,7 @@ class ModelStatus(BaseModel):
model_name: str
display_name: str
downloaded: bool
downloading: bool = False # True if download is in progress
size_mb: Optional[float] = None
loaded: bool = False
@@ -161,6 +170,13 @@ class ActiveTasksResponse(BaseModel):
generations: List[ActiveGenerationTask]
class FolderPathsResponse(BaseModel):
"""Response model for system folder paths."""
data_dir: str
models_dir: str
providers_dir: str
class AudioChannelCreate(BaseModel):
"""Request model for creating an audio channel."""
name: str = Field(..., min_length=1, max_length=100)
@@ -221,6 +237,8 @@ class StoryItemDetail(BaseModel):
generation_id: str
start_time_ms: int
track: int = 0
trim_start_ms: int = 0
trim_end_ms: int = 0
created_at: datetime
# Generation details
profile_id: str
@@ -277,3 +295,14 @@ class StoryItemMove(BaseModel):
"""Request model for moving a story item (position and/or track)."""
start_time_ms: int = Field(..., ge=0)
track: int = 0
class StoryItemTrim(BaseModel):
"""Request model for trimming a story item."""
trim_start_ms: int = Field(..., ge=0)
trim_end_ms: int = Field(..., ge=0)
class StoryItemSplit(BaseModel):
"""Request model for splitting a story item."""
split_time_ms: int = Field(..., ge=0) # Time within the clip to split at (relative to clip start)
+33
View File
@@ -0,0 +1,33 @@
"""
Platform detection for backend selection.
"""
import platform
from typing import Literal
def is_apple_silicon() -> bool:
"""
Check if running on Apple Silicon (arm64 macOS).
Returns:
True if on Apple Silicon, False otherwise
"""
return platform.system() == "Darwin" and platform.machine() == "arm64"
def get_backend_type() -> Literal["mlx", "pytorch"]:
"""
Detect the best backend for the current platform.
Returns:
"mlx" on Apple Silicon (if MLX is available), "pytorch" otherwise
"""
if is_apple_silicon():
try:
import mlx
return "mlx"
except ImportError:
# MLX not installed, fallback to PyTorch
return "pytorch"
return "pytorch"
+172 -22
View File
@@ -21,6 +21,8 @@ from .database import (
ProfileSample as DBProfileSample,
)
from .utils.audio import validate_reference_audio, load_audio, save_audio
from .utils.images import validate_image, process_avatar
from .utils.cache import _get_cache_dir, clear_profile_cache
from .tts import get_tts_model
from . import config
@@ -119,6 +121,10 @@ async def add_profile_sample(
db.commit()
db.refresh(db_sample)
# Invalidate combined audio cache for this profile
# Since a new sample was added, any cached combined audio is now stale
clear_profile_cache(profile_id)
return ProfileSampleResponse.model_validate(db_sample)
@@ -240,6 +246,9 @@ async def delete_profile(
if profile_dir.exists():
shutil.rmtree(profile_dir)
# Clean up combined audio cache files for this profile
clear_profile_cache(profile_id)
return True
@@ -261,6 +270,9 @@ async def delete_profile_sample(
if not sample:
return False
# Store profile_id before deleting
profile_id = sample.profile_id
# Delete audio file
audio_path = Path(sample.audio_path)
if audio_path.exists():
@@ -270,9 +282,47 @@ async def delete_profile_sample(
db.delete(sample)
db.commit()
# Invalidate combined audio cache for this profile
# Since the sample set changed, any cached combined audio is now stale
clear_profile_cache(profile_id)
return True
async def update_profile_sample(
sample_id: str,
reference_text: str,
db: Session,
) -> Optional[ProfileSampleResponse]:
"""
Update a profile sample's reference text.
Args:
sample_id: Sample ID
reference_text: Updated reference text
db: Database session
Returns:
Updated sample or None if not found
"""
sample = db.query(DBProfileSample).filter_by(id=sample_id).first()
if not sample:
return None
# Store profile_id before updating
profile_id = sample.profile_id
sample.reference_text = reference_text
db.commit()
db.refresh(sample)
# Invalidate combined audio cache for this profile
# Since the reference text changed, cache keys and combined text are now stale
clear_profile_cache(profile_id)
return ProfileSampleResponse.model_validate(sample)
async def create_voice_prompt_for_profile(
profile_id: str,
db: Session,
@@ -280,23 +330,23 @@ async def create_voice_prompt_for_profile(
) -> dict:
"""
Create a combined voice prompt from all samples in a profile.
Args:
profile_id: Profile ID
db: Database session
use_cache: Whether to use cached prompts
Returns:
Voice prompt dictionary
"""
# Get all samples for profile
samples = db.query(DBProfileSample).filter_by(profile_id=profile_id).all()
if not samples:
raise ValueError(f"No samples found for profile {profile_id}")
tts_model = get_tts_model()
if len(samples) == 1:
# Single sample - use directly
sample = samples[0]
@@ -310,27 +360,127 @@ async def create_voice_prompt_for_profile(
# Multiple samples - combine them
audio_paths = [s.audio_path for s in samples]
reference_texts = [s.reference_text for s in samples]
# Combine audio
combined_audio, combined_text = await tts_model.combine_voice_prompts(
audio_paths,
reference_texts,
)
# Save combined audio to cache directory (persistent)
# Create a hash of sample IDs to identify this specific combination
import hashlib
sample_ids_str = "-".join(sorted([s.id for s in samples]))
combination_hash = hashlib.md5(sample_ids_str.encode()).hexdigest()[:12]
# Save combined audio temporarily
import tempfile
with tempfile.NamedTemporaryFile(suffix=".wav", delete=False) as tmp:
save_audio(combined_audio, tmp.name, 24000)
tmp_path = tmp.name
# Store in cache directory
cache_dir = _get_cache_dir()
cache_dir.mkdir(parents=True, exist_ok=True)
combined_path = cache_dir / f"combined_{profile_id}_{combination_hash}.wav"
try:
# Create prompt from combined audio
voice_prompt, _ = await tts_model.create_voice_prompt(
tmp_path,
combined_text,
use_cache=use_cache,
)
return voice_prompt
finally:
# Clean up temp file
Path(tmp_path).unlink(missing_ok=True)
# Save combined audio
save_audio(combined_audio, str(combined_path), 24000)
# Create prompt from combined audio
voice_prompt, _ = await tts_model.create_voice_prompt(
str(combined_path),
combined_text,
use_cache=use_cache,
)
return voice_prompt
async def upload_avatar(
profile_id: str,
image_path: str,
db: Session,
) -> VoiceProfileResponse:
"""
Upload and process avatar image for a profile.
Args:
profile_id: Profile ID
image_path: Path to uploaded image file
db: Database session
Returns:
Updated profile
"""
# Validate profile exists
profile = db.query(DBVoiceProfile).filter_by(id=profile_id).first()
if not profile:
raise ValueError(f"Profile {profile_id} not found")
# Validate image
is_valid, error_msg = validate_image(image_path)
if not is_valid:
raise ValueError(error_msg)
# Delete existing avatar if present
if profile.avatar_path:
old_avatar = Path(profile.avatar_path)
if old_avatar.exists():
old_avatar.unlink()
# Determine file extension from uploaded file
from PIL import Image
with Image.open(image_path) as img:
# Normalize JPEG variants (MPO is multi-picture format from some cameras)
img_format = img.format
if img_format in ('MPO', 'JPG'):
img_format = 'JPEG'
ext_map = {
'PNG': '.png',
'JPEG': '.jpg',
'WEBP': '.webp'
}
ext = ext_map.get(img_format, '.png')
# Save processed image to profile directory
profile_dir = _get_profiles_dir() / profile_id
profile_dir.mkdir(parents=True, exist_ok=True)
output_path = profile_dir / f"avatar{ext}"
process_avatar(image_path, str(output_path))
# Update database
profile.avatar_path = str(output_path)
profile.updated_at = datetime.utcnow()
db.commit()
db.refresh(profile)
return VoiceProfileResponse.model_validate(profile)
async def delete_avatar(
profile_id: str,
db: Session,
) -> bool:
"""
Delete avatar image for a profile.
Args:
profile_id: Profile ID
db: Database session
Returns:
True if deleted, False if not found or no avatar
"""
profile = db.query(DBVoiceProfile).filter_by(id=profile_id).first()
if not profile or not profile.avatar_path:
return False
# Delete avatar file
avatar_path = Path(profile.avatar_path)
if avatar_path.exists():
avatar_path.unlink()
# Update database
profile.avatar_path = None
profile.updated_at = datetime.utcnow()
db.commit()
return True
+327
View File
@@ -0,0 +1,327 @@
"""
Provider management system for TTS providers.
"""
from typing import Optional
import asyncio
import platform
from pathlib import Path
from .base import TTSProvider
from .types import ProviderType
from .bundled import BundledProvider
from .local import LocalProvider
from .installer import get_provider_binary_path, _get_providers_dir
from ..config import get_data_dir
import subprocess
import socket
class ProviderManager:
"""Manages TTS provider lifecycle."""
def __init__(self):
self.active_provider: Optional[TTSProvider] = None
self._default_provider: Optional[TTSProvider] = None
self._provider_process: Optional[subprocess.Popen] = None
self._provider_port: Optional[int] = None
def _get_default_provider(self) -> TTSProvider:
"""Get the default bundled provider."""
if self._default_provider is None:
self._default_provider = BundledProvider()
return self._default_provider
async def get_active_provider(self) -> TTSProvider:
"""
Get the currently active provider.
Returns:
Active TTS provider instance
"""
if self.active_provider is None:
# Default to bundled provider
self.active_provider = self._get_default_provider()
return self.active_provider
async def start_provider(self, provider_type: str) -> None:
"""
Start a TTS provider.
Args:
provider_type: Type of provider to start
"""
if provider_type == "apple-mlx":
# Use bundled MLX provider
self.active_provider = self._get_default_provider()
elif provider_type in ["pytorch-cpu", "pytorch-cuda"]:
# Try to start external provider subprocess if binary exists
provider_path = get_provider_binary_path(provider_type)
if provider_path and provider_path.exists():
# External downloaded provider exists, start it
# Find a free port
port = self._get_free_port()
# Start provider subprocess with stdout/stderr capture
from ..config import get_data_dir
import logging
logger = logging.getLogger(__name__)
logger.info(f"Starting provider {provider_type} on port {port}")
logger.info(f"Provider binary: {provider_path}")
logger.info(f"Data directory: {get_data_dir()}")
# Create log files for provider output (easier debugging on Windows)
logs_dir = get_data_dir() / "logs"
logs_dir.mkdir(exist_ok=True)
stdout_log = logs_dir / f"{provider_type}-stdout.log"
stderr_log = logs_dir / f"{provider_type}-stderr.log"
logger.info(f"Provider logs will be written to: {logs_dir}")
process = subprocess.Popen(
[
str(provider_path),
"--port", str(port),
"--data-dir", str(get_data_dir()),
],
stdout=open(stdout_log, 'w'),
stderr=open(stderr_log, 'w'),
text=True,
bufsize=1,
)
# Wait for provider to be ready
base_url = f"http://127.0.0.1:{port}"
try:
await self._wait_for_provider_health(base_url, timeout=30)
except TimeoutError as e:
# Read log files for debugging (works on all platforms unlike select)
stdout_content = ""
stderr_content = ""
# Try to read available output (works on Windows and Unix)
try:
# Use non-blocking read with timeout
import threading
import queue
def enqueue_output(stream, queue):
try:
for line in iter(stream.readline, ''):
queue.put(line)
except:
pass
stdout_queue = queue.Queue()
stderr_queue = queue.Queue()
if process.stdout:
t = threading.Thread(target=enqueue_output, args=(process.stdout, stdout_queue))
t.daemon = True
t.start()
if process.stderr:
t2 = threading.Thread(target=enqueue_output, args=(process.stderr, stderr_queue))
t2.daemon = True
t2.start()
# Give threads a moment to read
import time
time.sleep(0.5)
# Collect output
while not stdout_queue.empty():
stdout_lines.append(stdout_queue.get_nowait())
while not stderr_queue.empty():
stderr_lines.append(stderr_queue.get_nowait())
except Exception as ex:
logger.warning(f"Could not capture subprocess output: {ex}")
logger.error(f"Provider failed to start within 30 seconds")
logger.error(f"Check logs at: {logs_dir}")
if stdout_content:
logger.error(f"Stdout: {stdout_content[-2000:]}") # Last 2000 chars
if stderr_content:
logger.error(f"Stderr: {stderr_content[-2000:]}") # Last 2000 chars
# Terminate the process
process.terminate()
try:
process.wait(timeout=5)
except subprocess.TimeoutExpired:
process.kill()
# Raise with log file location for user
raise TimeoutError(
f"Provider {provider_type} failed to start. Check logs at: {logs_dir}"
)
# Create LocalProvider instance
self.active_provider = LocalProvider(base_url)
self._provider_process = process
self._provider_port = port
# Logs are written directly to files (stdout_log, stderr_log)
# No need for background task - users can check {logs_dir} for debugging
else:
# No external binary, use bundled provider (if available)
if provider_type == "pytorch-cpu":
# PyTorch CPU can use bundled backend
self.active_provider = self._get_default_provider()
else:
raise ValueError(f"Provider {provider_type} is not installed. Please download it first.")
elif provider_type == "remote":
# Remote provider - will be implemented in Phase 5
raise NotImplementedError("Remote provider not yet implemented")
elif provider_type == "openai":
# OpenAI provider - will be implemented in Phase 5
raise NotImplementedError("OpenAI provider not yet implemented")
else:
raise ValueError(f"Unknown provider type: {provider_type}")
async def stop_provider(self) -> None:
"""Stop the active provider."""
if self.active_provider:
# Only stop if it's not the default bundled provider
if self.active_provider is not self._default_provider:
if hasattr(self.active_provider, 'stop'):
await self.active_provider.stop()
self.active_provider = None
# Stop subprocess if running
if self._provider_process:
self._provider_process.terminate()
try:
self._provider_process.wait(timeout=5)
except subprocess.TimeoutExpired:
self._provider_process.kill()
self._provider_process = None
self._provider_port = None
async def list_installed(self) -> list[str]:
"""
List installed provider types.
Returns:
List of installed provider type strings
"""
installed = []
# Bundled providers are always available
system = platform.system()
machine = platform.machine()
if system == "Darwin" and machine == "arm64":
# Apple Silicon gets MLX bundled
installed.append("apple-mlx")
elif system == "Windows" or (system == "Darwin" and machine != "arm64"):
# Windows and Intel macOS get PyTorch CPU bundled
installed.append("pytorch-cpu")
# Linux: no bundled provider - users must download
# Check for downloaded providers by checking if binary path exists
for provider_type in ["pytorch-cpu", "pytorch-cuda"]:
binary_path = get_provider_binary_path(provider_type)
if binary_path and binary_path.exists() and provider_type not in installed:
installed.append(provider_type)
return installed
async def get_provider_info(self, provider_type: str) -> dict:
"""
Get information about a provider.
Args:
provider_type: Type of provider
Returns:
Provider information dictionary
"""
if provider_type in ["apple-mlx", "bundled-pytorch"]:
return {
"type": provider_type,
"name": "Bundled Provider",
"installed": True,
"size_mb": None, # Bundled, no separate size
}
elif provider_type == "pytorch-cpu":
return {
"type": provider_type,
"name": "PyTorch CPU",
"installed": provider_type in await self.list_installed(),
"size_mb": 300,
}
elif provider_type == "pytorch-cuda":
return {
"type": provider_type,
"name": "PyTorch CUDA",
"installed": provider_type in await self.list_installed(),
"size_mb": 2400,
}
else:
return {
"type": provider_type,
"name": provider_type,
"installed": False,
"size_mb": None,
}
def _get_free_port(self) -> int:
"""Get a free port for the provider server."""
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
s.bind(('', 0))
return s.getsockname()[1]
async def _wait_for_provider_health(self, base_url: str, timeout: int = 30) -> None:
"""Wait for provider to become healthy."""
import httpx
import asyncio
start_time = asyncio.get_event_loop().time()
while True:
try:
async with httpx.AsyncClient(timeout=2.0) as client:
response = await client.get(f"{base_url}/tts/health")
if response.status_code == 200:
return
except Exception:
pass
if asyncio.get_event_loop().time() - start_time > timeout:
raise TimeoutError(f"Provider did not become healthy within {timeout} seconds")
await asyncio.sleep(0.5)
async def _log_subprocess_output(self, process: subprocess.Popen) -> None:
"""Log subprocess stdout and stderr."""
import logging
logger = logging.getLogger(__name__)
async def read_stream(stream, prefix):
if stream:
loop = asyncio.get_event_loop()
while True:
line = await loop.run_in_executor(None, stream.readline)
if not line:
break
logger.info(f"{prefix}: {line.rstrip()}")
await asyncio.gather(
read_stream(process.stdout, "Provider stdout"),
read_stream(process.stderr, "Provider stderr"),
return_exceptions=True,
)
# Global provider manager instance
_provider_manager: Optional[ProviderManager] = None
def get_provider_manager() -> ProviderManager:
"""Get the global provider manager instance."""
global _provider_manager
if _provider_manager is None:
_provider_manager = ProviderManager()
return _provider_manager
+97
View File
@@ -0,0 +1,97 @@
"""
Base protocol for TTS providers.
"""
from typing import Protocol, Optional, Tuple
from typing_extensions import runtime_checkable
import numpy as np
from .types import ProviderHealth, ProviderStatus
@runtime_checkable
class TTSProvider(Protocol):
"""Protocol for TTS provider implementations."""
async def generate(
self,
text: str,
voice_prompt: dict,
language: str = "en",
seed: Optional[int] = None,
instruct: Optional[str] = None,
) -> Tuple[np.ndarray, int]:
"""
Generate speech audio from text.
Args:
text: Text to synthesize
voice_prompt: Voice prompt dictionary
language: Language code
seed: Random seed for reproducibility
instruct: Delivery instructions
Returns:
Tuple of (audio_array, sample_rate)
"""
...
async def create_voice_prompt(
self,
audio_path: str,
reference_text: str,
use_cache: bool = True,
) -> Tuple[dict, bool]:
"""
Create voice prompt from reference audio.
Args:
audio_path: Path to reference audio file
reference_text: Transcript of the audio
use_cache: Whether to use cached prompts
Returns:
Tuple of (voice_prompt_dict, was_cached)
"""
...
async def combine_voice_prompts(
self,
audio_paths: list[str],
reference_texts: list[str],
) -> Tuple[np.ndarray, str]:
"""
Combine multiple voice prompts.
Args:
audio_paths: List of audio file paths
reference_texts: List of reference texts
Returns:
Tuple of (combined_audio_array, combined_text)
"""
...
async def load_model_async(self, model_size: str) -> None:
"""Load TTS model."""
...
def unload_model(self) -> None:
"""Unload model to free memory."""
...
def is_loaded(self) -> bool:
"""Check if model is loaded."""
...
def _get_model_path(self, model_size: str) -> str:
"""Get model path for a given size."""
...
async def health(self) -> ProviderHealth:
"""Get provider health status."""
...
async def status(self) -> ProviderStatus:
"""Get provider model status."""
...
+144
View File
@@ -0,0 +1,144 @@
"""
Bundled provider that wraps existing MLX/PyTorch backends.
"""
from typing import Optional, Tuple
import numpy as np
import platform
from .base import TTSProvider
from .types import ProviderHealth, ProviderStatus
from ..backends import get_tts_backend, TTSBackend
from ..platform_detect import get_backend_type
class BundledProvider:
"""Provider that wraps the existing bundled TTS backend."""
def __init__(self):
self._backend: Optional[TTSBackend] = None
def _get_backend(self) -> TTSBackend:
"""Get or create backend instance."""
if self._backend is None:
self._backend = get_tts_backend()
return self._backend
async def generate(
self,
text: str,
voice_prompt: dict,
language: str = "en",
seed: Optional[int] = None,
instruct: Optional[str] = None,
) -> Tuple[np.ndarray, int]:
"""Generate speech audio."""
backend = self._get_backend()
return await backend.generate(text, voice_prompt, language, seed, instruct)
async def create_voice_prompt(
self,
audio_path: str,
reference_text: str,
use_cache: bool = True,
) -> Tuple[dict, bool]:
"""Create voice prompt from reference audio."""
backend = self._get_backend()
return await backend.create_voice_prompt(audio_path, reference_text, use_cache)
async def combine_voice_prompts(
self,
audio_paths: list[str],
reference_texts: list[str],
) -> Tuple[np.ndarray, str]:
"""Combine multiple voice prompts."""
backend = self._get_backend()
return await backend.combine_voice_prompts(audio_paths, reference_texts)
async def load_model_async(self, model_size: str) -> None:
"""Load TTS model."""
backend = self._get_backend()
if hasattr(backend, 'load_model_async'):
await backend.load_model_async(model_size)
else:
await backend.load_model(model_size)
# Alias for compatibility
load_model = load_model_async
def unload_model(self) -> None:
"""Unload model to free memory."""
backend = self._get_backend()
backend.unload_model()
def is_loaded(self) -> bool:
"""Check if model is loaded."""
backend = self._get_backend()
return backend.is_loaded()
def _get_model_path(self, model_size: str) -> str:
"""Get model path for a given size."""
backend = self._get_backend()
return backend._get_model_path(model_size)
async def health(self) -> ProviderHealth:
"""Get provider health status."""
backend = self._get_backend()
backend_type = get_backend_type()
model_size = None
if backend.is_loaded():
# Try to get current model size from backend
if hasattr(backend, '_current_model_size') and backend._current_model_size:
model_size = backend._current_model_size
device = None
if backend_type == "mlx":
device = "metal"
elif hasattr(backend, 'device'):
device = backend.device
# Use apple-mlx for MLX backend, pytorch-cpu for PyTorch
provider_name = "apple-mlx" if backend_type == "mlx" else "pytorch-cpu"
return ProviderHealth(
status="healthy",
provider=provider_name,
version=None, # Provider versioning not implemented yet
model=model_size,
device=device,
)
async def status(self) -> ProviderStatus:
"""Get provider model status."""
backend = self._get_backend()
backend_type = get_backend_type()
model_size = None
if backend.is_loaded():
if hasattr(backend, '_current_model_size') and backend._current_model_size:
model_size = backend._current_model_size
available_sizes = ["1.7B"]
if backend_type == "pytorch":
available_sizes.append("0.6B")
gpu_available = None
vram_used_mb = None
if backend_type == "pytorch":
try:
import torch
gpu_available = torch.cuda.is_available()
if gpu_available:
vram_used_mb = torch.cuda.memory_allocated() / 1024 / 1024
except ImportError:
pass
return ProviderStatus(
model_loaded=backend.is_loaded(),
model_size=model_size,
available_sizes=available_sizes,
gpu_available=gpu_available,
vram_used_mb=int(vram_used_mb) if vram_used_mb else None,
)
+11
View File
@@ -0,0 +1,11 @@
# Provider checksums - embedded at build time for security
# This file is auto-generated during CI builds
# In development, checksums are empty (verification is skipped)
PROVIDER_CHECKSUMS = {
# Populated during release builds with SHA256 checksums of provider binaries
# Example:
# "tts-provider-pytorch-cpu-windows.exe": "abc123...",
# "tts-provider-pytorch-cuda-windows.exe": "def456...",
# "tts-provider-pytorch-cuda-linux": "789xyz...",
}
+262
View File
@@ -0,0 +1,262 @@
"""
Provider download and installation manager.
"""
import asyncio
import httpx
import platform
from pathlib import Path
from typing import Optional
from .types import ProviderType
from ..utils.progress import get_progress_manager
from ..utils.tasks import get_task_manager
# Provider version (independent of app version)
PROVIDER_VERSION = "1.0.0"
# Base URL for provider downloads (Cloudflare R2)
PROVIDER_DOWNLOAD_BASE_URL = "https://downloads.voicebox.sh/providers"
def _get_providers_dir() -> Path:
"""Get the directory where providers are stored."""
system = platform.system()
if system == "Windows":
appdata = Path.home() / "AppData" / "Roaming"
elif system == "Darwin":
appdata = Path.home() / "Library" / "Application Support"
else: # Linux
appdata = Path.home() / ".local" / "share"
providers_dir = appdata / "voicebox" / "providers"
providers_dir.mkdir(parents=True, exist_ok=True)
return providers_dir
def _get_provider_binary_name(provider_type: str) -> str:
"""Get the local binary filename for a provider type."""
system = platform.system()
ext = ".exe" if system == "Windows" else ""
binary_map = {
"pytorch-cpu": f"tts-provider-pytorch-cpu{ext}",
"pytorch-cuda": f"tts-provider-pytorch-cuda{ext}",
}
if provider_type not in binary_map:
raise ValueError(f"Unknown provider type: {provider_type}")
return binary_map[provider_type]
def _get_provider_download_name(provider_type: str) -> str:
"""Get the remote download filename for a provider type (includes platform suffix)."""
system = platform.system()
if system == "Windows":
platform_suffix = "windows"
ext = ".zip"
elif system == "Linux":
platform_suffix = "linux"
ext = ".tar.gz"
elif system == "Darwin":
# Detect macOS architecture
machine = platform.machine()
if machine == "arm64":
platform_suffix = "macos-arm64"
else:
platform_suffix = "macos-x64"
ext = ".tar.gz"
else:
raise ValueError(f"Provider downloads not supported on {system}")
return f"tts-provider-{provider_type}-{platform_suffix}{ext}"
def _get_provider_download_url(provider_type: str) -> str:
"""Get the download URL for a provider."""
download_name = _get_provider_download_name(provider_type)
return f"{PROVIDER_DOWNLOAD_BASE_URL}/v{PROVIDER_VERSION}/{download_name}"
async def download_provider(provider_type: str) -> Path:
"""
Download and extract a provider archive from Cloudflare R2.
Args:
provider_type: Type of provider to download (e.g., "pytorch-cpu")
Returns:
Path to the extracted provider binary
Raises:
ValueError: If provider_type is invalid
httpx.HTTPError: If download fails
"""
if provider_type not in ["pytorch-cpu", "pytorch-cuda"]:
raise ValueError(f"Provider type {provider_type} cannot be downloaded")
progress_manager = get_progress_manager()
task_manager = get_task_manager()
archive_name = _get_provider_download_name(provider_type)
download_url = _get_provider_download_url(provider_type)
providers_dir = _get_providers_dir()
archive_path = providers_dir / archive_name
# Start tracking download
task_manager.start_download(provider_type)
# Initialize progress state
progress_manager.update_progress(
model_name=provider_type,
current=0,
total=0, # Will be updated once we get Content-Length
filename=archive_name,
status="downloading",
)
try:
# Download archive
async with httpx.AsyncClient(timeout=300.0) as client:
async with client.stream("GET", download_url) as response:
response.raise_for_status()
# Get total size from Content-Length header
total_size = int(response.headers.get("Content-Length", 0))
if total_size > 0:
progress_manager.update_progress(
model_name=provider_type,
current=0,
total=total_size,
filename=archive_name,
status="downloading",
)
# Download with progress tracking
downloaded = 0
with open(archive_path, "wb") as f:
async for chunk in response.aiter_bytes(chunk_size=8192):
f.write(chunk)
downloaded += len(chunk)
# Update progress
progress_manager.update_progress(
model_name=provider_type,
current=downloaded,
total=total_size if total_size > 0 else downloaded,
filename=archive_name,
status="downloading",
)
# Extract archive
progress_manager.update_progress(
model_name=provider_type,
current=downloaded,
total=downloaded,
filename="Extracting...",
status="downloading",
)
import zipfile
import tarfile
if archive_name.endswith('.zip'):
with zipfile.ZipFile(archive_path, 'r') as zip_ref:
zip_ref.extractall(providers_dir)
elif archive_name.endswith('.tar.gz'):
with tarfile.open(archive_path, 'r:gz') as tar_ref:
tar_ref.extractall(providers_dir)
else:
raise ValueError(f"Unsupported archive format: {archive_name}")
# Remove archive after extraction
archive_path.unlink()
# Get path to extracted binary
binary_path = get_provider_binary_path(provider_type)
if not binary_path:
raise ValueError(f"Provider binary not found after extraction")
# Make executable on Unix systems
if platform.system() != "Windows":
binary_path.chmod(0o755)
# Mark as complete
progress_manager.update_progress(
model_name=provider_type,
current=downloaded,
total=downloaded,
filename=_get_provider_binary_name(provider_type),
status="complete",
)
task_manager.complete_download(provider_type)
return binary_path
except Exception as e:
# Clean up archive if it exists
if archive_path.exists():
archive_path.unlink()
# Mark as error
progress_manager.update_progress(
model_name=provider_type,
current=0,
total=0,
filename=archive_name,
status="error",
)
task_manager.error_download(provider_type, str(e))
raise
def get_provider_binary_path(provider_type: str) -> Optional[Path]:
"""
Get the path to an installed provider binary.
Args:
provider_type: Type of provider
Returns:
Path to provider binary, or None if not installed
"""
providers_dir = _get_providers_dir()
binary_name = _get_provider_binary_name(provider_type)
# Check for --onedir structure (directory with binary inside)
provider_dir = providers_dir / f"tts-provider-{provider_type}"
if provider_dir.exists() and provider_dir.is_dir():
binary_path = provider_dir / binary_name
if binary_path.exists() and binary_path.is_file():
return binary_path
# Fallback: check for direct binary (legacy)
provider_path = providers_dir / binary_name
if provider_path.exists() and provider_path.is_file():
return provider_path
return None
def delete_provider(provider_type: str) -> bool:
"""
Delete an installed provider binary.
Args:
provider_type: Type of provider to delete
Returns:
True if deleted, False if not found
"""
provider_path = get_provider_binary_path(provider_type)
if provider_path and provider_path.exists():
provider_path.unlink()
return True
return False
+191
View File
@@ -0,0 +1,191 @@
"""
Local provider that communicates with standalone provider servers via HTTP.
"""
from typing import Optional, Tuple
import base64
import io
import numpy as np
import httpx
import soundfile as sf
from .base import TTSProvider
from .types import ProviderHealth, ProviderStatus
class LocalProvider:
"""Provider that communicates with local subprocess via HTTP."""
def __init__(self, base_url: str):
"""
Initialize local provider.
Args:
base_url: Base URL of the provider server (e.g., "http://localhost:8000")
"""
self.base_url = base_url.rstrip('/')
self.client = httpx.AsyncClient(timeout=300.0) # 5 minute timeout for generation
self._current_model_size = "1.7B" # Default model size
async def generate(
self,
text: str,
voice_prompt: dict,
language: str = "en",
seed: Optional[int] = None,
instruct: Optional[str] = None,
) -> Tuple[np.ndarray, int]:
"""Generate speech audio."""
response = await self.client.post(
f"{self.base_url}/tts/generate",
json={
"text": text,
"voice_prompt": voice_prompt,
"language": language,
"seed": seed,
"model_size": self._current_model_size,
}
)
response.raise_for_status()
data = response.json()
# Decode base64 audio
audio_bytes = base64.b64decode(data["audio"])
audio_buffer = io.BytesIO(audio_bytes)
audio, sample_rate = sf.read(audio_buffer)
return audio, data["sample_rate"]
async def create_voice_prompt(
self,
audio_path: str,
reference_text: str,
use_cache: bool = True,
) -> Tuple[dict, bool]:
"""Create voice prompt from reference audio."""
# Read audio file
with open(audio_path, 'rb') as f:
audio_data = f.read()
# Send multipart form data
files = {
"audio": ("audio.wav", audio_data, "audio/wav")
}
data = {
"reference_text": reference_text,
"use_cache": str(use_cache).lower(),
}
response = await self.client.post(
f"{self.base_url}/tts/create_voice_prompt",
files=files,
data=data,
)
response.raise_for_status()
result = response.json()
return result["voice_prompt"], result.get("was_cached", False)
async def combine_voice_prompts(
self,
audio_paths: list[str],
reference_texts: list[str],
) -> Tuple[np.ndarray, str]:
"""
Combine multiple voice prompts.
Note: This is not implemented in the provider API yet.
For now, we'll combine locally by concatenating audio.
"""
import numpy as np
from ..utils.audio import load_audio, normalize_audio
combined_audio = []
for audio_path in audio_paths:
audio, sr = load_audio(audio_path)
audio = normalize_audio(audio)
combined_audio.append(audio)
# Concatenate audio
mixed = np.concatenate(combined_audio)
mixed = normalize_audio(mixed)
# Combine texts
combined_text = " ".join(reference_texts)
return mixed, combined_text
async def load_model_async(self, model_size: str) -> None:
"""Load TTS model."""
# Track the requested model size - the provider server will load it
# when generate() is called with this size
self._current_model_size = model_size
# Alias for compatibility
load_model = load_model_async
def unload_model(self) -> None:
"""Unload model to free memory."""
# Model unloading is handled by the provider server
# This is a no-op for local providers
pass
def is_loaded(self) -> bool:
"""Check if model is loaded."""
# We can't know this without querying the provider
# Return True optimistically
return True
def _get_model_path(self, model_size: str) -> str:
"""Get model path for a given size."""
# For local providers, model paths are handled by the provider server
# Return a placeholder
return f"Qwen/Qwen3-TTS-12Hz-{model_size}-Base"
async def health(self) -> ProviderHealth:
"""Get provider health status."""
try:
response = await self.client.get(f"{self.base_url}/tts/health")
response.raise_for_status()
data = response.json()
return ProviderHealth(
status=data["status"],
provider=data["provider"],
version=data.get("version"),
model=data.get("model"),
device=data.get("device"),
)
except Exception as e:
return ProviderHealth(
status="unhealthy",
provider="local",
version=None,
model=None,
device=None,
)
async def status(self) -> ProviderStatus:
"""Get provider model status."""
try:
response = await self.client.get(f"{self.base_url}/tts/status")
response.raise_for_status()
data = response.json()
return ProviderStatus(
model_loaded=data["model_loaded"],
model_size=data.get("model_size"),
available_sizes=data.get("available_sizes", []),
gpu_available=data.get("gpu_available"),
vram_used_mb=data.get("vram_used_mb"),
)
except Exception as e:
return ProviderStatus(
model_loaded=False,
model_size=None,
available_sizes=[],
gpu_available=None,
vram_used_mb=None,
)
async def stop(self) -> None:
"""Stop the provider (close HTTP client)."""
await self.client.aclose()
+34
View File
@@ -0,0 +1,34 @@
"""
Shared types for TTS providers.
"""
from typing import Optional, TypedDict
from enum import Enum
class ProviderType(str, Enum):
"""Available provider types."""
BUNDLED_MLX = "apple-mlx"
BUNDLED_PYTORCH = "bundled-pytorch"
PYTORCH_CPU = "pytorch-cpu"
PYTORCH_CUDA = "pytorch-cuda"
REMOTE = "remote"
OPENAI = "openai"
class ProviderHealth(TypedDict):
"""Provider health status."""
status: str # "healthy", "unhealthy", "starting"
provider: str
version: Optional[str]
model: Optional[str]
device: Optional[str]
class ProviderStatus(TypedDict):
"""Provider model status."""
model_loaded: bool
model_size: Optional[str]
available_sizes: list[str]
gpu_available: Optional[bool]
vram_used_mb: Optional[int]
+5
View File
@@ -0,0 +1,5 @@
# MLX-specific dependencies (Apple Silicon only)
# These should only be installed on aarch64-apple-darwin platforms
mlx>=0.30.0
mlx-audio>=0.3.1
+1
View File
@@ -21,3 +21,4 @@ numpy>=1.24.0
# Utilities
python-multipart>=0.0.6
Pillow>=10.0.0
+311 -11
View File
@@ -18,6 +18,8 @@ from .models import (
StoryItemCreate,
StoryItemBatchUpdate,
StoryItemMove,
StoryItemTrim,
StoryItemSplit,
)
from .database import Story as DBStory, StoryItem as DBStoryItem, Generation as DBGeneration, VoiceProfile as DBVoiceProfile
from .utils.audio import load_audio, save_audio
@@ -129,6 +131,8 @@ async def get_story(
generation_id=item.generation_id,
start_time_ms=item.start_time_ms,
track=item.track,
trim_start_ms=getattr(item, 'trim_start_ms', 0),
trim_end_ms=getattr(item, 'trim_end_ms', 0),
created_at=item.created_at,
profile_id=generation.profile_id,
profile_name=profile_name,
@@ -252,6 +256,8 @@ async def add_item_to_story(
generation_id=existing.generation_id,
start_time_ms=existing.start_time_ms,
track=existing.track,
trim_start_ms=getattr(existing, 'trim_start_ms', 0),
trim_end_ms=getattr(existing, 'trim_end_ms', 0),
created_at=existing.created_at,
profile_id=generation.profile_id,
profile_name=profile.name if profile else "Unknown",
@@ -321,6 +327,8 @@ async def add_item_to_story(
generation_id=item.generation_id,
start_time_ms=item.start_time_ms,
track=item.track,
trim_start_ms=getattr(item, 'trim_start_ms', 0),
trim_end_ms=getattr(item, 'trim_end_ms', 0),
created_at=item.created_at,
profile_id=generation.profile_id,
profile_name=profile.name if profile else "Unknown",
@@ -336,7 +344,7 @@ async def add_item_to_story(
async def move_story_item(
story_id: str,
generation_id: str,
item_id: str,
data: StoryItemMove,
db: Session,
) -> Optional[StoryItemDetail]:
@@ -345,7 +353,7 @@ async def move_story_item(
Args:
story_id: Story ID
generation_id: Generation ID of the item to move
item_id: Story item ID
data: New position and track data
db: Database session
@@ -354,14 +362,14 @@ async def move_story_item(
"""
# Get the item
item = db.query(DBStoryItem).filter_by(
id=item_id,
story_id=story_id,
generation_id=generation_id
).first()
if not item:
return None
# Get the generation
generation = db.query(DBGeneration).filter_by(id=generation_id).first()
generation = db.query(DBGeneration).filter_by(id=item.generation_id).first()
if not generation:
return None
@@ -386,6 +394,8 @@ async def move_story_item(
generation_id=item.generation_id,
start_time_ms=item.start_time_ms,
track=item.track,
trim_start_ms=getattr(item, 'trim_start_ms', 0),
trim_end_ms=getattr(item, 'trim_end_ms', 0),
created_at=item.created_at,
profile_id=generation.profile_id,
profile_name=profile.name if profile else "Unknown",
@@ -401,23 +411,23 @@ async def move_story_item(
async def remove_item_from_story(
story_id: str,
generation_id: str,
item_id: str,
db: Session,
) -> bool:
"""
Remove a generation from a story.
Remove a story item from a story.
Args:
story_id: Story ID
generation_id: Generation ID to remove
item_id: Story item ID to remove
db: Database session
Returns:
True if removed, False if not found
"""
item = db.query(DBStoryItem).filter_by(
id=item_id,
story_id=story_id,
generation_id=generation_id
).first()
if not item:
return False
@@ -434,6 +444,277 @@ async def remove_item_from_story(
return True
async def trim_story_item(
story_id: str,
item_id: str,
data: StoryItemTrim,
db: Session,
) -> Optional[StoryItemDetail]:
"""
Trim a story item (update trim_start_ms and trim_end_ms).
Args:
story_id: Story ID
item_id: Story item ID
data: Trim data (trim_start_ms, trim_end_ms)
db: Database session
Returns:
Updated item detail or None if not found
"""
# Get the item
item = db.query(DBStoryItem).filter_by(
id=item_id,
story_id=story_id,
).first()
if not item:
return None
# Get the generation
generation = db.query(DBGeneration).filter_by(id=item.generation_id).first()
if not generation:
return None
# Validate trim values don't exceed duration
max_duration_ms = int(generation.duration * 1000)
if data.trim_start_ms + data.trim_end_ms >= max_duration_ms:
return None # Invalid trim - would result in zero or negative duration
# Update trim values
item.trim_start_ms = data.trim_start_ms
item.trim_end_ms = data.trim_end_ms
# Update story updated_at
story = db.query(DBStory).filter_by(id=story_id).first()
if story:
story.updated_at = datetime.utcnow()
db.commit()
db.refresh(item)
# Get profile name
profile = db.query(DBVoiceProfile).filter_by(id=generation.profile_id).first()
return StoryItemDetail(
id=item.id,
story_id=item.story_id,
generation_id=item.generation_id,
start_time_ms=item.start_time_ms,
track=item.track,
trim_start_ms=item.trim_start_ms,
trim_end_ms=item.trim_end_ms,
created_at=item.created_at,
profile_id=generation.profile_id,
profile_name=profile.name if profile else "Unknown",
text=generation.text,
language=generation.language,
audio_path=generation.audio_path,
duration=generation.duration,
seed=generation.seed,
instruct=generation.instruct,
generation_created_at=generation.created_at,
)
async def split_story_item(
story_id: str,
item_id: str,
data: StoryItemSplit,
db: Session,
) -> Optional[List[StoryItemDetail]]:
"""
Split a story item at a given time, creating two clips.
Args:
story_id: Story ID
item_id: Story item ID to split
data: Split data (split_time_ms - time within clip to split at)
db: Database session
Returns:
List of two updated item details (original and new) or None if not found/invalid
"""
# Get the item
item = db.query(DBStoryItem).filter_by(
id=item_id,
story_id=story_id,
).first()
if not item:
return None
# Get the generation
generation = db.query(DBGeneration).filter_by(id=item.generation_id).first()
if not generation:
return None
# Calculate effective duration and validate split point
current_trim_start = getattr(item, 'trim_start_ms', 0)
current_trim_end = getattr(item, 'trim_end_ms', 0)
original_duration_ms = int(generation.duration * 1000)
effective_duration_ms = original_duration_ms - current_trim_start - current_trim_end
# Validate split_time_ms is within the effective duration
if data.split_time_ms <= 0 or data.split_time_ms >= effective_duration_ms:
return None # Invalid split point
# Calculate the absolute time in the original audio where we're splitting
absolute_split_ms = current_trim_start + data.split_time_ms
# Update original clip: trim from the end
item.trim_end_ms = original_duration_ms - absolute_split_ms
# Create new clip: starts after the split, trimmed from the start
new_item = DBStoryItem(
id=str(uuid.uuid4()),
story_id=story_id,
generation_id=item.generation_id, # Same generation, different trim
start_time_ms=item.start_time_ms + data.split_time_ms,
track=item.track,
trim_start_ms=absolute_split_ms,
trim_end_ms=current_trim_end,
created_at=datetime.utcnow(),
)
db.add(new_item)
# Update story updated_at
story = db.query(DBStory).filter_by(id=story_id).first()
if story:
story.updated_at = datetime.utcnow()
db.commit()
db.refresh(item)
db.refresh(new_item)
# Get profile name
profile = db.query(DBVoiceProfile).filter_by(id=generation.profile_id).first()
profile_name = profile.name if profile else "Unknown"
# Build response items
original_item_detail = StoryItemDetail(
id=item.id,
story_id=item.story_id,
generation_id=item.generation_id,
start_time_ms=item.start_time_ms,
track=item.track,
trim_start_ms=item.trim_start_ms,
trim_end_ms=item.trim_end_ms,
created_at=item.created_at,
profile_id=generation.profile_id,
profile_name=profile_name,
text=generation.text,
language=generation.language,
audio_path=generation.audio_path,
duration=generation.duration,
seed=generation.seed,
instruct=generation.instruct,
generation_created_at=generation.created_at,
)
new_item_detail = StoryItemDetail(
id=new_item.id,
story_id=new_item.story_id,
generation_id=new_item.generation_id,
start_time_ms=new_item.start_time_ms,
track=new_item.track,
trim_start_ms=new_item.trim_start_ms,
trim_end_ms=new_item.trim_end_ms,
created_at=new_item.created_at,
profile_id=generation.profile_id,
profile_name=profile_name,
text=generation.text,
language=generation.language,
audio_path=generation.audio_path,
duration=generation.duration,
seed=generation.seed,
instruct=generation.instruct,
generation_created_at=generation.created_at,
)
return [original_item_detail, new_item_detail]
async def duplicate_story_item(
story_id: str,
item_id: str,
db: Session,
) -> Optional[StoryItemDetail]:
"""
Duplicate a story item, creating a copy with all properties.
Args:
story_id: Story ID
item_id: Story item ID to duplicate
db: Database session
Returns:
New item detail or None if not found
"""
# Get the original item
original_item = db.query(DBStoryItem).filter_by(
id=item_id,
story_id=story_id,
).first()
if not original_item:
return None
# Get the generation
generation = db.query(DBGeneration).filter_by(id=original_item.generation_id).first()
if not generation:
return None
# Calculate effective duration
current_trim_start = getattr(original_item, 'trim_start_ms', 0)
current_trim_end = getattr(original_item, 'trim_end_ms', 0)
original_duration_ms = int(generation.duration * 1000)
effective_duration_ms = original_duration_ms - current_trim_start - current_trim_end
# Create duplicate item - place it right after the original
new_item = DBStoryItem(
id=str(uuid.uuid4()),
story_id=story_id,
generation_id=original_item.generation_id, # Same generation as original
start_time_ms=original_item.start_time_ms + effective_duration_ms + 200, # 200ms gap
track=original_item.track,
trim_start_ms=current_trim_start,
trim_end_ms=current_trim_end,
created_at=datetime.utcnow(),
)
db.add(new_item)
# Update story updated_at
story = db.query(DBStory).filter_by(id=story_id).first()
if story:
story.updated_at = datetime.utcnow()
db.commit()
db.refresh(new_item)
# Get profile name
profile = db.query(DBVoiceProfile).filter_by(id=generation.profile_id).first()
return StoryItemDetail(
id=new_item.id,
story_id=new_item.story_id,
generation_id=new_item.generation_id,
start_time_ms=new_item.start_time_ms,
track=new_item.track,
trim_start_ms=new_item.trim_start_ms,
trim_end_ms=new_item.trim_end_ms,
created_at=new_item.created_at,
profile_id=generation.profile_id,
profile_name=profile.name if profile else "Unknown",
text=generation.text,
language=generation.language,
audio_path=generation.audio_path,
duration=generation.duration,
seed=generation.seed,
instruct=generation.instruct,
generation_created_at=generation.created_at,
)
async def update_story_item_times(
story_id: str,
data: StoryItemBatchUpdate,
@@ -538,6 +819,8 @@ async def reorder_story_items(
generation_id=item.generation_id,
start_time_ms=item.start_time_ms,
track=item.track,
trim_start_ms=getattr(item, 'trim_start_ms', 0),
trim_end_ms=getattr(item, 'trim_end_ms', 0),
created_at=item.created_at,
profile_id=generation.profile_id,
profile_name=profile_name,
@@ -602,14 +885,31 @@ async def export_story_audio(
audio, sr = load_audio(str(audio_path), sample_rate=sample_rate)
sample_rate = sr # Use actual sample rate from first file
# Get trim values
trim_start_ms = getattr(item, 'trim_start_ms', 0)
trim_end_ms = getattr(item, 'trim_end_ms', 0)
# Calculate effective duration
original_duration_ms = int(generation.duration * 1000)
effective_duration_ms = original_duration_ms - trim_start_ms - trim_end_ms
# Slice audio based on trim values
trim_start_sample = int((trim_start_ms / 1000.0) * sample_rate)
trim_end_sample = int((trim_end_ms / 1000.0) * sample_rate)
# Extract the trimmed portion
if trim_end_ms > 0:
trimmed_audio = audio[trim_start_sample:-trim_end_sample] if trim_end_sample > 0 else audio[trim_start_sample:]
else:
trimmed_audio = audio[trim_start_sample:]
# Store audio with its timecode info
start_time_ms = item.start_time_ms
duration_ms = int(generation.duration * 1000)
audio_data.append({
'audio': audio,
'audio': trimmed_audio,
'start_time_ms': start_time_ms,
'duration_ms': duration_ms,
'duration_ms': effective_duration_ms,
})
except Exception:
# Skip files that can't be loaded
+58
View File
@@ -0,0 +1,58 @@
# Backend Tests
Manual test scripts for debugging and validating backend functionality.
## Test Files
### `test_generation_progress.py`
Tests TTS generation with SSE progress monitoring to identify UX issues where users see download progress even when the model is already cached.
**Usage:**
```bash
cd backend
python tests/test_generation_progress.py
```
**Prerequisites:**
- Server must be running (`python main.py`)
- At least one voice profile must exist
### `test_real_download.py`
Tests real model download with SSE progress monitoring.
**Usage:**
```bash
cd backend
# Delete cache first to force fresh download
rm -rf ~/.cache/huggingface/hub/models--openai--whisper-base
python tests/test_real_download.py
```
**Prerequisites:**
- Server must be running (`python main.py`)
### `test_progress.py`
Unit tests for ProgressManager and HFProgressTracker functionality.
**Usage:**
```bash
cd backend
python tests/test_progress.py
```
### `test_check_progress_state.py`
Debugging script to inspect the internal state of ProgressManager and TaskManager.
**Usage:**
```bash
cd backend
python tests/test_check_progress_state.py
```
## Notes
These are manual test scripts, not automated unit tests. They're designed for:
- Debugging progress tracking issues
- Validating SSE event streams
- Monitoring real-time download behavior
- Inspecting internal state during development
+6
View File
@@ -0,0 +1,6 @@
"""
Test suite for Voicebox backend.
This directory contains manual test scripts for debugging and validating
progress tracking, model downloads, and generation functionality.
"""
+321
View File
@@ -0,0 +1,321 @@
"""
Test TTS generation with SSE progress monitoring.
This test captures the exact SSE events triggered during generation
to identify UX issues where users see download progress even when
the model is already cached.
"""
import asyncio
import json
import httpx
from typing import List, Dict, Optional
from datetime import datetime
async def monitor_sse_stream(model_name: str, timeout: int = 120):
"""Monitor SSE stream for a model during generation."""
events: List[Dict] = []
url = f"http://localhost:8000/models/progress/{model_name}"
print(f"[{_timestamp()}] Connecting to SSE endpoint: {url}")
try:
async with httpx.AsyncClient(timeout=timeout) as client:
async with client.stream("GET", url) as response:
print(f"[{_timestamp()}] SSE connected, status: {response.status_code}")
if response.status_code != 200:
print(f"[{_timestamp()}] Error: SSE endpoint returned {response.status_code}")
return events
async for line in response.aiter_lines():
if not line:
continue
timestamp = _timestamp()
if line.startswith("data: "):
try:
data = json.loads(line[6:])
print(f"[{timestamp}] → SSE Event: {data['status']:12} {data.get('progress', 0):6.1f}% {data.get('filename', '')}")
events.append({
**data,
"_timestamp": timestamp
})
# Stop if complete or error
if data.get("status") in ("complete", "error"):
print(f"[{timestamp}] → Model {data['status']}!")
break
except json.JSONDecodeError as e:
print(f"[{timestamp}] Error parsing JSON: {e}")
print(f" Line was: {line}")
elif line.startswith(": heartbeat"):
print(f"[{timestamp}] ♥ heartbeat")
except asyncio.TimeoutError:
print(f"[{_timestamp()}] SSE monitoring timed out")
except Exception as e:
print(f"[{_timestamp()}] SSE error: {e}")
return events
async def trigger_generation(profile_id: str, text: str, model_size: str = "1.7B"):
"""Trigger TTS generation via the API."""
url = "http://localhost:8000/generate"
print(f"\n[{_timestamp()}] Triggering generation...")
print(f" Profile: {profile_id}")
print(f" Text: {text[:50]}...")
print(f" Model: {model_size}")
try:
async with httpx.AsyncClient(timeout=120) as client:
response = await client.post(url, json={
"profile_id": profile_id,
"text": text,
"language": "en",
"model_size": model_size,
})
print(f"[{_timestamp()}] Response: {response.status_code}")
if response.status_code == 200:
result = response.json()
print(f"[{_timestamp()}] ✓ Generation successful!")
print(f" Generation ID: {result.get('id')}")
print(f" Duration: {result.get('duration', 0):.2f}s")
return True, result
elif response.status_code == 202:
# Model is being downloaded
result = response.json()
print(f"[{_timestamp()}] → Model download in progress")
print(f" Detail: {result}")
return False, result
else:
print(f"[{_timestamp()}] ✗ Error: {response.text}")
return False, None
except Exception as e:
print(f"[{_timestamp()}] ✗ Exception: {e}")
return False, None
async def get_first_profile():
"""Get the first available voice profile."""
url = "http://localhost:8000/profiles"
try:
async with httpx.AsyncClient(timeout=10) as client:
response = await client.get(url)
if response.status_code == 200:
profiles = response.json()
if profiles:
return profiles[0]["id"]
except Exception as e:
print(f"Error getting profiles: {e}")
return None
async def check_server():
"""Check if the server is running."""
try:
async with httpx.AsyncClient(timeout=5) as client:
response = await client.get("http://localhost:8000/health")
return response.status_code == 200
except Exception as e:
print(f"Server not running: {e}")
return False
def _timestamp():
"""Get current timestamp for logging."""
return datetime.now().strftime("%H:%M:%S.%f")[:-3]
async def test_generation_with_cached_model():
"""
Test Case 1: Generation when model is already cached.
This should NOT show any download progress events.
If it does, that's the UX bug we're trying to fix.
"""
print("\n" + "=" * 80)
print("TEST CASE 1: Generation with Cached Model")
print("=" * 80)
print("Expected: No download progress events (or minimal/instant completion)")
print("Actual UX Issue: Users see 'started' and 'finished' events even for cached models")
print("=" * 80)
model_size = "1.7B"
model_name = f"qwen-tts-{model_size}"
# Get a profile
profile_id = await get_first_profile()
if not profile_id:
print("✗ No voice profiles found. Please create a profile first.")
return False
print(f"\nUsing profile: {profile_id}")
# Start SSE monitor BEFORE triggering generation
monitor_task = asyncio.create_task(monitor_sse_stream(model_name, timeout=30))
# Wait for SSE to connect
await asyncio.sleep(1)
# Trigger generation
test_text = "Hello, this is a test of the voice generation system."
success, result = await trigger_generation(profile_id, test_text, model_size)
if not success and result and result.get("downloading"):
print("\n⚠ Model is being downloaded. Waiting for download to complete...")
# Wait for SSE monitor to capture download events
events = await monitor_task
return events
# Wait a bit more to catch any progress events
await asyncio.sleep(3)
# Cancel SSE monitor
monitor_task.cancel()
try:
events = await monitor_task
except asyncio.CancelledError:
events = []
return events
async def test_generation_with_fresh_download():
"""
Test Case 2: Generation when model needs to be downloaded.
This SHOULD show download progress events.
"""
print("\n" + "=" * 80)
print("TEST CASE 2: Generation with Model Download")
print("=" * 80)
print("Expected: Download progress events from 0% to 100%")
print("=" * 80)
# Use a different model size to force download
model_size = "0.6B" # Smaller model for faster testing
model_name = f"qwen-tts-{model_size}"
# Get a profile
profile_id = await get_first_profile()
if not profile_id:
print("✗ No voice profiles found. Please create a profile first.")
return False
print(f"\nUsing profile: {profile_id}")
print("Note: This will download the model if not cached")
# Start SSE monitor BEFORE triggering generation
monitor_task = asyncio.create_task(monitor_sse_stream(model_name, timeout=300))
# Wait for SSE to connect
await asyncio.sleep(1)
# Trigger generation
test_text = "This should trigger a model download if the model is not cached."
success, result = await trigger_generation(profile_id, test_text, model_size)
if not success and result and result.get("downloading"):
print("\n→ Model download initiated. Monitoring progress...")
# Wait for download to complete
events = await monitor_task
# Try generation again
print(f"\n[{_timestamp()}] Retrying generation after download...")
await asyncio.sleep(2)
success, result = await trigger_generation(profile_id, test_text, model_size)
if success:
print("✓ Generation successful after download")
return events
# If model was already cached
await asyncio.sleep(3)
monitor_task.cancel()
try:
events = await monitor_task
except asyncio.CancelledError:
events = []
return events
async def main():
print("=" * 80)
print("TTS Generation Progress Test")
print("=" * 80)
print("Purpose: Capture exact SSE events during generation to identify UX issues")
print("=" * 80)
# Check if server is running
print(f"\n[{_timestamp()}] Checking if server is running...")
if not await check_server():
print("✗ Server is not running on http://localhost:8000")
print("\nPlease start the server first:")
print(" cd backend && python main.py")
return False
print("✓ Server is running")
# Test Case 1: Cached model
print("\n" + "🧪 " * 20)
events_cached = await test_generation_with_cached_model()
# Results for Test Case 1
print("\n" + "=" * 80)
print("TEST CASE 1 RESULTS: Generation with Cached Model")
print("=" * 80)
if not events_cached:
print("✓ GOOD: No SSE progress events received")
print(" This is the expected behavior for a cached model.")
else:
print(f"⚠ ISSUE FOUND: Received {len(events_cached)} SSE events:")
print("\nEvent Timeline:")
for i, event in enumerate(events_cached, 1):
timestamp = event.pop("_timestamp", "??:??:??.???")
print(f" {i}. [{timestamp}] {event}")
print("\n⚠ This explains the UX issue!")
print(" Users see progress events even when the model is already cached,")
print(" making them think the model is downloading again.")
# Test Case 2: Fresh download (optional, commented out by default)
# Uncomment if you want to test download progress
# print("\n" + "🧪 " * 20)
# events_download = await test_generation_with_fresh_download()
#
# print("\n" + "=" * 80)
# print("TEST CASE 2 RESULTS: Generation with Model Download")
# print("=" * 80)
#
# if not events_download:
# print("ℹ Model was already cached, no download occurred")
# else:
# print(f"✓ Received {len(events_download)} download progress events")
# print("\nDownload Timeline:")
# for i, event in enumerate(events_download, 1):
# timestamp = event.pop("_timestamp", "??:??:??.???")
# print(f" {i}. [{timestamp}] {event}")
print("\n" + "=" * 80)
print("Test Complete!")
print("=" * 80)
return True
if __name__ == "__main__":
asyncio.run(main())
+313
View File
@@ -0,0 +1,313 @@
"""
Test script to debug model download progress tracking.
"""
import asyncio
import json
import time
from typing import List, Dict
import logging
# Set up logging to see what's happening
logging.basicConfig(
level=logging.DEBUG,
format='%(asctime)s - %(name)s - %(levelname)s - %(message)s'
)
from utils.progress import ProgressManager, get_progress_manager
from utils.hf_progress import HFProgressTracker, create_hf_progress_callback
def test_progress_manager_basic():
"""Test 1: Basic ProgressManager functionality."""
print("\n" + "=" * 60)
print("Test 1: ProgressManager Basic Operations")
print("=" * 60)
pm = ProgressManager()
# Test update_progress
pm.update_progress(
model_name="test-model",
current=50,
total=100,
filename="test.bin",
status="downloading"
)
# Test get_progress
progress = pm.get_progress("test-model")
print(f"✓ Progress stored: {progress}")
assert progress is not None
assert progress["progress"] == 50.0
assert progress["filename"] == "test.bin"
assert progress["status"] == "downloading"
# Test mark_complete
pm.mark_complete("test-model")
progress = pm.get_progress("test-model")
print(f"✓ Marked complete: {progress}")
assert progress["status"] == "complete"
assert progress["progress"] == 100.0
print("✓ Test 1 PASSED\n")
return True
async def test_progress_manager_sse():
"""Test 2: ProgressManager SSE streaming."""
print("\n" + "=" * 60)
print("Test 2: ProgressManager SSE Streaming")
print("=" * 60)
pm = ProgressManager()
collected_events: List[Dict] = []
# Simulate SSE client
async def sse_client():
"""Simulates a frontend SSE connection."""
print(" SSE client: Subscribing to test-model-sse...")
async for event in pm.subscribe("test-model-sse"):
# Parse SSE event
if event.startswith("data: "):
data = json.loads(event[6:])
print(f" SSE client: Received event: {data['status']} - {data.get('progress', 0):.1f}%")
collected_events.append(data)
# Stop when complete
if data.get("status") in ("complete", "error"):
break
elif event.startswith(": heartbeat"):
print(" SSE client: Received heartbeat")
# Simulate download progress updates (from backend thread)
async def simulate_download():
"""Simulates backend sending progress updates."""
print(" Backend: Starting simulated download...")
await asyncio.sleep(0.2) # Let SSE client subscribe first
# Send progress updates
for i in range(0, 101, 20):
print(f" Backend: Updating progress to {i}%")
pm.update_progress(
model_name="test-model-sse",
current=i,
total=100,
filename=f"file_{i}.bin",
status="downloading" if i < 100 else "downloading"
)
await asyncio.sleep(0.1)
# Mark complete
print(" Backend: Marking download complete")
pm.mark_complete("test-model-sse")
# Run SSE client and download simulation concurrently
await asyncio.gather(
sse_client(),
simulate_download()
)
# Verify we got events
print(f"\n Collected {len(collected_events)} events")
assert len(collected_events) > 0, "Should have received at least one event"
assert collected_events[-1]["status"] == "complete", "Last event should be 'complete'"
print("✓ Test 2 PASSED\n")
return True
def test_hf_progress_tracker():
"""Test 3: HFProgressTracker tqdm patching."""
print("\n" + "=" * 60)
print("Test 3: HFProgressTracker tqdm Patching")
print("=" * 60)
captured_progress: List[tuple] = []
def progress_callback(downloaded: int, total: int, filename: str):
"""Capture progress updates."""
captured_progress.append((downloaded, total, filename))
print(f" Progress callback: {downloaded}/{total} bytes ({filename})")
tracker = HFProgressTracker(progress_callback)
# Simulate a download with tqdm
with tracker.patch_download():
try:
from tqdm import tqdm
# Simulate downloading a file
print(" Simulating download with tqdm...")
total_size = 1000
with tqdm(total=total_size, desc="model.bin", unit="B", unit_scale=True) as pbar:
for chunk in range(0, total_size, 100):
pbar.update(100)
time.sleep(0.01)
print(f" Captured {len(captured_progress)} progress updates")
assert len(captured_progress) > 0, "Should have captured progress updates"
# Verify progress increases
last_downloaded = 0
for downloaded, total, filename in captured_progress:
assert downloaded >= last_downloaded, "Downloaded bytes should increase"
assert total == total_size, "Total should be consistent"
last_downloaded = downloaded
print("✓ Test 3 PASSED\n")
return True
except ImportError:
print("✗ tqdm not available, skipping test\n")
return None
async def test_full_integration():
"""Test 4: Full integration test."""
print("\n" + "=" * 60)
print("Test 4: Full Integration (ProgressManager + HFProgressTracker)")
print("=" * 60)
pm = get_progress_manager()
collected_events: List[Dict] = []
# SSE client
async def sse_client():
print(" SSE client: Subscribing...")
async for event in pm.subscribe("integration-test"):
if event.startswith("data: "):
data = json.loads(event[6:])
print(f" SSE client: {data['status']} - {data.get('progress', 0):.1f}% - {data.get('filename', '')}")
collected_events.append(data)
if data.get("status") in ("complete", "error"):
break
# Simulate backend download with HFProgressTracker
async def simulate_real_download():
await asyncio.sleep(0.2) # Let SSE subscribe
print(" Backend: Starting download with HFProgressTracker...")
# Set up tracking (like the real backend does)
progress_callback = create_hf_progress_callback("integration-test", pm)
tracker = HFProgressTracker(progress_callback)
# Initialize progress
pm.update_progress(
model_name="integration-test",
current=0,
total=1,
filename="",
status="downloading"
)
# Simulate download with tqdm patching
with tracker.patch_download():
try:
from tqdm import tqdm
# Simulate multi-file download (like HuggingFace does)
files = [
("model.safetensors", 5000),
("config.json", 1000),
("tokenizer.json", 500),
]
for filename, size in files:
print(f" Backend: Downloading {filename}...")
with tqdm(total=size, desc=filename, unit="B") as pbar:
for chunk in range(0, size, 500):
chunk_size = min(500, size - chunk)
pbar.update(chunk_size)
await asyncio.sleep(0.05)
# Mark complete
print(" Backend: Download complete")
pm.mark_complete("integration-test")
except ImportError:
print(" ✗ tqdm not available")
pm.mark_error("integration-test", "tqdm not available")
# Run both
await asyncio.gather(
sse_client(),
simulate_real_download()
)
# Verify
print(f"\n Collected {len(collected_events)} events")
if len(collected_events) > 0:
print(f" First event: {collected_events[0]}")
print(f" Last event: {collected_events[-1]}")
assert collected_events[-1]["status"] == "complete", "Should end with 'complete'"
print("✓ Test 4 PASSED\n")
return True
else:
print("✗ Test 4 FAILED - No events received\n")
return False
async def main():
"""Run all tests."""
print("\n" + "=" * 60)
print("Voicebox Progress Tracking Test Suite")
print("=" * 60)
results = []
# Test 1: Basic operations
try:
results.append(("Basic Operations", test_progress_manager_basic()))
except Exception as e:
print(f"✗ Test 1 FAILED: {e}\n")
results.append(("Basic Operations", False))
# Test 2: SSE streaming
try:
results.append(("SSE Streaming", await test_progress_manager_sse()))
except Exception as e:
print(f"✗ Test 2 FAILED: {e}\n")
results.append(("SSE Streaming", False))
# Test 3: tqdm patching
try:
results.append(("tqdm Patching", test_hf_progress_tracker()))
except Exception as e:
print(f"✗ Test 3 FAILED: {e}\n")
results.append(("tqdm Patching", False))
# Test 4: Full integration
try:
results.append(("Full Integration", await test_full_integration()))
except Exception as e:
print(f"✗ Test 4 FAILED: {e}\n")
results.append(("Full Integration", False))
# Summary
print("\n" + "=" * 60)
print("Test Results Summary")
print("=" * 60)
for name, result in results:
status = "✓ PASS" if result else ("⊘ SKIP" if result is None else "✗ FAIL")
print(f" {status:8} {name}")
passed = sum(1 for _, r in results if r is True)
failed = sum(1 for _, r in results if r is False)
skipped = sum(1 for _, r in results if r is None)
print()
print(f" Total: {len(results)} tests")
print(f" Passed: {passed}")
print(f" Failed: {failed}")
print(f" Skipped: {skipped}")
print("=" * 60 + "\n")
return failed == 0
if __name__ == "__main__":
success = asyncio.run(main())
exit(0 if success else 1)
+317
View File
@@ -0,0 +1,317 @@
"""
Test Qwen TTS model download with SSE progress monitoring.
This specifically tests the MLX TTS backend download progress tracking,
which requires tqdm to be patched BEFORE mlx_audio is imported.
Usage:
cd backend && python -m tests.test_qwen_download
Prerequisites:
- Server must be running: cd backend && python main.py
- Delete model first for fresh download test:
curl -X DELETE http://localhost:8000/models/qwen-tts-0.6B
"""
import asyncio
import json
import httpx
import time
from typing import List, Dict, Optional
async def monitor_sse_stream(model_name: str, timeout: int = 600) -> List[Dict]:
"""
Monitor SSE stream for a model download.
Args:
model_name: Name of the model to monitor
timeout: Maximum time to wait for download (seconds)
Returns:
List of SSE events received
"""
events: List[Dict] = []
url = f"http://localhost:8000/models/progress/{model_name}"
last_progress = -1
print(f"\n📡 Connecting to SSE endpoint: {url}")
try:
async with httpx.AsyncClient(timeout=timeout) as client:
async with client.stream("GET", url) as response:
print(f" SSE connected, status: {response.status_code}")
if response.status_code != 200:
print(f" ❌ Error: SSE endpoint returned {response.status_code}")
return events
async for line in response.aiter_lines():
if not line:
continue
if line.startswith("data: "):
try:
data = json.loads(line[6:])
events.append(data)
# Print progress (only when it changes significantly)
progress = data.get('progress', 0)
status = data.get('status', 'unknown')
filename = data.get('filename', '')
current = data.get('current', 0)
total = data.get('total', 0)
# Print every 5% change or status change
if abs(progress - last_progress) >= 5 or status in ('complete', 'error'):
current_mb = current / (1024 * 1024)
total_mb = total / (1024 * 1024)
print(f" 📊 {status:12} {progress:6.1f}% ({current_mb:.1f}MB / {total_mb:.1f}MB) {filename[:50]}")
last_progress = progress
# Stop if complete or error
if status in ("complete", "error"):
if status == "complete":
print(f" ✅ Download complete!")
else:
print(f" ❌ Download error: {data.get('error', 'unknown')}")
break
except json.JSONDecodeError as e:
print(f" ⚠️ Error parsing JSON: {e}")
elif line.startswith(": heartbeat"):
# Heartbeat every 1 second, don't spam
pass
except asyncio.CancelledError:
print(" ⏹️ SSE monitor cancelled")
except Exception as e:
print(f" ❌ SSE error: {e}")
return events
async def trigger_download(model_name: str) -> bool:
"""Trigger a model download via the API."""
url = "http://localhost:8000/models/download"
print(f"\n🚀 Triggering download for: {model_name}")
try:
async with httpx.AsyncClient(timeout=30) as client:
response = await client.post(url, json={"model_name": model_name})
result = response.json()
print(f" Response: {response.status_code} - {result}")
return response.status_code == 200
except Exception as e:
print(f" ❌ Error triggering download: {e}")
return False
async def delete_model(model_name: str) -> bool:
"""Delete a model from cache."""
url = f"http://localhost:8000/models/{model_name}"
print(f"\n🗑️ Deleting model: {model_name}")
try:
async with httpx.AsyncClient(timeout=30) as client:
response = await client.delete(url)
if response.status_code == 200:
print(f" ✅ Model deleted")
return True
elif response.status_code == 404:
print(f" ℹ️ Model not found (already deleted)")
return True
else:
print(f" ⚠️ Delete response: {response.status_code} - {response.text}")
return False
except Exception as e:
print(f" ❌ Error deleting model: {e}")
return False
async def check_model_status(model_name: str) -> Optional[Dict]:
"""Check the status of a model."""
try:
async with httpx.AsyncClient(timeout=10) as client:
response = await client.get("http://localhost:8000/models/status")
if response.status_code == 200:
data = response.json()
for model in data.get("models", []):
if model["model_name"] == model_name:
return model
except Exception as e:
print(f" ⚠️ Error checking model status: {e}")
return None
async def check_server() -> bool:
"""Check if the server is running."""
try:
async with httpx.AsyncClient(timeout=5) as client:
response = await client.get("http://localhost:8000/health")
return response.status_code == 200
except Exception:
return False
async def main():
print("=" * 70)
print("🧪 Qwen TTS Model Download Progress Test")
print("=" * 70)
print("\nThis test verifies that MLX TTS download progress tracking works.")
print("It specifically tests the tqdm patching for mlx_audio.tts imports.")
# Check if server is running
print("\n📡 Checking if server is running...")
if not await check_server():
print(" ❌ Server is not running on http://localhost:8000")
print("\n Please start the server first:")
print(" cd backend && python main.py")
return False
print(" ✅ Server is running")
# Test model
model_name = "qwen-tts-0.6B" # Note: 0.6B currently maps to 1.7B on MLX
# Check current status
print(f"\n📊 Checking status of {model_name}...")
status = await check_model_status(model_name)
if status:
print(f" Downloaded: {status.get('downloaded', False)}")
print(f" Downloading: {status.get('downloading', False)}")
print(f" Loaded: {status.get('loaded', False)}")
if status.get('size_mb'):
print(f" Size: {status['size_mb']:.1f} MB")
else:
print(" ⚠️ Could not get model status")
# Ask if user wants to delete first
print("\n" + "-" * 70)
if status and status.get('downloaded'):
print("⚠️ Model is already downloaded. Delete it for a fresh download test?")
print(" [y] Yes, delete and download fresh")
print(" [n] No, just test SSE connection")
print(" [q] Quit")
choice = input("\nChoice [y/n/q]: ").strip().lower()
if choice == 'q':
print("Exiting...")
return True
if choice == 'y':
if not await delete_model(model_name):
print("Failed to delete model. Continue anyway? [y/n]")
if input().strip().lower() != 'y':
return False
else:
print("Model not downloaded. Will perform fresh download test.")
input("Press Enter to continue...")
# Run the test
print("\n" + "=" * 70)
print("🏃 Starting Download Test")
print("=" * 70)
async def run_test():
# Start SSE monitor in background FIRST
monitor_task = asyncio.create_task(monitor_sse_stream(model_name, timeout=600))
# Wait for SSE to connect
await asyncio.sleep(1)
# Trigger download
success = await trigger_download(model_name)
if not success:
print(" ❌ Failed to trigger download")
monitor_task.cancel()
try:
await monitor_task
except asyncio.CancelledError:
pass
return []
# Wait for SSE monitor to complete
print("\n⏳ Waiting for download to complete (this may take several minutes)...")
events = await monitor_task
return events
start_time = time.time()
events = await run_test()
elapsed = time.time() - start_time
# Results
print("\n" + "=" * 70)
print("📋 Test Results")
print("=" * 70)
print(f"\n⏱️ Elapsed time: {elapsed:.1f} seconds")
print(f"📨 Total SSE events received: {len(events)}")
if not events:
print("\n❌ FAILED - No SSE events received!")
print("\nPossible causes:")
print(" 1. SSE endpoint not working")
print(" 2. tqdm not patched before mlx_audio import")
print(" 3. Progress callbacks not firing")
print(" 4. Model already fully downloaded")
print("\nDebug steps:")
print(" 1. Check server logs for [DEBUG] messages")
print(" 2. Look for 'tqdm patched' before 'mlx_audio.tts import'")
print(f" 3. Delete model: curl -X DELETE http://localhost:8000/models/{model_name}")
return False
# Analyze events
first_event = events[0]
last_event = events[-1]
print(f"\n📊 First event:")
print(f" Status: {first_event.get('status')}")
print(f" Progress: {first_event.get('progress', 0):.1f}%")
print(f"\n📊 Last event:")
print(f" Status: {last_event.get('status')}")
print(f" Progress: {last_event.get('progress', 0):.1f}%")
# Check for expected behaviors
has_progress_updates = len(events) > 2
has_increasing_progress = False
has_complete = any(e.get('status') == 'complete' for e in events)
has_100_percent = any(e.get('progress', 0) >= 100 for e in events)
# Check if progress increased over time
if len(events) >= 2:
progress_values = [e.get('progress', 0) for e in events]
has_increasing_progress = progress_values[-1] > progress_values[0]
print("\n📋 Checks:")
print(f" {'✅' if has_progress_updates else '❌'} Multiple progress updates received ({len(events)} events)")
print(f" {'✅' if has_increasing_progress else '❌'} Progress increased over time")
print(f" {'✅' if has_100_percent else '❌'} Reached 100% progress")
print(f" {'✅' if has_complete else '❌'} Received 'complete' status")
# Overall result
success = has_progress_updates and has_complete
if success:
print("\n" + "=" * 70)
print("✅ TEST PASSED - Qwen TTS download progress tracking works!")
print("=" * 70)
else:
print("\n" + "=" * 70)
print("❌ TEST FAILED - Progress tracking has issues")
print("=" * 70)
print("\nCheck the server logs for debug output.")
return success
if __name__ == "__main__":
result = asyncio.run(main())
exit(0 if result else 1)
+178
View File
@@ -0,0 +1,178 @@
"""
Test real model download with SSE progress monitoring.
"""
import asyncio
import json
import httpx
import time
from typing import List, Dict
async def monitor_sse_stream(model_name: str, timeout: int = 300):
"""Monitor SSE stream for a model download."""
events: List[Dict] = []
url = f"http://localhost:8000/models/progress/{model_name}"
print(f"Connecting to SSE endpoint: {url}")
async with httpx.AsyncClient(timeout=timeout) as client:
async with client.stream("GET", url) as response:
print(f"SSE connected, status: {response.status_code}")
if response.status_code != 200:
print(f"Error: SSE endpoint returned {response.status_code}")
return events
async for line in response.aiter_lines():
if not line:
continue
print(f" Raw SSE: {line[:100]}...") # Print first 100 chars
if line.startswith("data: "):
try:
data = json.loads(line[6:])
print(f" → {data['status']:12} {data.get('progress', 0):6.1f}% {data.get('filename', '')}")
events.append(data)
# Stop if complete or error
if data.get("status") in ("complete", "error"):
print(f" Download {data['status']}!")
break
except json.JSONDecodeError as e:
print(f" Error parsing JSON: {e}")
print(f" Line was: {line}")
elif line.startswith(": heartbeat"):
print(" ♥ heartbeat")
return events
async def trigger_download(model_name: str):
"""Trigger a model download via the API."""
url = "http://localhost:8000/models/download"
print(f"\nTriggering download for: {model_name}")
async with httpx.AsyncClient(timeout=300) as client:
response = await client.post(url, json={"model_name": model_name})
print(f"Response: {response.status_code} - {response.json()}")
return response.status_code == 200
async def check_server():
"""Check if the server is running."""
try:
async with httpx.AsyncClient(timeout=5) as client:
response = await client.get("http://localhost:8000/health")
return response.status_code == 200
except Exception as e:
print(f"Server not running: {e}")
return False
async def main():
print("=" * 60)
print("Real Model Download Progress Test")
print("=" * 60)
# Check if server is running
print("\nChecking if server is running...")
if not await check_server():
print("✗ Server is not running on http://localhost:8000")
print("\nPlease start the server first:")
print(" cd backend && python main.py")
return False
print("✓ Server is running")
# Choose a small model for testing
model_name = "whisper-base" # ~150MB, faster to download
print(f"\nUsing model: {model_name}")
# Option to delete model first if it exists
print("\nDo you want to delete the model first to force a fresh download? (y/n)")
# For automated testing, skip deletion prompt
# delete_first = input().strip().lower() == 'y'
delete_first = False
if delete_first:
print(f"Deleting {model_name}...")
async with httpx.AsyncClient(timeout=30) as client:
response = await client.delete(f"http://localhost:8000/models/{model_name}")
print(f"Delete response: {response.status_code}")
print("\n" + "=" * 60)
print("Starting Test")
print("=" * 60)
# Start monitoring SSE stream BEFORE triggering download
async def run_test():
# Start SSE monitor in background
monitor_task = asyncio.create_task(monitor_sse_stream(model_name))
# Wait a bit to ensure SSE is connected
await asyncio.sleep(1)
# Trigger download
success = await trigger_download(model_name)
if not success:
print("✗ Failed to trigger download")
monitor_task.cancel()
return False
# Wait for SSE monitor to complete
events = await monitor_task
return events
events = await run_test()
# Results
print("\n" + "=" * 60)
print("Test Results")
print("=" * 60)
if not events:
print("✗ FAILED - No SSE events received!")
print("\nPossible causes:")
print(" 1. SSE endpoint not working")
print(" 2. Progress updates not being sent")
print(" 3. Model already downloaded (no progress to report)")
print("\nTry deleting the model first to force a fresh download:")
print(f" curl -X DELETE http://localhost:8000/models/{model_name}")
return False
print(f"✓ Received {len(events)} SSE events")
print(f"\nFirst event: {events[0]}")
print(f"Last event: {events[-1]}")
# Check if we got meaningful progress
has_progress = any(e.get('progress', 0) > 0 for e in events)
has_complete = any(e.get('status') == 'complete' for e in events)
if has_progress:
print("✓ Progress updates received")
else:
print("✗ No progress updates (might be already downloaded)")
if has_complete:
print("✓ Download completed successfully")
else:
print("✗ Download did not complete")
success = has_progress and has_complete
if success:
print("\n✓ TEST PASSED - Progress tracking works!")
else:
print("\n⊘ TEST INCONCLUSIVE - Try with a fresh download")
return success
if __name__ == "__main__":
asyncio.run(main())
+12 -264
View File
@@ -1,274 +1,22 @@
"""
Whisper ASR module for transcription.
STT (Speech-to-Text) module - delegates to backend abstraction layer.
"""
from typing import Optional, List, Dict
import asyncio
import torch
import numpy as np
from pathlib import Path
from .utils.progress import get_progress_manager
from .utils.hf_progress import HFProgressTracker, create_hf_progress_callback
from .utils.tasks import get_task_manager
from typing import Optional
from .backends import get_stt_backend, STTBackend
class WhisperModel:
"""Manages Whisper model loading and transcription."""
def get_whisper_model() -> STTBackend:
"""
Get STT backend instance (MLX or PyTorch based on platform).
def __init__(self, model_size: str = "base"):
self.model = None
self.processor = None
self.model_size = model_size
self.device = self._get_device()
def _get_device(self) -> str:
"""Get the best available device."""
if torch.cuda.is_available():
return "cuda"
elif hasattr(torch.backends, 'mps') and torch.backends.mps.is_available():
# MPS support for Whisper
return "cpu" # Use CPU for stability
return "cpu"
def is_loaded(self) -> bool:
"""Check if model is loaded."""
return self.model is not None
def load_model(self, model_size: Optional[str] = None):
"""
Lazy load the Whisper model.
Args:
model_size: Model size (tiny, base, small, medium, large)
"""
if model_size is None:
model_size = self.model_size
if self.model is not None and self.model_size == model_size:
return
try:
from transformers import WhisperProcessor, WhisperForConditionalGeneration
model_name = f"openai/whisper-{model_size}"
# Set up progress tracking
progress_manager = get_progress_manager()
progress_model_name = f"whisper-{model_size}"
# Start tracking download task
task_manager = get_task_manager()
task_manager.start_download(progress_model_name)
print(f"Loading Whisper model {model_size} on {self.device}...")
# Initialize progress state to show download has started
progress_manager.update_progress(
model_name=progress_model_name,
current=0,
total=1, # Set to 1 initially, will be updated by callback
filename="",
status="downloading",
)
# Set up progress callback
progress_callback = create_hf_progress_callback(progress_model_name, progress_manager)
tracker = HFProgressTracker(progress_callback)
# Use progress tracker during download
with tracker.patch_download():
self.processor = WhisperProcessor.from_pretrained(model_name)
self.model = WhisperForConditionalGeneration.from_pretrained(model_name)
self.model.to(self.device)
self.model_size = model_size
# Mark as complete
progress_manager.mark_complete(progress_model_name)
task_manager.complete_download(progress_model_name)
print(f"Whisper model {model_size} loaded successfully")
except Exception as e:
print(f"Error loading Whisper model: {e}")
progress_manager = get_progress_manager()
task_manager = get_task_manager()
progress_model_name = f"whisper-{model_size}"
progress_manager.mark_error(progress_model_name, str(e))
task_manager.error_download(progress_model_name, str(e))
raise
async def load_model_async(self, model_size: Optional[str] = None):
"""
Async version of load_model that runs in thread pool.
This prevents blocking the event loop during model loading.
"""
if model_size is None:
model_size = self.model_size
# If already loaded with correct size, return immediately
if self.model is not None and self.model_size == model_size:
return
# Run the blocking load operation in a thread pool
await asyncio.to_thread(self.load_model, model_size)
def unload_model(self):
"""Unload the model to free memory."""
if self.model is not None:
del self.model
del self.processor
self.model = None
self.processor = None
if torch.cuda.is_available():
torch.cuda.empty_cache()
print("Whisper model unloaded")
async def transcribe(
self,
audio_path: str,
language: Optional[str] = None,
) -> str:
"""
Transcribe audio to text.
Args:
audio_path: Path to audio file
language: Optional language hint (en or zh)
Returns:
Transcribed text
"""
await self.load_model_async()
from .utils.audio import load_audio
def _transcribe_sync():
"""Run synchronous transcription in thread pool."""
# Load audio
audio, sr = load_audio(audio_path, sample_rate=16000)
# Process audio
inputs = self.processor(
audio,
sampling_rate=16000,
return_tensors="pt",
)
inputs = inputs.to(self.device)
# Set language if provided
forced_decoder_ids = None
if language:
lang_code = "en" if language == "en" else "zh"
forced_decoder_ids = self.processor.get_decoder_prompt_ids(
language=lang_code,
task="transcribe",
)
# Generate transcription
with torch.no_grad():
predicted_ids = self.model.generate(
inputs["input_features"],
forced_decoder_ids=forced_decoder_ids,
)
# Decode
transcription = self.processor.batch_decode(
predicted_ids,
skip_special_tokens=True,
)[0]
return transcription.strip()
# Run blocking transcription in thread pool
return await asyncio.to_thread(_transcribe_sync)
async def transcribe_with_timestamps(
self,
audio_path: str,
language: Optional[str] = None,
) -> List[Dict[str, any]]:
"""
Transcribe audio with word-level timestamps.
Args:
audio_path: Path to audio file
language: Optional language hint
Returns:
List of word segments with timestamps
"""
await self.load_model_async()
from .utils.audio import load_audio
def _transcribe_timestamps_sync():
"""Run synchronous transcription with timestamps in thread pool."""
# Load audio
audio, sr = load_audio(audio_path, sample_rate=16000)
# Process audio
inputs = self.processor(
audio,
sampling_rate=16000,
return_tensors="pt",
)
inputs = inputs.to(self.device)
# Set language if provided
forced_decoder_ids = None
if language:
lang_code = "en" if language == "en" else "zh"
forced_decoder_ids = self.processor.get_decoder_prompt_ids(
language=lang_code,
task="transcribe",
)
# Generate with timestamps
with torch.no_grad():
predicted_ids = self.model.generate(
inputs["input_features"],
forced_decoder_ids=forced_decoder_ids,
return_timestamps=True,
)
# Parse timestamps (simplified - would need more robust parsing)
# For now, return basic transcription
# TODO: Implement proper timestamp parsing
transcription = self.processor.batch_decode(
predicted_ids,
skip_special_tokens=True,
)[0]
return [
{
"text": transcription,
"start": 0.0,
"end": len(audio) / sr,
}
]
# Run blocking transcription in thread pool
return await asyncio.to_thread(_transcribe_timestamps_sync)
# Global model instance
_whisper_model: Optional[WhisperModel] = None
def get_whisper_model() -> WhisperModel:
"""Get or create Whisper model instance."""
global _whisper_model
if _whisper_model is None:
_whisper_model = WhisperModel()
return _whisper_model
Returns:
STT backend instance
"""
return get_stt_backend()
def unload_whisper_model():
"""Unload Whisper model to free memory."""
global _whisper_model
if _whisper_model is not None:
_whisper_model.unload_model()
backend = get_stt_backend()
backend.unload_model()
+38 -353
View File
@@ -1,372 +1,57 @@
"""
TTS inference module using Qwen3-TTS.
TTS inference module - delegates to provider abstraction layer.
"""
from typing import Optional, List, Tuple
import asyncio
import torch
from typing import Optional
import numpy as np
import io
import soundfile as sf
from pathlib import Path
from .utils.cache import get_cache_key, get_cached_voice_prompt, cache_voice_prompt
from .utils.audio import normalize_audio
from .utils.progress import get_progress_manager
from .utils.hf_progress import HFProgressTracker, create_hf_progress_callback
from .utils.tasks import get_task_manager
from . import config
from .backends import TTSBackend
from .providers import get_provider_manager
from .providers.base import TTSProvider
class TTSModel:
"""Manages Qwen3-TTS model loading and inference."""
def get_tts_model() -> TTSProvider:
"""
Get TTS provider instance (via ProviderManager).
def __init__(self, model_size: str = "1.7B"):
self.model = None
self.model_size = model_size
self.device = self._get_device()
self._current_model_size = None
def _get_device(self) -> str:
"""Get the best available device."""
if torch.cuda.is_available():
return "cuda"
elif hasattr(torch.backends, 'mps') and torch.backends.mps.is_available():
# MPS can have issues, use CPU for stability
return "cpu"
return "cpu"
def is_loaded(self) -> bool:
"""Check if model is loaded."""
return self.model is not None
def _get_model_path(self, model_size: str) -> str:
"""
Get the model path, downloading from HuggingFace Hub if needed.
Args:
model_size: Model size (1.7B or 0.6B)
Returns:
Path to model (either local or HuggingFace Hub ID)
"""
# HuggingFace Hub model IDs
hf_model_map = {
"1.7B": "Qwen/Qwen3-TTS-12Hz-1.7B-Base",
"0.6B": "Qwen/Qwen3-TTS-12Hz-0.6B-Base",
}
# Local directory names (for backwards compatibility)
local_model_map = {
"1.7B": "Qwen--Qwen3-TTS-12Hz-1.7B-Base",
"0.6B": "Qwen--Qwen3-TTS-12Hz-0.6B-Base",
}
if model_size not in hf_model_map:
raise ValueError(f"Unknown model size: {model_size}")
# Check if model exists locally (backwards compatibility)
local_path = config.get_models_dir() / local_model_map[model_size]
if local_path.exists():
print(f"Found local model at {local_path}")
return str(local_path)
# Use HuggingFace Hub model ID (will auto-download)
hf_model_id = hf_model_map[model_size]
print(f"Will download model from HuggingFace Hub: {hf_model_id}")
return hf_model_id
def load_model(self, model_size: Optional[str] = None):
"""
Lazy load the TTS model with automatic downloading from HuggingFace Hub.
The model will be automatically downloaded on first use and cached locally.
This works similar to how Whisper models are loaded.
Args:
model_size: Model size to load (1.7B or 0.6B)
"""
if model_size is None:
model_size = self.model_size
# If already loaded with correct size, return
if self.model is not None and self._current_model_size == model_size:
return
# Unload existing model if different size requested
if self.model is not None and self._current_model_size != model_size:
self.unload_model()
try:
from qwen_tts import Qwen3TTSModel
# Get model path (local or HuggingFace Hub ID)
model_path = self._get_model_path(model_size)
# Set up progress tracking
progress_manager = get_progress_manager()
model_name = f"qwen-tts-{model_size}"
# Check if model is being downloaded from HuggingFace Hub
if model_path.startswith("Qwen/"):
print(f"Loading TTS model {model_size} on {self.device}...")
# Start tracking download task
task_manager = get_task_manager()
task_manager.start_download(model_name)
# Initialize progress state to show download has started
progress_manager.update_progress(
model_name=model_name,
current=0,
total=1, # Set to 1 initially, will be updated by callback
filename="",
status="downloading",
)
# Set up progress callback
progress_callback = create_hf_progress_callback(model_name, progress_manager)
tracker = HFProgressTracker(progress_callback)
# Use progress tracker during download
with tracker.patch_download():
# Load the model - downloads will happen automatically with progress tracking
self.model = Qwen3TTSModel.from_pretrained(
model_path,
device_map=self.device,
torch_dtype=torch.float32 if self.device == "cpu" else torch.bfloat16,
)
# Mark as complete
progress_manager.mark_complete(model_name)
task_manager.complete_download(model_name)
else:
# Local model, no download needed
print(f"Loading TTS model {model_size} on {self.device}...")
self.model = Qwen3TTSModel.from_pretrained(
model_path,
device_map=self.device,
torch_dtype=torch.float32 if self.device == "cpu" else torch.bfloat16,
)
self._current_model_size = model_size
self.model_size = model_size
print(f"TTS model {model_size} loaded successfully")
except ImportError as e:
print(f"Error: qwen_tts package not found. Install with: pip install git+https://github.com/QwenLM/Qwen3-TTS.git")
progress_manager = get_progress_manager()
task_manager = get_task_manager()
model_name = f"qwen-tts-{model_size}"
progress_manager.mark_error(model_name, str(e))
task_manager.error_download(model_name, str(e))
raise
except Exception as e:
print(f"Error loading TTS model: {e}")
print(f"Tip: The model will be automatically downloaded from HuggingFace Hub on first use.")
progress_manager = get_progress_manager()
task_manager = get_task_manager()
model_name = f"qwen-tts-{model_size}"
progress_manager.mark_error(model_name, str(e))
task_manager.error_download(model_name, str(e))
raise
async def load_model_async(self, model_size: Optional[str] = None):
"""
Async version of load_model that runs in thread pool.
This prevents blocking the event loop during model loading.
"""
if model_size is None:
model_size = self.model_size
# If already loaded with correct size, return immediately
if self.model is not None and self._current_model_size == model_size:
return
# Run the blocking load operation in a thread pool
await asyncio.to_thread(self.load_model, model_size)
def unload_model(self):
"""Unload the model to free memory."""
if self.model is not None:
del self.model
self.model = None
self._current_model_size = None
if torch.cuda.is_available():
torch.cuda.empty_cache()
print("TTS model unloaded")
async def create_voice_prompt(
self,
audio_path: str,
reference_text: str,
use_cache: bool = True,
) -> Tuple[dict, bool]:
"""
Create voice prompt from reference audio.
Args:
audio_path: Path to reference audio file
reference_text: Transcript of reference audio
use_cache: Whether to use cached prompt if available
Returns:
Tuple of (voice_prompt_dict, was_cached)
"""
await self.load_model_async()
# Check cache if enabled
if use_cache:
cache_key = get_cache_key(audio_path, reference_text)
cached_prompt = get_cached_voice_prompt(cache_key)
if cached_prompt is not None:
return cached_prompt, True
def _create_prompt_sync():
"""Run synchronous voice prompt creation in thread pool."""
return self.model.create_voice_clone_prompt(
ref_audio=str(audio_path),
ref_text=reference_text,
x_vector_only_mode=False,
)
# Run blocking operation in thread pool
voice_prompt_items = await asyncio.to_thread(_create_prompt_sync)
# Cache if enabled
if use_cache:
cache_voice_prompt(cache_key, voice_prompt_items)
return voice_prompt_items, False
async def combine_voice_prompts(
self,
audio_paths: List[str],
reference_texts: List[str],
) -> Tuple[np.ndarray, str]:
"""
Combine multiple reference samples for better quality.
Args:
audio_paths: List of audio file paths
reference_texts: List of reference texts
Returns:
Tuple of (combined_audio, combined_text)
"""
from .utils.audio import load_audio
combined_audio = []
for audio_path in audio_paths:
audio, sr = load_audio(audio_path)
audio = normalize_audio(audio)
combined_audio.append(audio)
# Concatenate audio
mixed = np.concatenate(combined_audio)
mixed = normalize_audio(mixed)
# Combine texts
combined_text = " ".join(reference_texts)
return mixed, combined_text
async def generate(
self,
text: str,
voice_prompt: dict,
language: str = "en",
seed: Optional[int] = None,
instruct: Optional[str] = None,
) -> Tuple[np.ndarray, int]:
"""
Generate audio from text using voice prompt.
Args:
text: Text to synthesize
voice_prompt: Voice prompt dictionary from create_voice_prompt
language: Language code (en or zh)
seed: Random seed for reproducibility
instruct: Natural language instruction for speech delivery control
Returns:
Tuple of (audio_array, sample_rate)
"""
# Load model (already handles async via to_thread if needed)
await self.load_model_async()
def _generate_sync():
"""Run synchronous generation in thread pool."""
# Set seed if provided
if seed is not None:
torch.manual_seed(seed)
if torch.cuda.is_available():
torch.cuda.manual_seed(seed)
# Generate audio - this is the blocking operation
wavs, sample_rate = self.model.generate_voice_clone(
text=text,
voice_clone_prompt=voice_prompt,
instruct=instruct,
)
return wavs[0], sample_rate
# Run blocking inference in thread pool to avoid blocking event loop
audio, sample_rate = await asyncio.to_thread(_generate_sync)
return audio, sample_rate
async def generate_from_reference(
self,
text: str,
audio_path: str,
reference_text: str,
language: str = "en",
seed: Optional[int] = None,
) -> Tuple[np.ndarray, int]:
"""
Generate audio directly from reference (convenience method).
Args:
text: Text to synthesize
audio_path: Path to reference audio
reference_text: Transcript of reference audio
language: Language code
seed: Random seed
Returns:
Tuple of (audio_array, sample_rate)
"""
# Create voice prompt (with caching)
voice_prompt, _ = await self.create_voice_prompt(audio_path, reference_text)
# Generate
return await self.generate(text, voice_prompt, language, seed)
Returns:
TTS provider instance
"""
manager = get_provider_manager()
# Note: This is async but we need sync interface for backward compatibility
# In practice, this will be called from async contexts
import asyncio
try:
loop = asyncio.get_event_loop()
if loop.is_running():
# We're in an async context, but can't await here
# Return a wrapper that will use the provider manager
return manager._get_default_provider()
else:
return loop.run_until_complete(manager.get_active_provider())
except RuntimeError:
# No event loop, return default
return manager._get_default_provider()
# Global model instance
_tts_model: Optional[TTSModel] = None
def get_tts_model() -> TTSModel:
"""Get or create TTS model instance."""
global _tts_model
if _tts_model is None:
_tts_model = TTSModel()
return _tts_model
async def get_tts_model_async() -> TTSProvider:
"""
Get TTS provider instance asynchronously.
Returns:
TTS provider instance
"""
manager = get_provider_manager()
return await manager.get_active_provider()
def unload_tts_model():
"""Unload TTS model to free memory."""
global _tts_model
if _tts_model is not None:
_tts_model.unload_model()
manager = get_provider_manager()
provider = manager._get_default_provider()
provider.unload_model()
def audio_to_wav_bytes(audio: np.ndarray, sample_rate: int) -> bytes:

Some files were not shown because too many files have changed in this diff Show More