Compare commits

...
163 Commits
Author SHA1 Message Date
Jamie Pine eba1244add Bump version: 0.1.9 → 0.1.10 2026-01-29 23:12:16 -08:00
Jamie Pine 94487f32a5 Enhance MLX and PyTorch Backend Integration
- Added support for MLX backend on Apple Silicon, enabling optimized performance for TTS and STT tasks.
- Implemented platform detection to dynamically select between MLX and PyTorch based on the runtime environment.
- Updated build process to include MLX-specific dependencies and configurations for macOS.
- Refactored backend code to improve model loading and inference logic, accommodating backend-specific requirements.
- Enhanced documentation to clarify backend selection and performance benefits for different platforms.
- Streamlined installation instructions and troubleshooting guidance for MLX-related issues.
2026-01-29 23:11:48 -08:00
Jamie Pine 081f45e680 ADDED MLX FOR SUPER FAST GENERATIONS ON APPLE SILICON
- Added support for MLX backend on Apple Silicon, enabling optimized performance for TTS and STT tasks.
- Updated release workflow to include MLX-specific dependencies and configurations for macOS platforms.
- Refactored backend code to dynamically select between MLX and PyTorch based on the runtime environment.
- Enhanced model loading and inference logic to accommodate backend-specific requirements, including updated model IDs and hidden imports.
- Improved health check and model status reporting to reflect the active backend type.
- Streamlined caching mechanisms to support both backend types, ensuring compatibility and performance.
2026-01-29 21:50:46 -08:00
Jamie Pine 86768288ce Enhance MLX Audio Documentation and Testing Framework
- Updated MLX_AUDIO.md to reflect validated status and included detailed validation results, model mapping, and API usage examples.
- Added a demo script (demo.py) for testing audio generation speed and functionality.
- Introduced a test script (test_tts.py) to validate MLX audio model loading and generation, ensuring robust testing for future developments.
- Created a .gitignore file in the mlx-test directory to exclude unnecessary files from version control.
2026-01-29 21:28:22 -08:00
Jamie Pine 0fd063442a Remove risks and mitigations section from MLX_AUDIO.md and update open questions with responses for clarity. This streamlines the documentation and provides clearer guidance on future considerations. 2026-01-29 21:08:28 -08:00
Jamie Pine a0c2493e98 Add MLX Audio Integration for Apple Silicon Support
- Introduced a new backend for MLX audio to enable GPU acceleration on macOS Apple Silicon, improving performance and user experience.
- Implemented platform detection to switch between MLX and PyTorch backends based on the runtime environment.
- Added new streaming capabilities for TTS and STT, enhancing real-time audio generation.
- Updated API endpoints and frontend components to support new features while maintaining backward compatibility.
- Created documentation for backend integration and performance comparisons.
2026-01-29 20:57:52 -08:00
Jamie Pine b39f48cc81 Refactor useGenerationForm to streamline model download handling
- Removed unnecessary isDownloading variable and related logic.
- Consolidated model download state reset to the finally block for improved clarity and reliability.
- Enhanced error handling by ensuring model download state is reset in case of failure.
2026-01-29 20:17:55 -08:00
Jamie Pine 4ff775bc98 Update packageManager version in package.json to [email protected] 2026-01-29 19:53:38 -08:00
Jamie Pine 6351aa75e9 Refactor StoryList component for improved readability and organization
- Reorganized import statements for clarity and consistency.
- Adjusted formatting of state declarations for better readability.
- Streamlined JSX structure for improved visual hierarchy.
- Updated dialog descriptions for consistency in presentation.
- Made minor adjustments to spacing and layout for enhanced UI consistency.
2026-01-29 19:46:27 -08:00
Jamie Pine 43873a883b Update StoryList component styles for improved UI consistency
- Changed border radius of the "No stories yet" message to rounded-2xl for a softer appearance.
- Updated story item borders to rounded-2xl to enhance visual cohesion across the component.
2026-01-29 19:34:28 -08:00
Jamie Pine 60012b81c0 Refactor ProfileForm and SampleList components for improved UI and functionality
- Updated button styles in ProfileForm for better visual consistency and user experience.
- Replaced Pencil icon with Edit in SampleList for clearer action representation.
- Introduced CircleButton component for action buttons in SampleList, enhancing UI responsiveness and clarity.
- Improved layout and hover effects for action buttons in SampleList to streamline user interactions.
2026-01-29 19:32:05 -08:00
Jamie Pine 89f3127c37 Implement avatar upload and management for voice profiles
- Added functionality to upload, delete, and retrieve avatar images for voice profiles.
- Introduced new API endpoints for avatar management, including upload and delete operations.
- Enhanced profile forms and components to support avatar image handling, including previews and error handling.
- Updated database schema to include avatar_path for profiles and added necessary migrations.
- Implemented image validation and processing utilities to ensure proper avatar uploads.
2026-01-29 19:28:42 -08:00
Jamie Pine ef3c3a7f8c Refactor ProfileForm for improved readability and maintainability
- Reorganized import statements for clarity.
- Enhanced conditional checks for restoring saved files with improved formatting.
- Streamlined draft saving logic by consolidating variable declarations.
- Updated UI components for better structure and readability in the form layout.
2026-01-29 18:56:22 -08:00
Jamie Pine 7b5e73cfa8 Add .npmrc for bun usage and update dependencies
- Created a new .npmrc file to enforce bun usage.
- Bumped version numbers for multiple packages to 0.1.9 in bun.lock.
- Added react-sound-visualizer dependency to enhance audio visualization features.
- Introduced convert:assets script in package.json for asset optimization.
- Updated CONTRIBUTING.md with instructions for converting assets to web formats.
- Added documentation files for API endpoints and developer guidelines in the docs directory.
2026-01-29 18:56:10 -08:00
Jamie Pine 462f104494 Enhance ProgressManager for thread safety and event loop integration
- Added thread-safe mechanisms to the ProgressManager for handling model download progress updates.
- Introduced a main event loop setter to ensure safe operations from background threads.
- Improved listener notification to handle updates in a thread-safe manner.
- Updated methods to ensure thread safety when accessing progress data.
2026-01-29 16:23:46 -08:00
Jamie Pine fadb57164e Update README to reflect API endpoint changes and enhance profile creation example
- Updated API endpoints from `/api/...` to `/...` for consistency.
- Modified the speech generation example to include a language parameter.
- Revised the profile creation example to use JSON format instead of form data.
2026-01-29 16:14:19 -08:00
Jamie Pine e870d65136 Add badges to README for downloads, releases, stars, and license 2026-01-29 16:09:10 -08:00
Jamie PineandGitHub 3df40278cc Merge pull request #5 from Snowy7/fix/dev-mode-sidecar
Fix dev mode sidecar and cross-platform HuggingFace cache paths
2026-01-29 16:05:00 -08:00
Jamie PineandGitHub deeef5a474 Merge pull request #12 from tomasmach/feat/makefile
feat: add Makefile for streamlined development workflow
2026-01-29 16:04:48 -08:00
Jamie Pine 236e464525 Bump version: 0.1.8 → 0.1.9 2026-01-29 15:58:31 -08:00
Jamie Pine cf3cf3f002 Enhance model download handling in useGenerationForm and ProgressManager
- Introduced a flag to track download status in useGenerationForm, ensuring proper UI updates during model downloads.
- Updated ProgressManager to only send initial progress updates if the model is actively downloading or extracting, preventing outdated status messages from being sent.
- Improved error handling and logging for better visibility into model download processes.
2026-01-29 15:57:53 -08:00
tomasmach 9d98e1e768 fix: improve Makefile robustness and update CONTRIBUTING docs
- Add exit 1 to test-backend when pytest not installed
- Add exit 1 to test-frontend when no test script configured
- Add venv dependency to db-init target
- Document Makefile usage in CONTRIBUTING.md
2026-01-30 00:51:41 +01:00
Jamie Pine 3be8980f48 Refactor ProfileForm to support draft state management and improve file handling
- Introduced functionality to save and restore form state as a draft when creating a new voice profile.
- Added helper functions for converting files to and from base64 format to facilitate file handling.
- Updated the API types to use a more flexible LanguageCode type for language parameters.
- Enhanced the UI store to manage profile form drafts, improving user experience during profile creation.
2026-01-29 15:45:14 -08:00
tomasmach 76bc070f5b docs: update CHANGELOG with Makefile feature 2026-01-30 00:44:10 +01:00
tomasmach 01838f4773 docs: add Makefile reference and setup instructions to README 2026-01-30 00:35:20 +01:00
tomasmach 39e4f9d08c fix: correct backend server port to match frontend expectations (17493) 2026-01-30 00:33:50 +01:00
Jamie Pine 341d71470c Implement auto-scroll feature in StoryTrackEditor to keep playhead centered during playback
- Added a useEffect hook to automatically scroll the timeline when the playhead moves past the halfway point of the visible area, enhancing user experience during playback.
2026-01-29 15:32:34 -08:00
Jamie Pine bb6cea24ba Refactor StoryTrackEditor to account for time ruler height during drag operations
- Introduced a constant for TIME_RULER_HEIGHT to improve code readability.
- Updated drag position calculations to subtract the time ruler height, ensuring accurate positioning of clips relative to the tracks area.
2026-01-29 15:31:23 -08:00
Jamie Pine fa7ac88abc Reset playback timing anchors in story store for fresh initialization by playback hook 2026-01-29 15:27:27 -08:00
Jamie Pine c68ddc45b1 Enhance contribution guidelines and improve FloatingGenerateBox component
- Updated CONTRIBUTING.md to include instructions for building with a local Qwen3-TTS development version, facilitating easier testing and development.
- Refactored FloatingGenerateBox component to streamline the rendering of text and instruct fields, improving code readability and maintainability.
- Added functionality to handle auto-resizing of text areas based on content changes, enhancing user experience.
- Improved event handling for keyboard interactions in StoryTrackEditor, allowing for play/pause functionality with the spacebar.
- Introduced a MiniSamplePlayer component in SampleList for better audio playback control, including play, pause, and seek features.
- Implemented sample update functionality in the backend, allowing users to edit reference text for audio samples, with appropriate error handling and user feedback.
2026-01-29 15:25:40 -08:00
tomasmach f89dc66d0c feat: add Python version fallback (3.12 > 3.13 > python3) and compatibility warning 2026-01-30 00:10:30 +01:00
tomasmach cba7d7bc23 feat: add Makefile for streamlined development workflow 2026-01-30 00:05:36 +01:00
Jamie PineandGitHub 3c89b068f3 Merge pull request #6 from jamiepine/windows-server-shutdown
Windows server shutdown
2026-01-29 03:12:40 -08:00
Jamie Pine 229841e05e Add GPU type information to health check response
- Updated the health check endpoint to include the type of GPU available (CUDA or MPS).
- Modified the HealthResponse model to accommodate the new gpu_type field, enhancing the response with detailed GPU information.
- This change improves the clarity of system capabilities for users and developers.
2026-01-29 03:12:11 -08:00
Jamie Pine 123e8215e4 Merge branch 'main' into windows-server-shutdown 2026-01-29 03:00:08 -08:00
Jamie Pine 2a3afec2ca Implement graceful shutdown for the server and enhance process management on Windows
- Added a new `/shutdown` endpoint to allow graceful server shutdown via HTTP.
- Implemented process tree management functions to handle child processes during shutdown on Windows.
- Updated the `stop_server` function to attempt graceful shutdown before forcefully terminating processes.
- Enhanced error handling and logging for shutdown operations.
2026-01-29 02:58:21 -08:00
Jamie Pine 99ddd5a0b4 Add asynchronous model download handling for TTS and Whisper models
- Implemented background tasks for downloading TTS and Whisper models to prevent blocking HTTP responses.
- Enhanced error handling during model downloads, providing users with real-time feedback on download status.
- Updated HTTP responses to indicate when models are being downloaded, improving user experience during model initialization.
2026-01-29 02:55:17 -08:00
Jamie Pine 8d730621bc Refactor model download handling to use background tasks
- Moved model download logic into a separate asynchronous function to allow non-blocking HTTP responses.
- Improved error handling by tracking download status and reporting errors without interrupting the main request flow.
- The frontend is now expected to poll the progress endpoint for download status updates.
2026-01-29 02:42:02 -08:00
Jamie Pine e23118f610 Bump version: 0.1.7 → 0.1.8 2026-01-29 02:21:01 -08:00
Jamie Pine d4bfdc0d68 Update version handling in backend and improve HuggingFace cache management
- Added __version__ variable in backend/__init__.py to centralize versioning.
- Updated main.py to use __version__ for API versioning in the FastAPI app.
- Enhanced cache directory handling by utilizing HuggingFace's constants for improved compatibility across platforms.
2026-01-29 02:20:33 -08:00
Jamie Pine 116c108906 Update screenshot asset in landing page for consistency with current design 2026-01-29 00:06:06 -08:00
Jamie Pine 2d23c8e06a Swap screenshot assets in landing page for improved visual representation
- Replaced app screenshot paths to ensure correct images are displayed.
- Adjusted alt text for screenshots to accurately reflect their content.
2026-01-29 00:06:00 -08:00
Jamie Pine 3973a59ba3 Revise README to clarify Voicebox features and benefits
- Changed section title from "Why Voicebox?" to "What is Voicebox?" for better clarity.
- Expanded description to emphasize local-first voice cloning capabilities and professional tools.
- Highlighted privacy, model flexibility, and native performance as key advantages over cloud services.
2026-01-28 23:59:47 -08:00
Jamie Pine d9aa75253a Enhance README with new features and multi-track editor details
- Added multi-sample support for higher quality cloning.
- Introduced a new Stories Editor section with features for multi-track composition, inline audio editing, auto-playback, and voice mixing.
- Updated recording section to include system audio capture for macOS and Windows.
2026-01-28 23:55:44 -08:00
Jamie Pine b22bf36565 Update README and landing page with new screenshots; bump version to 0.1.7
- Replaced existing screenshot paths in README and landing page with new assets.
- Added additional screenshots to the landing page for enhanced visual representation.
- Updated version in Cargo.lock from 0.1.6 to 0.1.7.
2026-01-28 23:51:43 -08:00
Jamie Pine 33f4ed9b44 Bump version: 0.1.6 → 0.1.7 2026-01-28 22:28:18 -08:00
Jamie Pine cc37e04221 Refactor HistoryTable and SampleList components for improved code consistency
- Cleaned up formatting in HistoryTable for better readability.
- Adjusted import statements in SampleList to maintain consistent structure.
2026-01-28 22:28:01 -08:00
Jamie Pine 2b4fbe5173 Refactor AudioPlayer and related components to support conditional auto-play functionality
- Updated AudioPlayer to auto-play only if the shouldAutoPlay flag is set, enhancing user control over playback.
- Refactored HistoryTable, SampleList, and useGenerationForm to utilize setAudioWithAutoPlay for consistent audio loading and playback behavior.
- Improved user experience by ensuring audio is only played when explicitly intended, reducing unexpected playback.
2026-01-28 22:27:37 -08:00
Snowy 423d69b7cc Use HuggingFace's built-in cache detection for cross-platform support
Replace hardcoded ~/.cache/huggingface/hub paths with
huggingface_hub.constants.HF_HUB_CACHE which correctly handles
OS-specific cache locations (Windows uses AppData, etc.)
2026-01-29 09:25:41 +03:00
Jamie Pine ea943876dc formatting 2026-01-28 22:23:27 -08:00
Jamie Pine b55d8cc567 Implement auto-activation of stories in StoryTrackEditor and improve playback state management
- Added useEffect to automatically activate the story when the editor is shown, ensuring the playhead is visible.
- Introduced setActiveStory function in storyStore to manage story activation without playback.
- Updated playback state checks to reflect the current playing status accurately.
- Enhanced UI to always display the playhead for better user experience during playback.
2026-01-28 22:22:47 -08:00
Jamie Pine 036d90dc8e Enhance story item management with trimming, splitting, and duplication features
- Updated StoryTrackEditor and StoryContent components to support trimming and splitting of story items.
- Introduced new API endpoints for trimming, splitting, and duplicating story items, enhancing item management capabilities.
- Refactored related hooks and state management to accommodate new functionalities.
- Improved data models to include trim start and end times for better audio playback control.
- Enhanced UI interactions for selecting and managing story items within the track editor.
2026-01-28 22:16:53 -08:00
Snowy c513451277 Fix dev mode to work without pre-built server binary
Previously, running `bun run dev` would fail because Tauri requires
the sidecar binary to exist at compile time, even in development mode.
This forced developers to build the full PyInstaller binary before
they could start development.

This change introduces a streamlined dev workflow:

1. Add `scripts/setup-dev-sidecar.js` - Creates minimal placeholder
   binaries that satisfy Tauri's compile-time check. Works cross-platform
   (Windows PE stub, Unix shell script).

2. Update Rust code to gracefully handle dev mode - When the sidecar
   fails to start, it checks if a manually-started server is already
   running on the expected port and connects to it instead.

3. Update npm scripts - `bun run dev` now auto-runs the setup script,
   and `dev:server` uses the correct port (17493).

4. Update CONTRIBUTING.md with clearer dev workflow documentation.

New development workflow:
  Terminal 1: bun run dev:server
  Terminal 2: bun run dev

The bundled binary is only required for production builds.
2026-01-29 09:11:41 +03:00
Jamie PineandGitHub 27ae6dfbab Merge pull request #3 from jamiepine/stories
Stories
2026-01-28 21:18:19 -08:00
Jamie Pine 51b9e2fd3d Bump version: 0.1.5 → 0.1.6 2026-01-28 20:48:48 -08:00
Jamie Pine be25ddbe0e Refactor FloatingGenerateBox for improved code organization and readability
- Cleaned up import statements for better structure and consistency.
- Adjusted formatting and spacing in the FloatingGenerateBox component for enhanced readability.
- Streamlined the use of hooks and state management within the component.
- Ensured consistent styling and layout adjustments for better user experience.
2026-01-28 20:48:45 -08:00
Jamie Pine 9cd4921291 Enhance FloatingGenerateBox with auto-resizing textarea and default voice selection
- Added auto-resizing functionality to the textarea in FloatingGenerateBox, improving user experience when inputting text.
- Implemented logic to set the first voice profile as default if none is selected, ensuring a smoother workflow.
- Updated StoryContent to remove hardcoded height for the generate box, simplifying layout calculations.
- Refactored StoryTrackEditor to improve background styling for better visual consistency.
2026-01-28 20:48:13 -08:00
Jamie Pine 2349bd24ba Enhance FloatingGenerateBox and StoryContent with new features and improved UI
- Refactored FloatingGenerateBox to improve layout and ensure consistent styling for the voice selector.
- Added a popover component to StoryContent for adding generations, including search functionality for better user experience.
- Implemented story item editing and deletion capabilities in StoryList, enhancing story management features.
- Updated import statements and added new hooks for better organization and functionality across components.
2026-01-28 20:39:09 -08:00
Jamie Pine cd82ed0664 Refactor FloatingGenerateBox and StoriesTab for improved layout and interaction
- Adjusted FloatingGenerateBox positioning to align with the story list, ensuring consistent UI across different routes.
- Modified StoriesTab layout to enhance responsiveness, including setting a maximum width for the story list and adjusting the right column for better content display.
- Streamlined StoryChatItem interaction by simplifying the play functionality, allowing double-click to trigger playback directly from the text area.
- Enhanced StoryContent component by cleaning up unused playback controls and improving overall structure for better readability.
2026-01-28 20:31:17 -08:00
Jamie Pine c4884a0443 Enhance StoryContent and StoryTrackEditor for improved playback and UI dynamics
- Added auto-scrolling functionality to StoryContent for the currently playing item, enhancing user experience during playback.
- Refactored StoryTrackEditor to dynamically calculate container width, ensuring proper layout for varying story lengths.
- Updated audio playback management to improve timing anchor handling and playback scheduling.
- Cleaned up import statements for better organization and readability across components.
2026-01-28 20:15:59 -08:00
Jamie Pine 232d231788 Refactor story management components and enhance track editor integration
- Updated AppFrame to conditionally render StoryTrackEditor based on the selected story and route.
- Modified FloatingGenerateBox to adjust its position based on the visibility of the track editor.
- Improved StoriesTab by removing direct track editor rendering and relying on the new store state for height management.
- Enhanced StoryContent to dynamically calculate bottom padding based on the track editor's height.
- Introduced trackEditorHeight state in storyStore for better UI management of the track editor's visibility and size.
2026-01-28 19:50:46 -08:00
Jamie Pine 1cf90c81dd Enhance story item management with track editing functionality
- Introduced StoryTrackEditor component for managing story item positions and tracks.
- Updated StoriesTab to conditionally render the track editor based on selected story.
- Implemented moveStoryItem API endpoint to handle item repositioning and track changes.
- Enhanced story item data model to include track information.
- Improved audio playback management to support multiple tracks using Web Audio API.
- Added hooks for moving story items and managing playback timing.
2026-01-28 19:35:53 -08:00
Jamie Pine 3204e193fa Implement story management features and update dependencies
- Introduced story management functionality, including creating, listing, and managing story items.
- Added new components for story display and interaction, including StoriesTab, StoryList, and StoryContent.
- Integrated drag-and-drop functionality for reordering story items using @dnd-kit.
- Updated dependencies for @dnd-kit packages to enhance drag-and-drop capabilities.
- Bumped version for @voicebox/app, @voicebox/landing, @voicebox/tauri, and @voicebox/web to 0.1.5.
- Enhanced audio playback features to support story mode with auto-play functionality.
- Improved error handling and user feedback through toast notifications in story-related actions.
2026-01-28 19:10:58 -08:00
Jamie Pine 9d5d6cb56a Update UpdateStatus component logic and bump voicebox version to 0.1.5
- Modified conditional rendering in UpdateStatus to display content when the status is ready to install, improving user feedback.
- Bumped voicebox package version from 0.1.4 to 0.1.5 for dependency updates.
2026-01-28 15:14:23 -08:00
Jamie Pine 153eaba5f3 Refactor UpdateStatus component for improved UI and code organization
- Adjusted the order of import statements for better readability.
- Updated the conditional rendering logic to display a different message when an update is not ready to install, enhancing user feedback.
2026-01-28 15:14:05 -08:00
Jamie Pine 3370e3b419 Bump version: 0.1.4 → 0.1.5 2026-01-28 14:38:43 -08:00
Jamie Pine 615bd188a0 Refactor import statements in SampleUpload component for improved readability
- Adjusted the order of imports in SampleUpload.tsx to follow a more conventional structure, enhancing code organization.
2026-01-28 14:38:31 -08:00
Jamie Pine 7208f51eee Refactor audio generation components and improve debugging capabilities
- Introduced useGenerationForm hook to streamline audio generation form handling, including validation and model download management.
- Updated FloatingGenerateBox and GenerationForm components to utilize the new hook, enhancing code organization and reducing duplication.
- Replaced console logging with a debug utility for better logging control during audio playback and generation processes.
- Improved error handling in HistoryTable and MainEditor components by integrating toast notifications for user feedback.
- Adjusted audio recording duration limits across various components for consistency.
2026-01-28 14:30:33 -08:00
Jamie Pine 07a91a2381 Update app version to 0.1.4 and integrate @tanstack/react-router for improved routing functionality
- Bumped version for @voicebox/app, @voicebox/landing, @voicebox/tauri, and @voicebox/web to 0.1.4.
- Added @tanstack/react-router dependency for enhanced routing capabilities.
- Refactored App component to utilize RouterProvider for routing management.
- Created a new router configuration in router.tsx to define application routes and layout.
- Updated Sidebar component to use Link from @tanstack/react-router for navigation.
2026-01-28 00:00:18 -08:00
Jamie Pine 70cc36857d Bump version: 0.1.3 → 0.1.4 2026-01-27 17:25:15 -08:00
Jamie PineandGitHub 7f66d02591 Merge pull request #2 from jamiepine/channels
Channels
2026-01-27 17:23:58 -08:00
Jamie Pine 9f7a5a492e Refactor ConnectionForm and Checkbox component for improved functionality and UI
- Updated ConnectionForm to utilize a Checkbox component for managing the "keep server running" setting, enhancing user interaction.
- Refactored Checkbox component to use a button element for better accessibility and visual feedback.
- Streamlined import statements and improved code organization across multiple components for better readability.
2026-01-27 17:23:13 -08:00
Jamie Pine cb44377b09 Remove CheckCircle2 icon from various components for a cleaner UI
- Eliminated CheckCircle2 icon from ModelManagement, ModelProgress, ServerStatus, and UpdateStatus components to streamline the visual presentation.
- Updated import statements accordingly to reflect the removal of unused icons.
2026-01-27 17:14:17 -08:00
Jamie Pine a42a946586 Refactor UI components for improved layout consistency and responsiveness
- Updated App and AudioTab components to utilize TOP_SAFE_AREA_PADDING and BOTTOM_SAFE_AREA_PADDING constants for better layout adjustments.
- Enhanced HistoryTable and ModelManagement components for improved visual consistency.
- Streamlined import statements and component structure across various files for better organization and readability.
2026-01-27 17:12:16 -08:00
Jamie Pine a3cbe7f2b6 Refactor UI layout and introduce safe area constants for improved responsiveness
- Updated App, AppFrame, and AudioTab components to utilize new TOP_SAFE_AREA_PADDING and BOTTOM_SAFE_AREA_PADDING constants for consistent layout adjustments.
- Enhanced Sidebar and MainEditor components for better organization and user experience.
- Improved VoicesTab and ProfileList components by integrating new layout features and removing redundant import functionality.
- Streamlined HistoryTable and ModelManagement components for better visual consistency and interaction.
2026-01-27 17:11:38 -08:00
Jamie Pine d8d9eeaa6a Refactor App layout and introduce new components for improved organization
- Replaced the main layout in App component with AppFrame for better structure.
- Introduced MainEditor component to encapsulate the main editing interface, including ProfileList and HistoryTable.
- Added ModelsTab component to manage model-related functionalities.
- Updated Sidebar to include a new Models tab for navigation.
- Removed unused components and streamlined the layout for enhanced user experience.
2026-01-27 16:38:37 -08:00
Jamie Pine f7cb219f6d Refactor AudioPlayer for improved native playback handling and debugging
- Introduced a stop flag mechanism to manage audio playback more effectively.
- Enhanced native playback logic to ensure proper stopping of existing streams before starting new playback.
- Updated error handling and logging for better visibility during playback operations.
- Refactored audio output handling in Rust to support stopping playback and outputting silence when required.
- Improved the integration of native playback with WaveSurfer for seamless audio visualization.
2026-01-27 16:25:38 -08:00
Jamie Pine 7f18c09628 Enhance AudioPlayer component for native playback and debugging
- Improved the useNativePlayback logic to include detailed console logging for better debugging.
- Updated auto-play functionality to fetch runtime profile channels and channels, ensuring accurate playback decisions.
- Refactored audio playback handling to support native audio routing with enhanced error handling and logging.
- Introduced a new MultiSelect component for improved channel selection in VoicesTab.
- Updated FloatingGenerateBox to include selectedProfileId in audio setting.
- Added new dependencies for audio processing in Cargo.toml and Cargo.lock.
2026-01-27 16:15:16 -08:00
Jamie Pine d9c7121c5b Merge branch 'main' into channels 2026-01-27 15:24:18 -08:00
Jamie Pine 008b58f91c put back gen files for now 2026-01-27 14:08:30 -08:00
Jamie Pine c7404411d5 Bump version: 0.1.2 → 0.1.3 2026-01-27 13:46:21 -08:00
Jamie Pine ac08c4fcf4 Remove unused .gitkeep files and adjust ProfileList component padding
- Deleted .gitkeep files from Generation, ServerSettings, VoiceProfiles, and hooks directories as they are no longer necessary.
- Updated the ProfileList component to increase bottom padding for improved layout consistency.
2026-01-27 13:46:08 -08:00
Jamie Pine 3ce7498495 Remove unused schema files and update .gitignore
- Deleted several JSON schema files related to capabilities and desktop configurations that are no longer needed.
- Updated .gitignore to exclude the entire generated assets directory instead of a specific file, improving version control management.
2026-01-27 13:42:35 -08:00
Jamie Pine ce2f09d29e Refactor useAutoUpdater hook for improved readability and update handling
- Rearranged import statements for better organization.
- Enhanced download progress handling logic for clarity and consistency.
- Ensured proper state updates during the download process, including setting download progress to 100% upon completion.
2026-01-27 13:18:33 -08:00
Jamie Pine f1be633dca Implement FloatingGenerateBox component and update App layout
- Introduced the FloatingGenerateBox component for audio generation, enhancing user interaction with voice profiles.
- Updated App component to integrate FloatingGenerateBox and removed the GenerationForm component.
- Enhanced the layout for better responsiveness and added functionality to manage audio playback state.
- Updated UpdateStatus component to include new update handling logic and improved UI feedback for update readiness.
2026-01-27 13:18:12 -08:00
Jamie Pine 5f58c4dc3d Refactor VoiceProfiles components and remove ProfileDetail
- Removed the ProfileDetail component to streamline the ProfileCard functionality.
- Updated ProfileCard to eliminate the detail view and associated state management.
- Enhanced ProfileForm to manage audio samples more effectively, including improved UI for sample management.
- Adjusted SampleList to ensure proper button types for better accessibility.
2026-01-26 22:28:29 -08:00
Jamie Pine 892f363e3a Update development server port and enhance GPU status reporting
- Changed the development server port in package.json from 17493 to 8000 for dev.
- Improved GPU availability checks in main.py to include support for MPS on Apple Silicon.
- Removed the Assets.car file from version control as it is no longer needed.
- Updated .gitignore to reflect the removal of Assets.car.
2026-01-26 22:18:59 -08:00
Jamie Pine 535cf362de Bump version: 0.1.1 → 0.1.2 2026-01-26 21:04:11 -08:00
Jamie Pine f1116e05a6 Merge branch 'main' of https://github.com/jamiepine/voicebox 2026-01-26 21:01:55 -08:00
Jamie Pine f9aca9d418 Update development server port in package.json
- Changed the port for the development server from 8000 to 17493 to avoid conflicts and improve accessibility during local development.
2026-01-26 21:01:27 -08:00
Jamie Pine d913a9ae2a Implement audio format conversion and enhance recording completion handling
- Added a new utility function to convert audio blobs to WAV format, ensuring compatibility without requiring ffmpeg on the backend.
- Updated the useAudioRecording hook to convert recorded audio from WebM to WAV upon completion, with error handling for conversion failures.
- Improved the organization of imports in useAudioRecording for better readability.
2026-01-26 20:41:18 -08:00
Jamie Pine 36031a0df5 Enhance audio capture functionality and update dependencies
- Added support for capturing system audio on Windows using WASAPI with improved error handling and thread safety.
- Introduced the 'scopeguard' crate for better resource management during audio capture.
- Updated Cargo.toml to include 'scopeguard' and modified Windows-specific dependencies for enhanced functionality.
- Added a new test for validating audio capture output, ensuring the captured audio data is valid and non-empty.
2026-01-26 20:40:10 -08:00
Jamie Pine e59f86aa63 Refactor audio capture error handling and cleanup logic
- Removed console logging from the useSystemAudioCapture hook to streamline the code.
- Introduced error handling in the audio capture state to capture and report errors more effectively.
- Updated the cleanup logic to ensure proper handling of errors during audio capture on unmount.
- Enhanced error messages for better clarity when audio capture fails.
2026-01-26 20:12:37 -08:00
Jamie Pine b59c0f44e5 Enhance audio capture functionality in useSystemAudioCapture hook
- Added isRecordingRef to track recording state more reliably.
- Implemented console logging for key actions in startRecording and cancelRecording functions to aid in debugging.
- Updated cleanup logic on component unmount to ensure proper cancellation of recording if still active.
- Refactored condition checks to utilize isRecordingRef for improved performance and clarity.
2026-01-26 20:02:39 -08:00
Jamie Pine f8c5e54962 Add audio input entitlement and enhance audio sample extraction logic
- Added the `com.apple.security.device.audio-input` entitlement to the Entitlements.plist for improved audio capture capabilities.
- Refactored the audio sample extraction logic in macOS to handle both interleaved and planar audio formats, improving sample processing and interleaving of channels.
- Updated the Assets.car file to reflect changes in the audio capture implementation.
2026-01-26 19:39:09 -08:00
Jamie Pine 8b67faf96d Enhance update status display and audio sample components
- Improved the UpdateStatus component to show download progress and total bytes downloaded during updates.
- Refactored AudioSampleRecording and AudioSampleSystem components for cleaner button rendering and consistent layout.
- Updated import order in SampleUpload component for better organization.
2026-01-26 19:22:43 -08:00
Jamie Pine 30ea627ae8 Implement audio channel management features
- Added new components for managing audio channels, including creation, updating, and deletion of channels.
- Introduced a new AudioTab for channel management and integrated it into the main application layout.
- Updated the API client to support audio channel operations and added corresponding backend endpoints.
- Enhanced the player store to handle audio playback routing through assigned channels.
- Refactored existing components to accommodate the new audio channel functionality, including updates to the HistoryTable and GenerationForm for profile-channel associations.
- Improved sidebar navigation to include new tabs for Voices and Audio management.
2026-01-26 19:20:20 -08:00
Jamie Pine cd77b80b4f Refactor Windows audio capture implementation
- Introduced AtomicBool for stop signal handling to improve thread safety.
- Updated audio capture logic to utilize WASAPI more effectively, including error handling and buffer management.
- Enhanced the spawn mechanism for audio capture tasks to ensure compatibility with non-Send types.
- Added a new dependency on the 'windows' crate in Cargo.lock for improved functionality.
2026-01-26 18:31:08 -08:00
Jamie Pine 12174010f9 Fix Windows audio capture API compatibility with wasapi 2026-01-26 17:56:18 -08:00
Jamie Pine 57c68040bc Remove bumpversion dependency from backend requirements 2026-01-26 17:41:25 -08:00
Jamie Pine d65704aa68 Enhance icon generation and update dependencies
- Added support for generating a multi-size Windows icon (icon.ico) in the update-icons.sh script.
- Updated Cargo.toml to include the 'windows' crate with specific features for Windows support.
- Bumped version of the 'voicebox' package from 0.1.0 to 0.1.1 in Cargo.lock.
- Updated .gitignore to exclude the generated Assets.car file.
2026-01-26 17:39:02 -08:00
Jamie Pine 83a6aca1ff Bump version: 0.1.0 → 0.1.1 2026-01-26 17:30:05 -08:00
Jamie Pine f18abc0da6 Add .bumpversion.cfg for version management
- Introduced .bumpversion.cfg to automate versioning across multiple files.
- Configured version updates for tauri.conf.json, Cargo.toml, and various package.json files.
- Set up commit and tag generation for new releases, streamlining the release process.
2026-01-26 17:29:06 -08:00
Jamie Pine 0f616ff1c1 Update CONTRIBUTING.md to include bumpversion instructions and enhance release process clarity; add bumpversion to backend requirements 2026-01-26 17:26:50 -08:00
Jamie Pine 7c4b1d4dd2 Remove UpdateNotification component and its usage in App.tsx
- Deleted the UpdateNotification component to streamline the application.
- Removed its invocation from the App component, enhancing the overall code clarity.
- Updated Assets.car file to reflect changes in the application structure.
2026-01-26 17:23:18 -08:00
Jamie Pine 48acc10422 Refactor Tauri server management and improve logging
- Updated the server management logic in main.rs to include the PID of the existing voicebox server when reusing it, enhancing clarity in logging.
- Improved the formatting of imports in App.tsx for better readability.
2026-01-26 17:20:46 -08:00
Jamie Pine 1fdf61ca2e Implement server reuse logic and improve build script formatting
- Added logic in main.rs to check for an existing voicebox server running on the designated port, allowing reuse of the server if found.
- Enhanced the build.rs script by improving formatting and readability of the Swift library path definitions.
- Updated logging to provide clearer warnings when the icon source is not found during the build process.
2026-01-26 17:17:57 -08:00
Jamie Pine 6a0601bd6c Update server URL handling and improve logging
- Changed the server URL from 'http://localhost:8000' to 'http://127.0.0.1:17493' in the server store and connection form.
- Enhanced server startup logging to display the dynamically assigned server URL.
- Updated server management logic in main.rs to reflect the new port configuration and improve orphaned process handling.
2026-01-26 17:13:13 -08:00
Jamie Pine 88d41f342b Implement server running preference and cleanup on exit
- Added `setKeepServerRunning` function to manage server persistence on app close.
- Integrated server running preference in `ConnectionForm` and synced settings on app startup.
- Enhanced server management in `main.rs` to handle orphaned processes based on user preference.
- Improved cleanup logic to ensure proper termination of server processes when not set to keep running.
2026-01-26 17:02:00 -08:00
Jamie Pine b1cf7926c7 Add audio sample components for recording, system capture, and upload
- Introduced `AudioSampleRecording`, `AudioSampleSystem`, and `AudioSampleUpload` components to handle audio recording, system audio capture, and file uploads respectively.
- Implemented play/pause functionality for audio playback across all components, enhancing user interaction.
- Refactored `ProfileForm` and `SampleUpload` to utilize new audio components, improving code organization and maintainability.
- Added hooks for audio playback management, ensuring consistent audio handling and cleanup across the application.
2026-01-26 16:44:49 -08:00
Jamie Pine 834323068d Enhance audio sample handling in ProfileForm and SampleUpload components
- Introduced play/pause functionality for audio samples, allowing users to preview uploaded audio files.
- Added drag-and-drop support for file uploads, improving user experience when selecting audio files.
- Refactored audio validation and error handling to ensure proper feedback for audio file requirements.
- Updated UI elements for better clarity and consistency in audio file management.
2026-01-26 16:38:52 -08:00
Jamie Pine 8cd868d33f Enhance audio recording functionality to improve duration handling
- Updated getAudioDuration function to utilize recordedDuration property for files, addressing metadata issues on Windows.
- Modified onRecordingComplete callbacks in audio recording hooks to pass the actual recorded duration.
- Adjusted error handling in ProfileForm and SampleUpload components to clear validation errors for recorded files.
- Ensured consistent handling of audio file duration across components.
2026-01-26 16:22:31 -08:00
Jamie Pine 446182e16c macos audio capture for sample creation 2026-01-26 16:05:42 -08:00
Jamie Pine c7c401b98c Refactor getLatestRelease function to improve file filtering for downloads
- Added logic to skip non-downloadable files such as signature, JSON, and text files.
- Updated conditions for identifying downloadable files for macOS, Windows, and Linux platforms to use `endsWith` for better accuracy.
2026-01-26 01:58:06 -08:00
Jamie Pine 595d735143 Update landing page to remove Linux support and adjust platform descriptions
- Modified metadata and various components to reflect the removal of Linux support, focusing on macOS and Windows.
- Updated descriptions in the layout, page, and footer components for consistency.
- Adjusted download links and icons to align with the new platform availability.
2026-01-26 01:56:44 -08:00
Jamie Pine 85935c1bbb Update HistoryTable component to improve user feedback for empty history state
- Modified the message displayed when there are no voice generations to be more concise and user-friendly.
- Adjusted the styling of the empty state message for better visual appeal.
2026-01-26 01:17:59 -08:00
Jamie Pine 8058360744 Update README.md to enhance demo video visibility
- Wrapped the app screenshot in a link to the demo video on voicebox.sh for better accessibility.
- Added a descriptive text below the image to encourage users to click and watch the demo video.
2026-01-26 01:11:59 -08:00
Jamie Pine 47e4da7ce2 Update date formatting logic and replace binary assets
- Enhanced the formatDate function to handle date strings without timezone information by treating them as UTC.
- Updated binary assets including screenshots and application images for improved visual representation.
2026-01-26 01:10:22 -08:00
Jamie Pine b7ab4410a6 Update README.md to reflect new download links and remove Linux support
- Updated download links for macOS and Windows, including new file formats and naming conventions.
- Removed Linux download option, with a note indicating that Linux builds are coming soon due to GitHub runner disk space limitations.
2026-01-26 00:58:14 -08:00
Jamie Pine afd0381243 Update landing page metadata and reintroduce Header component
- Changed the title in metadata to reflect the open-source nature of the app.
- Reorganized the import statements to include the Header component for better structure.
- Streamlined the apple touch icon configuration in metadata.
2026-01-26 00:52:03 -08:00
Jamie Pine 2ceccaec51 Refactor global styles to utilize Tailwind CSS directives
- Replaced direct imports of Tailwind CSS with @tailwind directives for base, components, and utilities.
- Improved organization of global styles for better maintainability and adherence to Tailwind CSS conventions.
2026-01-26 00:45:03 -08:00
Jamie Pine 551abc9856 Enhance history management with export and import functionalities
- Added endpoints for exporting generations as ZIP archives and audio files.
- Implemented import functionality for generations from ZIP archives with validation.
- Updated HistoryTable component to support new export and import features.
- Improved error handling and user notifications for export/import processes.
- Refactored related hooks and API client methods to accommodate new functionalities.
2026-01-26 00:30:37 -08:00
Jamie Pine 333cb262e0 Add voice profile import functionality with file size validation
- Implemented a new endpoint to import voice profiles from ZIP archives.
- Added file size validation to ensure uploads do not exceed 100MB.
- Enhanced error handling for various exceptions during the import process.
- Cleaned up the import function by removing the previous implementation.
2026-01-26 00:08:27 -08:00
Jamie Pine 04bc1aded4 Implement active task management for downloads and generations, enhancing user experience with toast notifications for ongoing tasks. Refactor language handling in forms to support multiple languages. Update audio player to manage restart functionality and improve sidebar icon representation. Adjust progress tracking for model downloads in the backend. 2026-01-26 00:00:00 -08:00
Jamie Pine b2659e6a6d Refactor App and Sidebar components to support macOS, add TitleBarDragRegion for improved window dragging, and enhance model management with delete functionality and download progress tracking. Update audio player to handle audio resets more effectively and improve generation form with model download notifications. 2026-01-25 23:25:21 -08:00
Jamie Pine 090b1f6dde Add granular import logging and increase timeout to 120s 2026-01-25 22:25:27 -08:00
Jamie Pine d943e1d6d4 Remove all module exclusions 2026-01-25 22:01:57 -08:00
Jamie Pine d1273c3d33 Remove aggressive module exclusions - torch and stdlib need them 2026-01-25 22:01:38 -08:00
Jamie Pine 1ce62e8b15 Enhance contribution guidelines and remove outdated setup documentation
- Updated CONTRIBUTING.md with detailed setup instructions for Bun, Python, and Rust.
- Removed SETUP.md as its content is now integrated into CONTRIBUTING.md.
- Adjusted README.md to reference the new contribution guidelines.
- Improved the HistoryTable component for better accessibility and user interaction.
- Updated global styles and landing page elements for improved aesthetics and functionality.
2026-01-25 21:50:54 -08:00
Jamie Pine 240b9b71a7 Remove torch.testing exclusion - needed by torch internals 2026-01-25 21:44:12 -08:00
Jamie Pine 9396c6c86d better docs 2026-01-25 21:42:38 -08:00
Jamie Pine 82431dc5f5 Update demo video link in README.md to direct users to the official website for better accessibility 2026-01-25 21:33:36 -08:00
Jamie Pine f6e5111f68 Replace demo video section in README.md with a clickable image link for improved user experience 2026-01-25 21:30:40 -08:00
Jamie Pine 658e967558 Update demo video source in README.md to use the correct GitHub raw URL for improved accessibility 2026-01-25 21:29:35 -08:00
Jamie Pine 777e73f195 Update demo video source in README.md to use the raw GitHub URL for improved accessibility 2026-01-25 21:28:26 -08:00
Jamie Pine c7004d5776 Add demo video section to README and landing page
- Introduced a new section in README.md showcasing a demo video for Voicebox.
- Added a demo video section in the landing page with responsive design and improved aesthetics.
- Updated video source paths for consistency across the application.
2026-01-25 21:27:15 -08:00
Jamie Pine 5d731d900b Simplify jaraco imports - use collect-submodules only 2026-01-25 21:23:31 -08:00
Jamie Pine fd89831d83 Fix PyInstaller missing jaraco dependencies 2026-01-25 21:14:03 -08:00
Jamie Pine b4b3762ef0 Update global styles and enhance landing page layout
- Added overflow-x hidden to prevent horizontal scrolling on html and body elements.
- Centered the heading and paragraph text on the landing page for improved aesthetics.
- Introduced a mobile-friendly centered screenshot above download buttons.
- Updated comments for clarity regarding screenshot positioning on different devices.
2026-01-25 21:06:41 -08:00
Jamie Pine af2f37ccb1 Enhance server startup logging and error handling
- Implemented detailed logging for server startup in server.py, including Python version, executable path, and parsed arguments.
- Added error handling for module imports and server initialization to improve robustness.
- Introduced an Entitlements.plist file for macOS to manage security settings.
- Updated tauri.conf.json to reference the new Entitlements.plist.
- Enhanced error reporting in main.rs for better debugging during server process management.
2026-01-25 20:55:13 -08:00
Jamie Pine 5feda1519c Add Apple API key installation and codesigning certificate setup in release workflow
- Introduced steps to install the Apple API key and codesigning certificate for macOS platforms.
- Enhanced the release workflow to support secure handling of Apple credentials for code signing.
- Updated environment variables to include necessary Apple signing information for Tauri builds.
2026-01-25 20:09:15 -08:00
Jamie Pine e56bfdc694 Update download links in constants.ts to use versioned filenames for consistency with release structure 2026-01-25 20:07:02 -08:00
Jamie Pine caff18dabe Update download links and repository URL in constants.ts to reflect the correct GitHub username 2026-01-25 19:47:38 -08:00
Jamie Pine 36bd2d2656 Update README.md to change link text for consistency 2026-01-25 19:37:25 -08:00
Jamie Pine 9ca0f43afb Update README.md 2026-01-25 19:34:36 -08:00
Jamie Pine c14fb937ef Revamp README.md for improved clarity and presentation
- Updated the README to feature a new layout with centered headings and images for better visual appeal.
- Enhanced the introduction to clearly define Voicebox as an open-source voice synthesis studio.
- Added sections for API usage, tech stack, and roadmap to provide comprehensive information about the project.
- Removed outdated content and streamlined the structure for easier navigation and understanding.
2026-01-25 19:32:30 -08:00
Jamie Pine 7d43938b49 Update application assets and modify screenshot references
- Replaced the old AppScreenshot.webp with a new VoiceBoxAppScreenshot.webp.
- Removed the obsolete AppScreenshot.webp file.
- Updated the landing page to reference the new VoiceBoxAppScreenshot.webp.
- Commented out the ReactQueryDevtools import in main.tsx for cleaner code.
2026-01-25 19:30:11 -08:00
Jamie Pine a7463968e4 Add creator attribution to settings tab in App component 2026-01-25 19:15:33 -08:00
Jamie Pine d9de8f04f2 Refactor components and implement generation state management
- Updated import order in App component for consistency.
- Enhanced Sidebar component with loading indicator for audio generation state.
- Integrated generation state management using Zustand in GenerationForm and Sidebar.
- Improved ProfileList component formatting for better readability.
- Added new generationStore for managing audio generation state across components.
2026-01-25 19:13:23 -08:00
Jamie Pine 1f075c1c15 Enhance App component with loading messages and UI improvements
- Added a loading message feature that cycles through various messages while the server is starting in Tauri.
- Improved the loading screen UI with a new layout and animations for the voicebox logo and loading text.
- Refactored the App component to include necessary imports and state management for loading messages.
- Updated styles for better visual appeal and user experience during the loading phase.
2026-01-25 19:07:21 -08:00
Jamie Pine 9125a4abe0 Add profile export and import functionality
- Implemented API endpoints for exporting and importing voice profiles as ZIP archives.
- Enhanced the frontend with new hooks and components for profile export and import, including file handling and user dialogs.
- Integrated Tauri plugins for file system access and dialog interactions to facilitate seamless user experience.
- Updated ProfileCard and ProfileList components to support new export and import features, improving overall functionality.
- Added necessary error handling and validation for file operations to ensure robustness.
2026-01-25 19:00:16 -08:00
Jamie Pine c62f615162 Implement sample audio retrieval and update playback functionality
- Added a new API endpoint to serve profile sample audio files.
- Introduced a method in the apiClient to generate sample audio URLs.
- Refactored the SampleList component to utilize the new API for audio playback, enhancing the user experience.
- Cleaned up unused imports and optimized the handlePlay function for better performance.
2026-01-25 18:44:25 -08:00
Jamie Pine 6acad47335 Enhance GenerationForm to autoplay generated audio
- Integrated audio playback functionality by utilizing the apiClient to fetch audio URLs.
- Updated the GenerationForm to set audio state with the generated audio details after successful generation.
- Improved user experience by automatically playing the generated audio upon completion.
2026-01-25 18:33:41 -08:00
Jamie Pine 75520e0c29 Refactor landing page layout and update assets
- Replaced the Hero component with a new section layout for improved structure and responsiveness.
- Updated download buttons for macOS, Windows, and Linux with enhanced styling.
- Added a new application screenshot in WebP format and removed the old App.webp asset.
- Adjusted Header component to capitalize the application name for consistency.
- Minor formatting improvements in HistoryTable for better readability.
2026-01-25 18:14:36 -08:00
Jamie Pine e6a05f7208 Update DownloadSection to link to macArm download for improved compatibility 2026-01-25 17:54:00 -08:00
Jamie Pine 57880fc2c7 Update Tauri configuration for autoupdater with new signing key
- Replaced the existing public key in the updater configuration with a new key for enhanced security.
- Maintained the endpoint for fetching update information from the GitHub releases.
2026-01-25 17:47:18 -08:00
Jamie Pine dc44a128de Enhance App UI with logo and animations
- Added a voicebox logo to the loading screen in the App component for improved branding.
- Introduced fade-in animations for the logo and loading text to enhance user experience.
- Updated Sidebar component styles for better visual consistency.
- Refactored HistoryTable to implement a new fixed-height row layout, improving readability and interaction.
- Removed unused Badge component from GenerationForm for cleaner code.
2026-01-25 17:46:39 -08:00
Jamie PineandGitHub 2617936d39 Merge pull request #1 from jamiepine/improvements
Improvements
2026-01-25 17:20:52 -08:00
Jamie Pine 05adc4e013 Remove additional CUDA and torch compiler module exclusions from PyInstaller build to streamline the process 2026-01-25 13:19:22 -08:00
Jamie Pine b479178e91 Exclude CUDA libraries and torch compiler modules to reduce bundle size 2026-01-25 13:13:24 -08:00
Jamie Pine a57e7dbc54 Exclude sklearn, pandas, and torchaudio to reduce bundle size 2026-01-25 12:40:51 -08:00
Jamie Pine 530ee407ee Enhance PyInstaller build by excluding additional unnecessary modules
- Added exclusions for 'torch.utils.tensorboard', 'scipy', 'PIL', 'tkinter', 'unittest', and 'test' to reduce bundle size and improve build efficiency.
2026-01-25 12:17:24 -08:00
Jamie PineandClaude Sonnet 4.5 bc21b4c422 Pin LLVM to version 20 for llvmlite compatibility
llvmlite only supports LLVM up to version 20, but brew install llvm
installs version 21. Update macOS runners to install llvm@20 specifically.

Co-Authored-By: Claude Sonnet 4.5 (1M context) <[email protected]>
2026-01-25 11:44:34 -08:00
226 changed files with 42242 additions and 8182 deletions
+39
View File
@@ -0,0 +1,39 @@
[bumpversion]
current_version = 0.1.10
commit = True
tag = True
tag_name = v{new_version}
tag_message = Release v{new_version}
message = Bump version: {current_version} → {new_version}
[bumpversion:file:tauri/src-tauri/tauri.conf.json]
search = "version": "{current_version}"
replace = "version": "{new_version}"
[bumpversion:file:tauri/src-tauri/Cargo.toml]
search = version = "{current_version}"
replace = version = "{new_version}"
[bumpversion:file:package.json]
search = "version": "{current_version}"
replace = "version": "{new_version}"
[bumpversion:file:app/package.json]
search = "version": "{current_version}"
replace = "version": "{new_version}"
[bumpversion:file:tauri/package.json]
search = "version": "{current_version}"
replace = "version": "{new_version}"
[bumpversion:file:landing/package.json]
search = "version": "{current_version}"
replace = "version": "{new_version}"
[bumpversion:file:web/package.json]
search = "version": "{current_version}"
replace = "version": "{new_version}"
[bumpversion:file:backend/__init__.py]
search = __version__ = "{current_version}"
replace = __version__ = "{new_version}"
Binary file not shown.

After

Width:  |  Height:  |  Size: 10 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 187 KiB

+40 -7
View File
@@ -17,15 +17,19 @@ jobs:
- platform: 'macos-latest'
args: '--target aarch64-apple-darwin'
python-version: '3.12'
backend: 'mlx'
- platform: 'macos-15-intel'
args: '--target x86_64-apple-darwin'
python-version: '3.12'
- platform: 'ubuntu-22.04'
args: ''
python-version: '3.12'
backend: 'pytorch'
# - platform: 'ubuntu-22.04'
# args: ''
# python-version: '3.12'
# backend: 'pytorch'
- platform: 'windows-latest'
args: ''
python-version: '3.12'
backend: 'pytorch'
runs-on: ${{ matrix.platform }}
@@ -41,9 +45,9 @@ jobs:
- name: Install LLVM (macOS)
if: matrix.platform == 'macos-latest' || matrix.platform == 'macos-15-intel'
run: |
brew install llvm
echo "$(brew --prefix llvm)/bin" >> $GITHUB_PATH
echo "LLVM_CONFIG=$(brew --prefix llvm)/bin/llvm-config" >> $GITHUB_ENV
brew install llvm@20
echo "$(brew --prefix llvm@20)/bin" >> $GITHUB_PATH
echo "LLVM_CONFIG=$(brew --prefix llvm@20)/bin/llvm-config" >> $GITHUB_ENV
- name: Setup Python
uses: actions/setup-python@v5
@@ -57,6 +61,11 @@ jobs:
pip install pyinstaller
pip install -r backend/requirements.txt
- name: Install MLX dependencies (Apple Silicon only)
if: matrix.backend == 'mlx'
run: |
pip install -r backend/requirements-mlx.txt
- name: Build Python server (Linux/macOS)
if: matrix.platform != 'windows-latest'
run: |
@@ -96,11 +105,34 @@ jobs:
- name: Install dependencies
run: bun install
- name: Install Apple API key
if: matrix.platform == 'macos-latest' || matrix.platform == 'macos-15-intel'
run: |
mkdir -p ~/.appstoreconnect/private_keys/
cd ~/.appstoreconnect/private_keys/
echo ${{ secrets.APPLE_API_KEY_BASE64 }} >> AuthKey_${{ secrets.APPLE_API_KEY }}.p8.base64
base64 --decode -i AuthKey_${{ secrets.APPLE_API_KEY }}.p8.base64 -o AuthKey_${{ secrets.APPLE_API_KEY }}.p8
rm AuthKey_${{ secrets.APPLE_API_KEY }}.p8.base64
- name: Install Codesigning Certificate
if: matrix.platform == 'macos-latest' || matrix.platform == 'macos-15-intel'
uses: apple-actions/import-codesign-certs@v3
with:
p12-file-base64: ${{ secrets.APPLE_CERTIFICATE }}
p12-password: ${{ secrets.APPLE_CERTIFICATE_PASSWORD }}
- uses: tauri-apps/tauri-action@v0
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
TAURI_SIGNING_PRIVATE_KEY: ${{ secrets.TAURI_SIGNING_PRIVATE_KEY }}
TAURI_SIGNING_PRIVATE_KEY_PASSWORD: ${{ secrets.TAURI_SIGNING_PRIVATE_KEY_PASSWORD }}
ENABLE_CODE_SIGNING: ${{ secrets.APPLE_CERTIFICATE }}
APPLE_CERTIFICATE: ${{ secrets.APPLE_CERTIFICATE }}
APPLE_CERTIFICATE_PASSWORD: ${{ secrets.APPLE_CERTIFICATE_PASSWORD }}
APPLE_SIGNING_IDENTITY: ${{ secrets.APPLE_SIGNING_IDENTITY }}
APPLE_PROVIDER_SHORT_NAME: ${{ secrets.APPLE_PROVIDER_SHORT_NAME }}
APPLE_API_ISSUER: ${{ secrets.APPLE_API_ISSUER }}
APPLE_API_KEY: ${{ secrets.APPLE_API_KEY }}
with:
projectPath: tauri
tagName: v__VERSION__
@@ -110,7 +142,8 @@ jobs:
See the assets below to download and install this version.
### Installation
- **macOS**: Download the `.dmg` file
- **macOS (Apple Silicon)**: Download the `aarch64.dmg` file - uses MLX for fast native inference
- **macOS (Intel)**: Download the `x64.dmg` file - uses PyTorch
- **Windows**: Download the `.msi` installer
- **Linux**: Download the `.AppImage` or `.deb` package
+2
View File
@@ -0,0 +1,2 @@
# Force bun usage
engine-strict=true
+82
View File
@@ -0,0 +1,82 @@
# Changelog
All notable changes to Voicebox will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [0.1.0] - 2026-01-25
### Added
#### Core Features
- **Voice Cloning** - Clone voices from audio samples using Qwen3-TTS (1.7B and 0.6B models)
- **Voice Profile Management** - Create, edit, and organize voice profiles with multiple samples
- **Speech Generation** - Generate high-quality speech from text using cloned voices
- **Generation History** - Track all generations with search and filtering capabilities
- **Audio Transcription** - Automatic transcription powered by Whisper
- **In-App Recording** - Record audio samples directly in the app with waveform visualization
#### Desktop App
- **Tauri Desktop App** - Native desktop application for macOS, Windows, and Linux
- **Local Server Mode** - Embedded Python server runs automatically
- **Remote Server Mode** - Connect to a remote Voicebox server on your network
- **Auto-Updates** - Automatic update notifications and installation
#### API
- **REST API** - Full REST API for voice synthesis and profile management
- **OpenAPI Documentation** - Interactive API docs at `/docs` endpoint
- **Type-Safe Client** - Auto-generated TypeScript client from OpenAPI schema
#### Technical
- **Voice Prompt Caching** - Fast regeneration with cached voice prompts
- **Multi-Sample Support** - Combine multiple audio samples for better voice quality
- **GPU/CPU/MPS Support** - Automatic device detection and optimization
- **Model Management** - Lazy loading and VRAM management
- **SQLite Database** - Local data persistence
### Technical Details
- Built with Tauri v2 (Rust + React)
- FastAPI backend with async Python
- TypeScript frontend with React Query and Zustand
- Qwen3-TTS for voice cloning
- Whisper for transcription
### Platform Support
- macOS (Apple Silicon and Intel)
- Windows
- Linux (AppImage)
---
## [Unreleased]
### Added
- **Makefile** - Comprehensive development workflow automation with commands for setup, development, building, testing, and code quality checks
- Includes Python version detection and compatibility warnings
- Self-documenting help system with `make help`
- Colored output for better readability
- Supports parallel development server execution
### Changed
- **README** - Added Makefile reference and updated Quick Start with Makefile-based setup instructions alongside manual setup
---
## [Unreleased - Planned]
### Planned
- Real-time streaming synthesis
- Conversation mode with multiple speakers
- Voice effects (pitch shift, reverb, M3GAN-style)
- Timeline-based audio editor
- Additional voice models (XTTS, Bark)
- Voice design from text descriptions
- Project system for saving sessions
- Plugin architecture
---
[0.1.0]: https://github.com/jamiepine/voicebox/releases/tag/v0.1.0
+433
View File
@@ -0,0 +1,433 @@
# Contributing to Voicebox
Thank you for your interest in contributing to Voicebox! This document provides guidelines and instructions for contributing.
## Code of Conduct
- Be respectful and inclusive
- Welcome newcomers and help them learn
- Focus on constructive feedback
- Respect different viewpoints and experiences
## Getting Started
### Prerequisites
- **[Bun](https://bun.sh)** - Fast JavaScript runtime and package manager
```bash
curl -fsSL https://bun.sh/install | bash
```
- **[Python 3.11+](https://python.org)** - For backend development
```bash
python --version # Should be 3.11 or higher
```
- **[Rust](https://rustup.rs)** - For Tauri desktop app (installed automatically by Tauri CLI)
```bash
rustc --version # Check if installed
```
- **Git** - Version control
### Development Setup
**Using the Makefile (recommended for macOS/Linux):** Run `make setup` to install all dependencies, then `make dev` to start development servers. See `make help` for all available commands.
**Manual setup (required for Windows):**
1. **Fork and clone the repository**
```bash
git clone https://github.com/YOUR_USERNAME/voicebox.git
cd voicebox
```
2. **Install JavaScript dependencies**
```bash
bun install
```
This installs dependencies for:
- `app/` - Shared React frontend
- `tauri/` - Tauri desktop wrapper
- `web/` - Web deployment wrapper
3. **Set up Python backend**
```bash
cd backend
# Create virtual environment
python -m venv venv
# Activate virtual environment
source venv/bin/activate # On macOS/Linux
# or
venv\Scripts\activate # On Windows
# Install Python dependencies
pip install -r requirements.txt
# Install MLX dependencies (Apple Silicon only - for faster inference)
# On Apple Silicon, this enables native Metal acceleration
if [[ $(uname -m) == "arm64" ]]; then
pip install -r requirements-mlx.txt
fi
# Install Qwen3-TTS (required for voice synthesis)
pip install git+https://github.com/QwenLM/Qwen3-TTS.git
```
4. **Start development servers**
Development requires two terminals: one for the Python backend, one for the Tauri app.
**Terminal 1: Backend server** (start this first)
```bash
cd backend
source venv/bin/activate # Activate venv if not already active
bun run dev:server
# Or manually: uvicorn main:app --reload --port 17493
```
Backend will be available at `http://localhost:17493`
**Terminal 2: Desktop app**
```bash
bun run dev
```
This will:
- Create a placeholder sidecar binary (for Tauri compilation)
- Start Vite dev server on port 5173
- Launch Tauri window pointing to localhost:5173
- Connect to the Python server you started in Terminal 1
- Enable hot reload
> **Note:** In dev mode, the app connects to your manually-started Python server.
> The bundled server binary is only used in production builds.
**Optional: Web app**
```bash
bun run dev:web
```
Web app will be available at `http://localhost:5174`
### Model Downloads
Models are automatically downloaded from HuggingFace Hub on first use:
- **Whisper** (transcription): Auto-downloads on first transcription
- **Qwen3-TTS** (voice cloning): Auto-downloads on first generation (~2-4GB)
First-time usage will be slower due to model downloads, but subsequent runs will use cached models.
### Building
**Build everything (recommended):**
```bash
bun run build
```
This automatically:
1. Builds the Python server binary (`./scripts/build-server.sh`)
2. Builds the Tauri desktop app (`cd tauri && bun run tauri build`)
Creates platform-specific installers (`.dmg`, `.msi`, `.AppImage`) in `tauri/src-tauri/target/release/bundle/`.
**Note:** The build process detects your platform and includes the appropriate backend (MLX for Apple Silicon, PyTorch for others).
**Build server binary only:**
```bash
bun run build:server
# or
./scripts/build-server.sh
```
Creates platform-specific binary in `tauri/src-tauri/binaries/`
**Building with local Qwen3-TTS development version:**
If you're actively developing or modifying the Qwen3-TTS library, set the `QWEN_TTS_PATH` environment variable to point to your local clone:
```bash
export QWEN_TTS_PATH=~/path/to/your/Qwen3-TTS
bun run build:server
```
This makes PyInstaller use your local qwen-tts version instead of the pip-installed package. Useful when testing changes to the TTS library before they're published to PyPI or when using an editable install (`pip install -e`).
**Build web app:**
```bash
cd web
bun run build
```
Output in `web/dist/`
### Generate OpenAPI Client
After starting the backend server:
```bash
./scripts/generate-api.sh
```
This downloads the OpenAPI schema and generates the TypeScript client in `app/src/lib/api/`
### Convert Assets to Web Formats
To optimize images and videos for the web, run:
```bash
bun run convert:assets
```
This script:
- Converts PNG → WebP (better compression, same quality)
- Converts MOV → WebM (VP9 codec, smaller file size)
- Processes files in `landing/public/` and `docs/public/`
- **Deletes original files** after successful conversion
**Requirements:** Install `webp` and `ffmpeg`:
```bash
brew install webp ffmpeg
```
> **Note:** Run this before committing new images or videos to keep the repository size small.
## Development Workflow
### 1. Create a Branch
```bash
git checkout -b feature/your-feature-name
# or
git checkout -b fix/your-bug-fix
```
### 2. Make Your Changes
- Write clean, readable code
- Follow existing code style
- Add comments for complex logic
- Update documentation as needed
### 3. Test Your Changes
- Test manually in the app
- Ensure backend API endpoints work
- Check for TypeScript/Python errors
- Verify UI components render correctly
### 4. Commit Your Changes
Write clear, descriptive commit messages:
```bash
git commit -m "Add feature: voice profile export"
git commit -m "Fix: audio playback stops after 30 seconds"
```
### 5. Push and Create Pull Request
```bash
git push origin feature/your-feature-name
```
Then create a pull request on GitHub with:
- Clear description of changes
- Screenshots (for UI changes)
- Reference to related issues
## Code Style
### TypeScript/React
- Use TypeScript strict mode
- Follow React best practices
- Use functional components with hooks
- Prefer named exports
- Format with Biome (runs automatically)
```typescript
// Good
export function ProfileCard({ profile }: { profile: Profile }) {
return <div>{profile.name}</div>;
}
// Avoid
export const ProfileCard = (props) => { ... }
```
### Python
- Follow PEP 8 style guide
- Use type hints
- Use async/await for I/O operations
- Format with Black (if configured)
```python
# Good
async def create_profile(name: str, language: str) -> Profile:
"""Create a new voice profile."""
...
# Avoid
def create_profile(name, language):
...
```
### Rust
- Follow Rust conventions
- Use meaningful variable names
- Handle errors explicitly
- Format with `rustfmt`
## Project Structure
```
voicebox/
├── app/ # Shared React frontend
│ └── src/
│ ├── components/ # UI components
│ ├── lib/ # Utilities and API client
│ └── hooks/ # React hooks
├── backend/ # Python FastAPI server
│ ├── main.py # API routes
│ ├── tts.py # Voice synthesis
│ └── ...
├── tauri/ # Desktop app wrapper
│ └── src-tauri/ # Rust backend
└── scripts/ # Build scripts
```
## Areas for Contribution
### 🐛 Bug Fixes
- Check existing issues for bugs to fix
- Test your fix thoroughly
- Add tests if possible
### ✨ New Features
- Check the roadmap in README.md
- Discuss major features in an issue first
- Keep features focused and well-scoped
### 📚 Documentation
- Improve README clarity
- Add code comments
- Write API documentation
- Create tutorials or guides
### 🎨 UI/UX Improvements
- Improve accessibility
- Enhance visual design
- Optimize performance
- Add animations/transitions
### 🔧 Infrastructure
- Improve build process
- Add CI/CD improvements
- Optimize bundle size
- Add testing infrastructure
## API Development
When adding new API endpoints:
1. **Add route in `backend/main.py`**
2. **Create Pydantic models in `backend/models.py`**
3. **Implement business logic in appropriate module**
4. **Update OpenAPI schema** (automatic with FastAPI)
5. **Regenerate TypeScript client:**
```bash
bun run generate:api
```
6. **Update `backend/README.md`** with endpoint documentation
## Testing
Currently, testing is primarily manual. When adding tests:
- **Backend**: Use pytest for Python tests
- **Frontend**: Use Vitest for React component tests
- **E2E**: Use Playwright for end-to-end tests (future)
## Pull Request Process
1. **Update documentation** if needed
2. **Ensure code follows style guidelines**
3. **Test your changes thoroughly**
4. **Update CHANGELOG.md** with your changes
5. **Request review** from maintainers
### PR Checklist
- [ ] Code follows style guidelines
- [ ] Documentation updated
- [ ] Changes tested
- [ ] No breaking changes (or documented)
- [ ] CHANGELOG.md updated
## Release Process
Releases are managed by maintainers:
1. **Bump version using bumpversion:**
```bash
# Install bumpversion (if not already installed)
pip install bumpversion
# Bump patch version (0.1.0 -> 0.1.1)
bumpversion patch
# Or bump minor version (0.1.0 -> 0.2.0)
bumpversion minor
# Or bump major version (0.1.0 -> 1.0.0)
bumpversion major
```
This automatically:
- Updates version numbers in all files (`tauri.conf.json`, `Cargo.toml`, all `package.json` files, `backend/main.py`)
- Creates a git commit with the version bump
- Creates a git tag (e.g., `v0.1.1`, `v0.2.0`)
2. **Update CHANGELOG.md** with release notes
3. **Push commits and tags:**
```bash
git push
git push --tags
```
4. **GitHub Actions builds and releases** automatically when tags are pushed
## Troubleshooting
See [docs/TROUBLESHOOTING.md](docs/TROUBLESHOOTING.md) for common issues and solutions.
**Quick fixes:**
- **Backend won't start:** Check Python version (3.11+), ensure venv is activated, install dependencies
- **Tauri build fails:** Ensure Rust is installed, clean build with `cd tauri/src-tauri && cargo clean`
- **OpenAPI client generation fails:** Ensure backend is running, check `curl http://localhost:8000/openapi.json`
## Questions?
- Open an issue for bugs or feature requests
- Check existing issues and discussions
- Review the codebase to understand patterns
- See [docs/TROUBLESHOOTING.md](docs/TROUBLESHOOTING.md) for common issues
## Additional Resources
- [README.md](README.md) - Project overview
- [backend/README.md](backend/README.md) - API documentation
- [docs/AUTOUPDATER_QUICKSTART.md](docs/AUTOUPDATER_QUICKSTART.md) - Auto-updater setup
- [SECURITY.md](SECURITY.md) - Security policy
- [CHANGELOG.md](CHANGELOG.md) - Version history
## License
By contributing, you agree that your contributions will be licensed under the MIT License.
---
Thank you for contributing to Voicebox! 🎉
-447
View File
@@ -1,447 +0,0 @@
# voicebox - Current State Overview
**Last Updated:** January 25, 2026
**Status:** ✅ MVP Core Features Working - Voice generation from Tauri app successful!
---
## 🎯 What We Have
### ✅ **Fully Implemented & Working**
#### **Backend (Python FastAPI)**
- **Voice Profile Management**
- Create, read, update, delete profiles
- Add multiple audio samples per profile
- Multi-reference voice combination (combines multiple samples)
- Profile storage in SQLite + file system (`data/profiles/`)
- **Voice Generation**
- Qwen3-TTS model integration (1.7B and 0.6B support)
- Automatic model downloading from HuggingFace Hub
- Voice prompt caching for instant re-generation
- Support for English and Chinese
- Seed-based reproducibility
- GPU/CPU/MPS device detection
- **Generation History**
- Full CRUD operations
- Search by text content
- Filter by profile
- Pagination support
- Statistics endpoint
- Audio file storage (`data/generations/`)
- **Audio Transcription**
- Whisper integration for speech-to-text
- Language detection/selection
- Used for reference text extraction from samples
- **Database**
- SQLite with SQLAlchemy ORM
- Tables: `profiles`, `profile_samples`, `generations`, `projects` (ready for future)
- Automatic schema initialization
- **API Endpoints**
- RESTful API with FastAPI
- OpenAPI schema generation
- CORS enabled
- Health check endpoint
- File serving for audio files
#### **Frontend (React + TypeScript + Tauri)**
- **Voice Profile UI**
- Profile list with cards
- Create/edit profile dialog
- Upload audio samples with transcription
- Sample management (view/delete)
- Profile detail view
- **Generation UI**
- Form with profile selection
- Text input (up to 5000 chars)
- Language selection (en/zh)
- Optional seed input
- Loading states and error handling
- **History UI**
- Table view with pagination
- Search functionality
- Play audio inline
- Download audio files
- Delete generations
- **Server Settings**
- Connection form (local/remote mode)
- Server status display
- Health check integration
- **State Management**
- React Query for server state
- Zustand for client state (server URL, connection status)
- Type-safe API client
- **UI Components**
- shadcn/ui component library
- Tailwind CSS styling
- Responsive design
- Toast notifications
- Form validation with Zod
#### **Tauri Desktop App**
- **Rust Backend**
- Sidecar management for Python server
- Start/stop server commands
- Remote mode support (0.0.0.0 binding)
- Process lifecycle management
- **Build System**
- Tauri v2 configuration
- Platform-specific builds
- Dev tools in debug mode
---
## 🏗️ Architecture
### **Project Structure**
```
voicebox/
├── app/ # Shared React frontend
│ ├── src/
│ │ ├── components/ # React components
│ │ │ ├── VoiceProfiles/ ✅ Complete
│ │ │ ├── Generation/ ✅ Complete
│ │ │ ├── History/ ✅ Complete
│ │ │ ├── ServerSettings/ ✅ Complete
│ │ │ └── AudioStudio/ 📦 Placeholder (future)
│ │ ├── lib/
│ │ │ ├── api/ # Type-safe API client ✅
│ │ │ ├── hooks/ # React Query hooks ✅
│ │ │ └── utils/ # Utilities ✅
│ │ └── stores/ # Zustand stores ✅
│
├── backend/ # Python FastAPI server
│ ├── main.py # FastAPI app + routes ✅
│ ├── models.py # Pydantic models ✅
│ ├── database.py # SQLAlchemy ORM ✅
│ ├── profiles.py # Profile management ✅
│ ├── history.py # History management ✅
│ ├── tts.py # Qwen3-TTS integration ✅
│ ├── transcribe.py # Whisper integration ✅
│ ├── studio.py # Audio studio (future)
│ └── utils/
│ ├── audio.py # Audio processing ✅
│ ├── cache.py # Voice prompt caching ✅
│ └── validation.py # Validation helpers ✅
│
├── tauri/ # Tauri desktop wrapper
│ ├── src/ # React entry point ✅
│ └── src-tauri/ # Rust backend ✅
│ └── src/main.rs # Sidecar management ✅
│
├── data/ # User data directory
│ ├── profiles/ # Profile audio samples
│ ├── generations/ # Generated audio files
│ ├── cache/ # Cached voice prompts
│ └── voicebox.db # SQLite database
│
└── scripts/ # Build & generation scripts
├── generate-api.sh # OpenAPI client generation
└── build-server.sh # Python binary build
```
### **Data Flow**
```
User Action (Tauri App)
↓
React Component (Form Submit)
↓
React Query Hook (useGeneration)
↓
API Client (apiClient.generateSpeech)
↓
HTTP Request → FastAPI Backend
↓
Backend Route Handler (/generate)
↓
Business Logic:
1. Get profile from DB
2. Create voice prompt (with caching)
3. Generate audio with Qwen3-TTS
4. Save audio file
5. Create history entry
↓
Response (GenerationResponse)
↓
React Query Cache Update
↓
UI Refresh (History table updates)
```
### **Key Technologies**
| Layer | Technology | Purpose |
|-------|-----------|---------|
| **Desktop Framework** | Tauri v2 | Native desktop app wrapper |
| **Frontend Framework** | React 18 | UI components |
| **Language** | TypeScript | Type safety |
| **Styling** | Tailwind CSS | Utility-first CSS |
| **UI Components** | shadcn/ui | Component library |
| **State Management** | React Query + Zustand | Server & client state |
| **Form Handling** | React Hook Form + Zod | Form validation |
| **Backend Framework** | FastAPI | Async REST API |
| **Database** | SQLite + SQLAlchemy | Data persistence |
| **ML Models** | Qwen3-TTS + Whisper | Voice cloning + transcription |
| **Audio Processing** | librosa + soundfile | Audio I/O and processing |
| **Package Manager** | Bun | Fast JS/TS package management |
| **Build Tool** | Vite | Frontend bundling |
---
## 🔑 Key Features & Capabilities
### **1. Voice Profile System**
- **Multi-sample support**: Add multiple audio samples per profile
- **Automatic combination**: Multiple samples are combined for better quality
- **Voice prompt caching**: Re-use voice prompts for instant re-generation
- **Audio validation**: Ensures samples meet quality requirements
### **2. Generation Pipeline**
- **Lazy model loading**: Model loads on first use
- **Device detection**: Automatically uses GPU if available
- **Caching layer**: Voice prompts cached by audio hash + text
- **Error handling**: Graceful degradation and clear error messages
### **3. History & Search**
- **Full-text search**: Search generations by text content
- **Pagination**: Efficient loading of large histories
- **Audio playback**: Inline audio player
- **File management**: Download and delete operations
### **4. Server/Client Architecture**
- **Local mode**: Backend runs alongside Tauri app
- **Remote mode**: Connect to remote GPU machine
- **One-click server**: Start server from UI
- **Connection management**: Persistent server URL storage
---
## 📊 Database Schema
### **Tables**
```sql
-- Voice Profiles
profiles
- id (PK, UUID)
- name (unique)
- description
- language (en/zh)
- created_at
- updated_at
-- Profile Samples
profile_samples
- id (PK, UUID)
- profile_id (FK → profiles.id)
- audio_path
- reference_text
-- Generations
generations
- id (PK, UUID)
- profile_id (FK → profiles.id)
- text
- language
- audio_path
- duration (seconds)
- seed (optional)
- created_at
-- Projects (ready for future)
projects
- id (PK, UUID)
- name
- data (JSON)
- created_at
- updated_at
```
---
## 🎨 UI Components Status
| Component | Status | Features |
|-----------|--------|----------|
| **ProfileList** | ✅ Complete | List, create, empty state |
| **ProfileCard** | ✅ Complete | Display profile info |
| **ProfileForm** | ✅ Complete | Create/edit dialog |
| **ProfileDetail** | ✅ Complete | View samples, add samples |
| **SampleUpload** | ✅ Complete | File upload + transcription |
| **GenerationForm** | ✅ Complete | Full generation form |
| **HistoryTable** | ✅ Complete | Table, search, pagination, play/download |
| **ConnectionForm** | ✅ Complete | Server URL input |
| **ServerStatus** | ✅ Complete | Health check display |
| **AudioStudio** | 📦 Placeholder | Timeline editor (future) |
---
## 🔌 API Endpoints
### **Profiles**
- `POST /profiles` - Create profile
- `GET /profiles` - List all profiles
- `GET /profiles/{id}` - Get profile
- `PUT /profiles/{id}` - Update profile
- `DELETE /profiles/{id}` - Delete profile
- `POST /profiles/{id}/samples` - Add sample
- `GET /profiles/{id}/samples` - List samples
- `DELETE /profiles/samples/{id}` - Delete sample
### **Generation**
- `POST /generate` - Generate speech
### **History**
- `GET /history` - List generations (with filters)
- `GET /history/{id}` - Get generation
- `DELETE /history/{id}` - Delete generation
- `GET /history/stats` - Get statistics
### **Transcription**
- `POST /transcribe` - Transcribe audio
### **Audio**
- `GET /audio/{id}` - Serve audio file
### **Health**
- `GET /health` - Health check with model status
### **Model Management**
- `POST /models/load` - Load TTS model
- `POST /models/unload` - Unload TTS model
---
## 🚀 What's Next (Planned Features)
### **Phase 2: Advanced Features**
- [ ] Multi-reference voice combination UI
- [ ] Batch generation (multiple variations)
- [ ] Advanced audio normalization
- [ ] Export options (MP3, OGG, etc.)
- [ ] M3GAN voice effect
### **Phase 3: Audio Studio**
- [ ] Timeline-based audio editor
- [ ] Word-level timestamps
- [ ] Project system (save/load sessions)
- [ ] Audio effects and filters
- [ ] Multi-track editing
### **Phase 4: Voice Design**
- [ ] Text-to-voice (no reference needed)
- [ ] Preset voices with style control
- [ ] Conversation mode (multi-speaker)
- [ ] Custom audio effects library
---
## 📝 Code Quality Standards
- ✅ **Type safety**: TypeScript strict mode, Pydantic models
- ✅ **Modular architecture**: No files over 500 lines
- ✅ **Error handling**: Comprehensive error messages
- ✅ **Caching**: Voice prompt caching for performance
- ✅ **Database**: SQLAlchemy ORM with proper relationships
- ✅ **API design**: RESTful with OpenAPI schema
- ✅ **UI/UX**: Responsive, accessible, loading states
---
## 🧪 Testing Status
- ✅ **Manual testing**: Voice generation working end-to-end
- 📦 **Unit tests**: Not yet implemented
- 📦 **Integration tests**: Not yet implemented
- 📦 **E2E tests**: Not yet implemented
---
## 📦 Dependencies
### **Backend**
- FastAPI - Web framework
- SQLAlchemy - ORM
- Pydantic - Validation
- Qwen3-TTS - Voice cloning model
- Whisper - Speech recognition
- librosa - Audio processing
- soundfile - Audio I/O
- PyTorch - ML framework
### **Frontend**
- React 18 - UI framework
- TypeScript - Type safety
- React Query - Server state
- Zustand - Client state
- React Hook Form - Forms
- Zod - Schema validation
- Tailwind CSS - Styling
- shadcn/ui - Components
- Lucide React - Icons
### **Desktop**
- Tauri v2 - Desktop framework
- Rust - System backend
---
## 🎯 Current Capabilities Summary
✅ **Working End-to-End:**
1. Create voice profiles with audio samples
2. Generate speech from text using cloned voices
3. View and manage generation history
4. Play and download generated audio
5. Search and filter history
6. Connect to local or remote backend
7. Automatic model downloading
8. Voice prompt caching for speed
🎉 **You just successfully generated voice from the Tauri app!**
---
## 🔍 Key Files Reference
### **Backend Core**
- `backend/main.py` - FastAPI app and routes
- `backend/tts.py` - Qwen3-TTS model wrapper
- `backend/profiles.py` - Profile business logic
- `backend/history.py` - History business logic
- `backend/database.py` - Database models
### **Frontend Core**
- `app/src/App.tsx` - Main app component
- `app/src/lib/api/client.ts` - API client
- `app/src/lib/hooks/` - React Query hooks
- `app/src/stores/` - Zustand stores
### **Tauri**
- `tauri/src-tauri/src/main.rs` - Rust backend
- `tauri/src/main.tsx` - React entry point
---
## 💡 Development Workflow
1. **Start backend**: `bun run dev:server` (or via Tauri)
2. **Start frontend**: `bun run dev` (Tauri) or `bun run dev:web` (web)
3. **Generate API client**: `bun run generate:api` (after backend changes)
4. **Build server binary**: `bun run build:server` (for Tauri bundling)
---
**Ready to build more features! 🚀**
+21
View File
@@ -0,0 +1,21 @@
MIT License
Copyright (c) 2026 Voicebox Contributors
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
+245
View File
@@ -0,0 +1,245 @@
# Voicebox Makefile
# Unix-only (macOS/Linux). Windows users should use WSL.
SHELL := /bin/bash
.DEFAULT_GOAL := help
# Directories
BACKEND_DIR := backend
TAURI_DIR := tauri
WEB_DIR := web
APP_DIR := app
# Python (prefer 3.12, fallback to 3.13, then python3)
PYTHON := $(shell command -v python3.12 2>/dev/null || command -v python3.13 2>/dev/null || echo python3)
VENV := $(CURDIR)/$(BACKEND_DIR)/venv
VENV_BIN := $(VENV)/bin
PIP := $(VENV_BIN)/pip
PYTHON_VENV := $(VENV_BIN)/python
# Colors for output
BLUE := \033[0;34m
GREEN := \033[0;32m
YELLOW := \033[0;33m
NC := \033[0m # No Color
.PHONY: help
help: ## Show this help message
@echo -e "$(BLUE)Voicebox$(NC) - Development Commands"
@echo ""
@grep -E '^[a-zA-Z_-]+:.*?## .*$$' $(MAKEFILE_LIST) | sort | \
awk 'BEGIN {FS = ":.*?## "}; {printf " $(GREEN)%-20s$(NC) %s\n", $$1, $$2}'
# =============================================================================
# SETUP
# =============================================================================
.PHONY: setup setup-js setup-python setup-rust
setup: setup-js setup-python ## Full project setup (all dependencies)
@echo -e "$(GREEN)✓ Setup complete!$(NC)"
@echo -e " Run $(YELLOW)make dev$(NC) to start development servers"
setup-js: ## Install JavaScript dependencies (bun)
@echo -e "$(BLUE)Installing JavaScript dependencies...$(NC)"
bun install
setup-python: $(VENV)/bin/activate ## Set up Python virtual environment and dependencies
@echo -e "$(BLUE)Installing Python dependencies...$(NC)"
$(PIP) install --upgrade pip
$(PIP) install -r $(BACKEND_DIR)/requirements.txt
@if [ "$$(uname -m)" = "arm64" ] && [ "$$(uname)" = "Darwin" ]; then \
echo -e "$(BLUE)Detected Apple Silicon - installing MLX dependencies...$(NC)"; \
$(PIP) install -r $(BACKEND_DIR)/requirements-mlx.txt; \
echo -e "$(GREEN)✓ MLX backend enabled (native Metal acceleration)$(NC)"; \
fi
$(PIP) install git+https://github.com/QwenLM/Qwen3-TTS.git
@echo -e "$(GREEN)✓ Python environment ready$(NC)"
$(VENV)/bin/activate:
@echo -e "$(BLUE)Creating Python virtual environment...$(NC)"
@PY_MINOR=$$($(PYTHON) -c "import sys; print(sys.version_info[1])"); \
if [ "$$PY_MINOR" -gt 13 ]; then \
echo -e "$(YELLOW)Warning: Python 3.$$PY_MINOR detected. ML packages may not be compatible.$(NC)"; \
echo -e "$(YELLOW)Recommended: Use Python 3.12 or 3.13 (brew install [email protected])$(NC)"; \
fi
$(PYTHON) -m venv $(VENV)
setup-rust: ## Install Rust toolchain (if not present)
@command -v rustc >/dev/null 2>&1 || curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
# =============================================================================
# DEVELOPMENT
# =============================================================================
.PHONY: dev dev-backend dev-frontend dev-web kill-dev
dev: ## Start backend + desktop app (parallel)
@echo -e "$(BLUE)Starting development servers...$(NC)"
@echo -e "$(YELLOW)Note: If Tauri fails, run 'make build-server' first or use separate terminals$(NC)"
@trap 'kill 0' EXIT; \
$(MAKE) dev-backend & \
sleep 2 && $(MAKE) dev-frontend & \
wait
dev-backend: ## Start FastAPI backend server
@echo -e "$(BLUE)Starting backend server on http://localhost:17493$(NC)"
$(VENV_BIN)/uvicorn backend.main:app --reload --port 17493
dev-frontend: ## Start Tauri desktop app
@echo -e "$(BLUE)Starting Tauri desktop app...$(NC)"
bun run dev
dev-web: ## Start backend + web app (parallel)
@echo -e "$(BLUE)Starting web development servers...$(NC)"
@trap 'kill 0' EXIT; \
$(MAKE) dev-backend & \
sleep 2 && cd $(WEB_DIR) && bun run dev & \
wait
kill-dev: ## Kill all development processes
@echo -e "$(YELLOW)Killing development processes...$(NC)"
-pkill -f "uvicorn main:app" 2>/dev/null || true
-pkill -f "vite" 2>/dev/null || true
@echo -e "$(GREEN)✓ Processes killed$(NC)"
# =============================================================================
# BUILD
# =============================================================================
.PHONY: build build-server build-tauri build-web
build: build-server build-tauri ## Build everything (server binary + desktop app)
@echo -e "$(GREEN)✓ Build complete!$(NC)"
build-server: ## Build Python server binary
@echo -e "$(BLUE)Building server binary...$(NC)"
PATH="$(VENV_BIN):$$PATH" ./scripts/build-server.sh
build-tauri: ## Build Tauri desktop app
@echo -e "$(BLUE)Building Tauri desktop app...$(NC)"
cd $(TAURI_DIR) && bun run tauri build
build-web: ## Build web app
@echo -e "$(BLUE)Building web app...$(NC)"
cd $(WEB_DIR) && bun run build
@echo -e "$(GREEN)✓ Web build output in $(WEB_DIR)/dist/$(NC)"
# =============================================================================
# DATABASE & API
# =============================================================================
.PHONY: db-init db-reset generate-api
db-init: $(VENV)/bin/activate ## Initialize SQLite database
@echo -e "$(BLUE)Initializing database...$(NC)"
cd $(BACKEND_DIR) && $(PYTHON_VENV) -c "from database import init_db; init_db()"
@echo -e "$(GREEN)✓ Database created at $(BACKEND_DIR)/data/voicebox.db$(NC)"
db-reset: ## Reset database (delete and reinitialize)
@echo -e "$(YELLOW)Resetting database...$(NC)"
rm -f $(BACKEND_DIR)/data/voicebox.db
$(MAKE) db-init
generate-api: ## Generate TypeScript API client from OpenAPI schema
@echo -e "$(BLUE)Generating API client...$(NC)"
@echo -e "$(YELLOW)Note: Backend must be running (make dev-backend)$(NC)"
./scripts/generate-api.sh
@echo -e "$(GREEN)✓ API client generated in $(APP_DIR)/src/lib/api/$(NC)"
# =============================================================================
# CODE QUALITY
# =============================================================================
.PHONY: lint format typecheck check
lint: ## Run linter (Biome)
@echo -e "$(BLUE)Linting...$(NC)"
bun run lint
format: ## Format code (Biome)
@echo -e "$(BLUE)Formatting...$(NC)"
bun run format
typecheck: ## Run TypeScript type checking
@echo -e "$(BLUE)Type checking...$(NC)"
bun run tsc --noEmit
check: ## Run all checks (Biome lint + format + type check)
@echo -e "$(BLUE)Running all checks...$(NC)"
bun run check
@echo -e "$(GREEN)✓ All checks passed$(NC)"
# =============================================================================
# TESTING
# =============================================================================
.PHONY: test test-backend test-frontend
test: test-backend test-frontend ## Run all tests
@echo -e "$(GREEN)✓ All tests passed$(NC)"
test-backend: ## Run Python backend tests (requires pytest)
@echo -e "$(BLUE)Running backend tests...$(NC)"
@if [ -f "$(VENV_BIN)/pytest" ]; then \
cd $(BACKEND_DIR) && $(VENV_BIN)/pytest -v; \
else \
echo -e "$(YELLOW)pytest not installed. Run: $(PIP) install pytest$(NC)"; \
exit 1; \
fi
test-frontend: ## Run frontend tests (requires test script in package.json)
@echo -e "$(BLUE)Running frontend tests...$(NC)"
@if bun run test --help >/dev/null 2>&1; then \
bun run test; \
else \
echo -e "$(YELLOW)No test script configured$(NC)"; \
exit 1; \
fi
# =============================================================================
# LOGS & DEBUGGING
# =============================================================================
.PHONY: logs docs
logs: ## Tail backend logs
@echo -e "$(BLUE)Tailing logs (Ctrl+C to stop)...$(NC)"
tail -f $(BACKEND_DIR)/logs/*.log 2>/dev/null || echo "No log files found"
docs: ## Open API documentation (backend must be running)
@echo -e "$(BLUE)Opening API docs...$(NC)"
open http://localhost:17493/docs 2>/dev/null || xdg-open http://localhost:17493/docs
# =============================================================================
# CLEAN
# =============================================================================
.PHONY: clean clean-python clean-build clean-all
clean: ## Clean build artifacts
@echo -e "$(BLUE)Cleaning build artifacts...$(NC)"
rm -rf $(TAURI_DIR)/src-tauri/target/release
rm -rf $(WEB_DIR)/dist
rm -rf $(APP_DIR)/dist
@echo -e "$(GREEN)✓ Build artifacts cleaned$(NC)"
clean-python: ## Clean Python cache and virtual environment
@echo -e "$(BLUE)Cleaning Python files...$(NC)"
rm -rf $(VENV)
find $(BACKEND_DIR) -type d -name "__pycache__" -exec rm -rf {} + 2>/dev/null || true
find $(BACKEND_DIR) -type f -name "*.pyc" -delete 2>/dev/null || true
@echo -e "$(GREEN)✓ Python environment cleaned$(NC)"
clean-build: ## Clean Rust/Tauri build cache
@echo -e "$(BLUE)Cleaning Rust build cache...$(NC)"
cd $(TAURI_DIR)/src-tauri && cargo clean
@echo -e "$(GREEN)✓ Rust cache cleaned$(NC)"
clean-all: clean clean-python clean-build ## Nuclear clean (everything)
@echo -e "$(BLUE)Cleaning node_modules...$(NC)"
rm -rf node_modules
rm -rf $(APP_DIR)/node_modules
rm -rf $(TAURI_DIR)/node_modules
rm -rf $(WEB_DIR)/node_modules
@echo -e "$(GREEN)✓ Full clean complete$(NC)"
+259 -364
View File
@@ -1,409 +1,304 @@
# voicebox
<p align="center">
<img src=".github/assets/icon-dark.webp" alt="Voicebox" width="120" height="120" />
</p>
A production-quality desktop app for Qwen3-TTS voice cloning and generation.
<h1 align="center">Voicebox</h1>
**Domain:** voicebox.sh
<p align="center">
<strong>The open-source voice synthesis studio.</strong><br/>
Clone voices. Generate speech. Build voice-powered apps.<br/>
All running locally on your machine.
</p>
<p align="center">
<a href="https://github.com/jamiepine/voicebox/releases">
<img src="https://img.shields.io/github/downloads/jamiepine/voicebox/total?style=flat&color=blue" alt="Downloads" />
</a>
<a href="https://github.com/jamiepine/voicebox/releases/latest">
<img src="https://img.shields.io/github/v/release/jamiepine/voicebox?style=flat" alt="Release" />
</a>
<a href="https://github.com/jamiepine/voicebox/stargazers">
<img src="https://img.shields.io/github/stars/jamiepine/voicebox?style=flat" alt="Stars" />
</a>
<a href="https://github.com/jamiepine/voicebox/blob/main/LICENSE">
<img src="https://img.shields.io/github/license/jamiepine/voicebox?style=flat" alt="License" />
</a>
</p>
<p align="center">
<a href="https://voicebox.sh">voicebox.sh</a> •
<a href="#download">Download</a> •
<a href="#features">Features</a> •
<a href="#api">API</a> •
<a href="#roadmap">Roadmap</a>
</p>
<br/>
<p align="center">
<a href="https://voicebox.sh">
<img src="landing/public/assets/app-screenshot-1.webp" alt="Voicebox App Screenshot" width="800" />
</a>
</p>
<p align="center">
<em>Click the image above to watch the demo video on <a href="https://voicebox.sh">voicebox.sh</a></em>
</p>
<br/>
<p align="center">
<img src="landing/public/assets/app-screenshot-2.webp" alt="Voicebox Screenshot 2" width="800" />
</p>
<p align="center">
<img src="landing/public/assets/app-screenshot-3.webp" alt="Voicebox Screenshot 3" width="800" />
</p>
<br/>
## What is Voicebox?
Voicebox is a **local-first voice cloning studio** with DAW-like features for professional voice synthesis. Think of it as the **Ollama for voice** — download models, clone voices, and generate speech entirely on your machine.
Unlike cloud services that lock your voice data behind subscriptions, Voicebox gives you:
- **Complete privacy** — models and voice data stay on your machine
- **Professional tools** — multi-track timeline editor, audio trimming, conversation mixing
- **Model flexibility** — currently powered by Qwen3-TTS, with support for XTTS, Bark, and other models coming soon
- **API-first** — use the desktop app or integrate voice synthesis into your own projects
- **Native performance** — built with Tauri (Rust), not Electron
Download a voice model, clone any voice from a few seconds of audio, and compose multi-voice projects with studio-grade editing tools. No Python install required, no cloud dependency, no limits.
---
## Vision
## Download
Qwen3-TTS is a breakthrough model from Alibaba that achieves near-perfect voice cloning. The existing implementations (Voice-Clone-Studio, mimic, etc.) are either feature-rich but architecturally messy, or well-structured but limited in scope.
Voicebox is available now for macOS and Windows.
voicebox aims to build the definitive Qwen3-TTS application by combining the best patterns from existing projects while avoiding their architectural mistakes.
| Platform | Download |
|----------|----------|
| macOS (Apple Silicon) | [voicebox_aarch64.app.tar.gz](https://github.com/jamiepine/voicebox/releases/download/v0.1.0/voicebox_aarch64.app.tar.gz) |
| macOS (Intel) | [voicebox_x64.app.tar.gz](https://github.com/jamiepine/voicebox/releases/download/v0.1.0/voicebox_x64.app.tar.gz) |
| Windows (MSI) | [voicebox_0.1.0_x64_en-US.msi](https://github.com/jamiepine/voicebox/releases/download/v0.1.0/voicebox_0.1.0_x64_en-US.msi) |
| Windows (Setup) | [voicebox_0.1.0_x64-setup.exe](https://github.com/jamiepine/voicebox/releases/download/v0.1.0/voicebox_0.1.0_x64-setup.exe) |
## Design Principles
> **Linux builds coming soon** — Currently blocked by GitHub runner disk space limitations.
1. **Clean architecture from day one** - No monolithic files, proper separation of concerns
2. **Desktop-first experience** - Native feel via Tauri, not a web app in disguise
3. **Production code quality** - Type safety, modularity, maintainability
4. **Performance and UX** - Smart caching, async operations, responsive UI
5. **Extensible design** - Easy to add new models, effects, and features
6. **Flexible deployment** - Run backend locally or connect to remote GPU machine with one click
---
## Technology Stack
## Features
### Backend (Python)
- **FastAPI** - Async REST API
- **SQLAlchemy** - Database ORM with migrations
- **Pydantic** - Request/response validation
- **Qwen3-TTS** - Voice cloning model
- **Whisper** - Speech-to-text transcription
- **librosa + soundfile** - Audio processing
### Voice Cloning with Qwen3-TTS
### Frontend (Tauri + TypeScript)
- **Tauri** - Native desktop framework
- **React** - UI framework
- **TypeScript** - Type safety throughout
- **Bun** - Fast package manager and JavaScript runtime
- **React Query** - Server state management and API calls
- **OpenAPI (generated)** - Type-safe API client from FastAPI schema
- **Tailwind CSS** - Styling
- **Zustand** - Client-side state management
- **WaveSurfer.js** - Audio visualization
Powered by Alibaba's **Qwen3-TTS** — a breakthrough model that achieves near-perfect voice cloning from just a few seconds of audio.
### Database
- **SQLite** - Local storage
- **Alembic** - Schema migrations
- **Instant cloning** — Upload a sample, get a voice profile
- **High fidelity** — Natural prosody, emotion, and cadence
- **Multi-language** — English, Chinese, and more coming
## Server/Client Mode
### Voice Profile Management
voicebox supports flexible deployment for users with multiple machines:
- **Create profiles** from audio files or record directly in-app
- **Import/Export** profiles to share or backup
- **Multi-sample support** — combine multiple samples for higher quality cloning
- **Organize** with descriptions and language tags
### Local Mode (Default)
- Backend runs locally alongside the Tauri app
- Best for users with GPU on their primary machine
### Speech Generation
### Remote Mode (One-Click Setup)
- **Use case:** Your laptop doesn't have a GPU, but your desktop does
- **Server:** Run voicebox on GPU machine, click "Start Server"
- Starts FastAPI backend on local network
- Shows connection URL (e.g., `http://192.168.1.100:8000`)
- **Client:** Run voicebox on laptop, enter server URL
- Connects to remote backend
- Full UI functionality, inference happens on GPU machine
- **Security:** Local network only for now (no internet exposure)
- **Text-to-speech** with any cloned voice
- **Batch generation** for long-form content
- **Smart caching** — regenerate instantly with voice prompt caching
### How It Works
```
┌─────────────────┐ ┌─────────────────┐
│ Laptop │ │ Desktop │
│ (Client) │ │ (Server) │
│ │ │ │
│ Tauri App ────────────────▶ FastAPI │
│ React UI │ HTTP │ Qwen3-TTS │
│ │ │ SQLite │
│ │ │ CUDA/GPU │
└─────────────────┘ └─────────────────┘
### Stories Editor
Create multi-voice narratives, podcasts, and conversations with a timeline-based editor.
- **Multi-track composition** — arrange multiple voice tracks in a single project
- **Inline audio editing** — trim and split clips directly in the timeline
- **Auto-playback** — preview stories with synchronized playhead
- **Voice mixing** — build conversations with multiple participants
### Recording & Transcription
- **In-app recording** with waveform visualization
- **System audio capture** — record desktop audio on macOS and Windows
- **Automatic transcription** powered by Whisper
- **Export recordings** in multiple formats
### Generation History
- **Full history** of all generated audio
- **Search & filter** by voice, text, or date
- **Re-generate** any past generation with one click
### Flexible Deployment
- **Local mode** — Everything runs on your machine
- **Remote mode** — Connect to a GPU server on your network
- **One-click server** — Turn any machine into a Voicebox server
---
## API
Voicebox exposes a full REST API, so you can integrate voice synthesis into your own apps.
```bash
# Generate speech
curl -X POST http://localhost:8000/generate \
-H "Content-Type: application/json" \
-d '{"text": "Hello world", "profile_id": "abc123", "language": "en"}'
# List voice profiles
curl http://localhost:8000/profiles
# Create a profile
curl -X POST http://localhost:8000/profiles \
-H "Content-Type: application/json" \
-d '{"name": "My Voice", "language": "en"}'
```
**Benefits:**
- Use powerful GPU machine from lightweight laptop
- No complex setup - just click "Start Server"
- All data (history, profiles) lives on server
- Client is just a UI - no local storage needed in remote mode
**Use cases:**
## Core Features
- Game dialogue systems
- Podcast/video production pipelines
- Accessibility tools
- Voice assistants
- Content creation automation
### Phase 1 (MVP)
- Voice profile management
- Single-reference voice cloning
- Generation history with search
- Basic audio playback and preview
- Server/client mode (local network)
- One-click server startup
Full API documentation available at `http://localhost:8000/docs` when running.
### Phase 2
- Multi-reference voice combination
- Batch variation generation
- Advanced audio normalization
- Export options and formats
---
### Phase 3
- Audio studio with timeline editing
- Word-level timestamps
- Project system (save/load sessions)
- Export options
## Tech Stack
### Phase 4
- Voice design (text-to-voice)
- Preset voices with style control
- Conversation mode (multi-speaker)
- Custom audio effects
| Layer | Technology |
|-------|------------|
| Desktop App | Tauri (Rust) |
| Frontend | React, TypeScript, Tailwind CSS |
| State | Zustand, React Query |
| Backend | FastAPI (Python) |
| Voice Model | Qwen3-TTS (PyTorch or MLX) |
| Transcription | Whisper (PyTorch or MLX) |
| Inference Engine | MLX (Apple Silicon) / PyTorch (Windows/Linux/Intel) |
| Database | SQLite |
| Audio | WaveSurfer.js, librosa |
## Key Differentiators
**Why this stack?**
What makes voicebox better than existing implementations:
- **Tauri over Electron** — 10x smaller bundle, native performance, lower memory
- **FastAPI** — Async Python with automatic OpenAPI schema generation
- **Type-safe end-to-end** — Generated TypeScript client from OpenAPI spec
1. **Clean codebase** - Modular architecture, no 2,000+ line files
2. **Type safety end-to-end** - OpenAPI-generated TypeScript client, Pydantic backend, React Query
3. **Smart caching** - Voice prompt caching for instant re-generation
4. **Desktop UX** - Native performance, keyboard shortcuts, native dialogs
5. **Server/client mode** - One-click remote GPU access from any device
6. **Multi-reference** - Combine voice samples for higher quality
7. **Audio studio** - Timeline-based editing with word-level precision
8. **Production patterns** - Cross-platform, graceful degradation, error recovery
9. **Database-backed** - Searchable history, project persistence
10. **Extensible** - Clean plugin system for models and features
---
## Architecture Overview
## Roadmap
Voicebox is the beginning of something bigger. Here's what's coming:
### Coming Soon
| Feature | Description |
|---------|-------------|
| **Real-time Synthesis** | Stream audio as it generates, word by word |
| **Conversation Mode** | Multi-speaker dialogues with automatic turn-taking |
| **Voice Effects** | Pitch shift, reverb, M3GAN-style effects |
| **Timeline Editor** | Audio studio with word-level precision editing |
| **More Models** | XTTS, Bark, and other open-source voice models |
### Future Vision
- **Voice Design** — Create new voices from text descriptions
- **Project System** — Save and load complex multi-voice sessions
- **Plugin Architecture** — Extend with custom models and effects
- **Mobile Companion** — Control Voicebox from your phone
Voicebox aims to be the **one-stop shop for everything voice** — cloning, synthesis, editing, effects, and beyond.
---
## Development
See [CONTRIBUTING.md](CONTRIBUTING.md) for detailed setup and contribution guidelines.
**Using the Makefile (recommended):** Run `make help` to see all available commands for setup, development, building, and testing.
### Quick Start
**With Makefile (Unix/macOS/Linux):**
```bash
# Clone the repo
git clone https://github.com/voicebox-sh/voicebox.git
cd voicebox
# Setup everything
make setup
# Start development
make dev
```
**Manual setup (all platforms):**
```bash
# Clone the repo
git clone https://github.com/voicebox-sh/voicebox.git
cd voicebox
# Install dependencies
bun install
# Install Python dependencies
cd backend && pip install -r requirements.txt && cd ..
# Start development
bun run dev
```
**Prerequisites:** [Bun](https://bun.sh), [Rust](https://rustup.rs), [Python 3.11+](https://python.org).
**Performance:**
- **Apple Silicon (M1/M2/M3)**: Uses MLX backend with native Metal acceleration for 4-5x faster inference
- **Windows/Linux/Intel Mac**: Uses PyTorch backend (CUDA GPU recommended, CPU supported but slower)
### Project Structure
```
voicebox/
├── app/ # Shared React frontend (used by web & desktop)
│ ├── src/
│ │ ├── components/ # React components
│ │ │ ├── VoiceProfiles/
│ │ │ ├── Generation/
│ │ │ ├── AudioStudio/
│ │ │ ├── History/
│ │ │ └── ServerSettings/
│ │ ├── lib/
│ │ │ ├── api/ # Generated OpenAPI client
│ │ │ ├── hooks/ # React Query hooks
│ │ │ └── utils/
│ │ ├── types/
│ │ └── App.tsx
│ ├── package.json
│ └── vite.config.ts
│
├── tauri/ # Tauri desktop app (thin wrapper)
│ ├── src/
│ │ └── main.tsx # Entry point, imports from ../app
│ ├── src-tauri/ # Rust backend
│ │ ├── src/
│ │ │ └── main.rs # Sidecar management, IPC
│ │ ├── binaries/ # Bundled Python server
│ │ │ └── voicebox-server-{platform}
│ │ ├── Cargo.toml
│ │ └── tauri.conf.json
│ └── package.json
│
├── web/ # Web deployment (thin wrapper)
│ ├── src/
│ │ └── main.tsx # Entry point, imports from ../app
│ ├── package.json
│ └── vite.config.ts
│
├── backend/ # Python FastAPI server
│ ├── main.py # FastAPI app + server mode
│ ├── models.py # Pydantic models
│ ├── tts.py # TTS inference
│ ├── transcribe.py # Whisper ASR
│ ├── profiles.py # Voice profiles
│ ├── history.py # Generation history
│ ├── studio.py # Audio editing
│ ├── database.py # SQLite ORM
│ ├── utils/
│ │ ├── audio.py # Audio processing
│ │ ├── cache.py # Prompt caching
│ │ └── validation.py
│ ├── requirements.txt
│ └── build_binary.py # PyInstaller build script
│
├── scripts/
│ ├── build-server.sh # Build Python binary for all platforms
│ └── generate-api.sh # Generate OpenAPI client
│
├── data/ # User data
│ ├── profiles/
│ ├── generations/
│ ├── projects/
│ └── voicebox.db
│
├── package.json # Root workspace config
└── docs/
├── ANALYSIS.md # Analysis of existing projects
├── TAURI_PLAN.md # Tauri app structure and bundling strategy
└── ARCHITECTURE.md # Detailed architecture docs
├── app/ # Shared React frontend
├── tauri/ # Desktop app (Tauri + Rust)
├── web/ # Web deployment
├── backend/ # Python FastAPI server
├── landing/ # Marketing website
└── scripts/ # Build & release scripts
```
**Key architectural decisions:**
- **Shared frontend** - `app/` contains all React code, used by both desktop and web
- **Thin wrappers** - `tauri/` and `web/` just configure build tools and entry points
- **Bundled backend** - Python server packaged as sidecar binary with PyInstaller
- **Type-safe API** - OpenAPI schema generated from FastAPI, TypeScript client auto-generated
---
See [TAURI_PLAN.md](./docs/TAURI_PLAN.md) for detailed bundling strategy.
## Contributing
## Lessons from Existing Projects
Contributions welcome! See [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines.
voicebox learns from five existing Qwen3-TTS implementations:
1. Fork the repo
2. Create a feature branch
3. Make your changes
4. Submit a PR
### voice (Rust CLI)
- ✅ Clean Rust/Python IPC pattern
- ✅ M3GAN voice effect
- ✅ Voice profile abstraction
- ❌ No concurrent requests
- ❌ No generation history
## Security
### Voice-Clone-Studio
- ✅ Brilliant voice prompt caching
- ✅ Feature-rich (voice design, presets, conversations)
- ✅ VRAM-efficient model management
- ❌ 2,815-line single file
- ❌ Global state everywhere
Found a security vulnerability? Please report it responsibly. See [SECURITY.md](SECURITY.md) for details.
### Qwen3-TTS_server
- ✅ Clean modular structure
- ✅ FastAPI REST API design
- ✅ Health endpoint for monitoring
- ❌ No authentication or rate limiting
- ❌ No caching or streaming
- ❌ No OpenAPI client generation
### mimic
- ✅ Excellent backend architecture (async, modular)
- ✅ Audio studio with timeline
- ✅ Database-backed history
- ✅ Multi-sample voice profiles
- ❌ 2,794-line app.js frontend
- ❌ Global state in UI
### qwen3-tts-enhanced
- ✅ Multi-reference combination
- ✅ Cross-platform graceful degradation
- ✅ Audio validation
- ✅ Production error handling
- ❌ Still monolithic (1,892 lines)
- ❌ No API layer
See [ANALYSIS.md](./docs/ANALYSIS.md) for detailed breakdown of each project.
## Development Roadmap
### Week 1: Foundation
- Project structure setup
- Backend skeleton (FastAPI + SQLite)
- OpenAPI schema generation
- Frontend skeleton (Tauri + React)
- TypeScript client generation from OpenAPI
- React Query setup
- Basic voice profile CRUD
- Server mode implementation
- Client connection UI
### Week 2: Core Features
- TTS integration
- Voice cloning pipeline
- Voice prompt caching
- Generation history
### Week 3: UX Polish
- Audio playback and preview
- Profile management UI
- History search and filters
- Error handling and validation
### Week 4: Advanced Features
- Multi-reference combination
- Batch generation
- Audio normalization
- M3GAN effect
### Week 5+: Studio Features
- Timeline editor
- Word-level timestamps
- Project system
- Export pipeline
## Technical Decisions
### Why Tauri over Electron?
- Smaller bundle size (Rust vs. Node.js)
- Better performance (native vs. V8)
- Lower memory usage
- Rust for system-level operations
### Why FastAPI over Flask?
- Native async/await support
- Automatic OpenAPI schema generation
- Pydantic validation built-in
- Better performance
### Why OpenAPI + React Query?
- **Type safety end-to-end** - FastAPI generates OpenAPI schema, we generate TypeScript client
- **No manual API code** - Client generated from `openapi.json` using openapi-typescript-codegen
- **Automatic caching** - React Query handles request deduplication and background refetching
- **Optimistic updates** - Update UI immediately, rollback on error
- **DevX** - Full autocomplete and type checking for all API calls
**Example workflow:**
```bash
# Backend generates OpenAPI schema
python backend/main.py --openapi > openapi.json
# Frontend generates TypeScript client
bun run generate-client
# Use type-safe hooks in React
import { useQuery } from '@tanstack/react-query';
import { ProfilesService } from '@/lib/api';
const { data: profiles } = useQuery({
queryKey: ['profiles'],
queryFn: () => ProfilesService.listProfiles()
});
```
### Why Bun over npm/yarn/pnpm?
- **Speed** - 20-30x faster than npm for install operations
- **Drop-in replacement** - Compatible with npm ecosystem, no migration needed
- **Built-in tooling** - Bundler, test runner, and package manager in one
- **Performance** - Faster script execution than Node.js
- **Developer experience** - Better error messages, workspaces support
### Why SQLite over file-based storage?
- Full-text search
- Transactions and integrity
- Migrations via Alembic
- Easy to backup/restore
### Why React over Vue/Svelte?
- Larger ecosystem
- Better TypeScript support
- Familiar to most developers
- Mature tooling
### Why bundle Python server with PyInstaller?
- **No Python installation required** - Users don't need Python on their system
- **Consistent environment** - Exact dependencies bundled, no version conflicts
- **Single-click install** - One installer includes everything
- **Tauri sidecar pattern** - Rust spawns/manages Python process lifecycle
- **Platform-specific binaries** - PyInstaller creates native executables for each platform
**Tradeoffs:**
- Larger bundle size (~500MB with models vs ~50MB without backend)
- Need separate build for each platform (macOS Intel/ARM, Windows, Linux)
- First launch slower (model loading time)
**Alternative considered:** Require users to install Python and run `pip install` - rejected for poor UX
### Why no Docker initially?
- Desktop app, not server deployment
- Users install locally
- Can add later for server mode
## Performance Targets
- **First generation:** < 10 seconds (cold start)
- **Cached generation:** < 2 seconds (warm start)
- **UI responsiveness:** 60 FPS at all times
- **Memory usage:** < 4GB VRAM for small models
- **Startup time:** < 3 seconds to UI
- **Database queries:** < 100ms for history search
## Quality Standards
- **No files over 500 lines** (except auto-generated)
- **Type hints on all Python functions**
- **TypeScript strict mode enabled**
- **OpenAPI client auto-generated from schema**
- **ESLint + Prettier for frontend**
- **Black + isort for backend**
- **All user-facing errors have context**
- **No global mutable state**
- **React Query for all server state**
## Project Status
**Current phase:** Planning and analysis
**Documentation:**
- [ANALYSIS.md](./docs/ANALYSIS.md) - Comprehensive analysis of existing implementations
- [TAURI_PLAN.md](./docs/TAURI_PLAN.md) - Tauri app architecture and Python server bundling strategy
---
## License
TBD
MIT License — see [LICENSE](LICENSE) for details.
## Credits
---
Built by analyzing and learning from:
- voice (Rust CLI)
- Voice-Clone-Studio
- Qwen3-TTS_server
- mimic
- qwen3-tts-enhanced
Powered by Alibaba's Qwen3-TTS model.
<p align="center">
<a href="https://voicebox.sh">voicebox.sh</a>
</p>
+92
View File
@@ -0,0 +1,92 @@
# Security Policy
## Supported Versions
We release patches for security vulnerabilities. Which versions are eligible for receiving such patches depends on the CVSS v3.0 Rating:
| Version | Supported |
| ------- | ------------------ |
| 0.1.x | :white_check_mark: |
| < 0.1 | :x: |
## Reporting a Vulnerability
If you discover a security vulnerability, please report it responsibly:
1. **Do not** open a public GitHub issue
2. Email security details to: [[email protected]](mailto:[email protected])
3. Include:
- Description of the vulnerability
- Steps to reproduce
- Potential impact
- Suggested fix (if any)
We will:
- Acknowledge receipt within 48 hours
- Provide a timeline for addressing the issue
- Keep you informed of progress
- Credit you in the security advisory (if desired)
## Security Best Practices
### For Users
- **Keep Voicebox updated** - Updates include security patches
- **Verify downloads** - Only download from official releases
- **Local processing** - Voice data stays on your machine
- **Network security** - Use HTTPS when connecting to remote servers
### For Developers
- **Dependencies** - Keep all dependencies up to date
- **Code review** - All PRs require review before merging
- **Secrets** - Never commit API keys or signing keys
- **Signing** - All releases are cryptographically signed
## Known Security Considerations
### Local Processing
Voicebox processes all audio locally by default. Your voice data never leaves your machine unless you explicitly enable remote server mode.
### Remote Server Mode
When connecting to a remote server:
- Ensure the server is on a trusted network
- Use HTTPS for remote connections
- Verify server identity before connecting
### Auto-Updates
- Updates are cryptographically signed
- Signature verification happens before installation
- Only HTTPS endpoints are allowed
### Python Server
The embedded Python server:
- Runs locally by default (localhost only)
- Can be configured for remote access
- Uses standard FastAPI security practices
## Disclosure Timeline
- **Day 0**: Vulnerability reported
- **Day 1-2**: Initial assessment and acknowledgment
- **Day 3-7**: Investigation and fix development
- **Day 8-14**: Testing and release preparation
- **Day 15+**: Public disclosure (if applicable)
Timeline may vary based on severity and complexity.
## Security Updates
Security updates will be:
- Released as patch versions (e.g., 0.1.1)
- Documented in CHANGELOG.md
- Announced via GitHub releases
- Automatically delivered via auto-updater
---
Thank you for helping keep Voicebox secure! 🔒
-207
View File
@@ -1,207 +0,0 @@
# voicebox Setup Guide
Quick start guide for setting up the voicebox development environment.
## Prerequisites
- **Bun** - Fast JavaScript runtime and package manager
```bash
curl -fsSL https://bun.sh/install | bash
```
- **Python 3.11+** - For backend development
```bash
python --version # Should be 3.11 or higher
```
- **Rust** - For Tauri desktop app (installed automatically by Tauri CLI)
```bash
rustc --version # Check if installed
```
- **Node.js 18+** (optional) - Fallback if Bun is not available
## Initial Setup
### 1. Install Dependencies
```bash
# Install all workspace dependencies
bun install
```
This will install dependencies for:
- `app/` - Shared React frontend
- `tauri/` - Tauri desktop wrapper
- `web/` - Web deployment wrapper
### 2. Setup Backend
```bash
cd backend
# Create virtual environment
python -m venv venv
# Activate virtual environment
source venv/bin/activate # On macOS/Linux
# or
venv\Scripts\activate # On Windows
# Install Python dependencies
pip install -r requirements.txt
```
### 3. Initialize Database
```bash
cd backend
python -c "from database import init_db; init_db()"
```
This creates the SQLite database at `data/voicebox.db`.
### 4. Install Qwen3-TTS (Optional)
The Qwen3-TTS models are automatically downloaded from HuggingFace Hub on first use. However, you need to install the `qwen_tts` package:
```bash
pip install git+https://github.com/QwenLM/Qwen3-TTS.git
```
**Note:** Models (~2-4GB) will be automatically downloaded on first generation. This may take a few minutes depending on your internet connection.
## Development
### Start Backend Server
```bash
cd backend
source venv/bin/activate # Activate venv if not already active
uvicorn main:app --reload --port 8000
```
Backend will be available at `http://localhost:8000`
### Start Tauri Desktop App
```bash
# From project root
bun run dev
```
Or manually:
```bash
cd tauri
bun run tauri dev
```
This will:
1. Start Vite dev server on port 5173
2. Launch Tauri window pointing to localhost:5173
3. Enable hot reload
### Start Web App
```bash
# From project root
bun run dev:web
```
Or manually:
```bash
cd web
bun run dev
```
Web app will be available at `http://localhost:5174` (or next available port)
## Building
### Build Python Server Binary
```bash
./scripts/build-server.sh
```
This creates a platform-specific binary in `tauri/src-tauri/binaries/`
### Build Tauri Desktop App
```bash
cd tauri
bun run tauri build
```
Creates platform-specific installers:
- macOS: `.app`, `.dmg`
- Windows: `.exe`, `.msi`
- Linux: `.deb`, `.AppImage`
### Build Web App
```bash
cd web
bun run build
```
Output in `web/dist/`
## Generate OpenAPI Client
After starting the backend server:
```bash
./scripts/generate-api.sh
```
This will:
1. Download OpenAPI schema from backend
2. Generate TypeScript client in `app/src/lib/api/`
## Project Structure
```
voicebox/
├── app/ # Shared React frontend
├── tauri/ # Tauri desktop wrapper
├── web/ # Web deployment wrapper
├── backend/ # Python FastAPI server
├── scripts/ # Build and utility scripts
├── data/ # User data (gitignored)
└── docs/ # Documentation
```
## Troubleshooting
### Backend won't start
- Check Python version: `python --version` (needs 3.11+)
- Ensure virtual environment is activated
- Install dependencies: `pip install -r requirements.txt`
### Tauri build fails
- Ensure Rust is installed: `rustc --version`
- Install Tauri CLI: `bunx @tauri-apps/cli install`
- Check `tauri/src-tauri/Cargo.toml` for correct dependencies
### OpenAPI client generation fails
- Ensure backend is running on port 8000
- Check `curl http://localhost:8000/openapi.json` returns valid JSON
- Install openapi-typescript-codegen: `bun add -d openapi-typescript-codegen`
## Model Downloads
Models are automatically downloaded from HuggingFace Hub on first use:
- **Whisper** (transcription): Auto-downloads on first transcription
- **Qwen3-TTS** (voice cloning): Auto-downloads on first generation
First-time usage will be slower due to model downloads, but subsequent runs will use cached models.
## Next Steps
1. ✅ TTS model loading implemented in `backend/tts.py`
2. ✅ API routes implemented in `backend/main.py`
3. Build React components in `app/src/components/`
4. Connect frontend to backend via generated API client
See [README.md](./README.md) for architecture details and [docs/](./docs/) for detailed documentation.
+9 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@voicebox/app",
"version": "0.1.0",
"version": "0.1.10",
"private": true,
"type": "module",
"scripts": {
@@ -13,6 +13,9 @@
"check": "biome check --write src"
},
"dependencies": {
"@dnd-kit/core": "^6.3.1",
"@dnd-kit/sortable": "^10.0.0",
"@dnd-kit/utilities": "^3.2.2",
"@hookform/resolvers": "^3.9.0",
"@radix-ui/react-alert-dialog": "^1.1.1",
"@radix-ui/react-avatar": "^1.1.0",
@@ -30,7 +33,10 @@
"@radix-ui/react-toast": "^1.2.1",
"@tanstack/react-query": "^5.0.0",
"@tanstack/react-query-devtools": "^5.0.0",
"@tanstack/react-router": "^1.157.16",
"@tauri-apps/api": "^2.0.0",
"@tauri-apps/plugin-dialog": "^2.0.0",
"@tauri-apps/plugin-fs": "^2.0.0",
"@tauri-apps/plugin-process": "^2.3.1",
"@tauri-apps/plugin-updater": "^2.9.0",
"class-variance-authority": "^0.7.0",
@@ -38,9 +44,11 @@
"date-fns": "^3.6.0",
"framer-motion": "^12.29.0",
"lucide-react": "^0.454.0",
"motion": "^12.29.0",
"react": "^18.3.0",
"react-dom": "^18.3.0",
"react-hook-form": "^7.53.0",
"react-sound-visualizer": "^1.4.0",
"tailwind-merge": "^2.5.4",
"wavesurfer.js": "^7.0.0",
"zod": "^3.23.8",
+98 -78
View File
@@ -1,23 +1,56 @@
import { useState, useEffect } from 'react';
import { GenerationForm } from '@/components/Generation/GenerationForm';
import { HistoryTable } from '@/components/History/HistoryTable';
import { ConnectionForm } from '@/components/ServerSettings/ConnectionForm';
import { ServerStatus } from '@/components/ServerSettings/ServerStatus';
import { UpdateStatus } from '@/components/ServerSettings/UpdateStatus';
import { ModelManagement } from '@/components/ServerSettings/ModelManagement';
import { Toaster } from '@/components/ui/toaster';
import { ProfileList } from '@/components/VoiceProfiles/ProfileList';
import { Sidebar } from '@/components/Sidebar';
import { AudioPlayer } from '@/components/AudioPlayer/AudioPlayer';
import { UpdateNotification } from '@/components/UpdateNotification';
import { isTauri, startServer, setupWindowCloseHandler } from '@/lib/tauri';
import { useEffect, useRef, useState } from 'react';
import { RouterProvider } from '@tanstack/react-router';
import voiceboxLogo from '@/assets/voicebox-logo.png';
import ShinyText from '@/components/ShinyText';
import { TitleBarDragRegion } from '@/components/TitleBarDragRegion';
import { TOP_SAFE_AREA_PADDING } from '@/lib/constants/ui';
import {
isTauri,
setKeepServerRunning,
setupWindowCloseHandler,
startServer,
} from '@/lib/tauri';
import { cn } from '@/lib/utils/cn';
import { router } from '@/router';
import { useServerStore } from '@/stores/serverStore';
// Track if server is starting to prevent duplicate starts
let serverStarting = false;
const LOADING_MESSAGES = [
'Warming up tensors...',
'Calibrating synthesizer engine...',
'Initializing voice models...',
'Loading neural networks...',
'Preparing audio pipelines...',
'Optimizing waveform generators...',
'Tuning frequency analyzers...',
'Building voice embeddings...',
'Configuring text-to-speech cores...',
'Syncing audio buffers...',
'Establishing model connections...',
'Preprocessing training data...',
'Validating voice samples...',
'Compiling inference engines...',
'Mapping phoneme sequences...',
'Aligning prosody parameters...',
'Activating speech synthesis...',
'Fine-tuning acoustic models...',
'Preparing voice cloning matrices...',
'Initializing Qwen TTS framework...',
];
function App() {
const [activeTab, setActiveTab] = useState('main');
const [serverReady, setServerReady] = useState(false);
const [loadingMessageIndex, setLoadingMessageIndex] = useState(0);
const serverStartingRef = useRef(false);
// Sync stored setting to Rust on startup
useEffect(() => {
if (isTauri()) {
const keepRunning = useServerStore.getState().keepServerRunningOnClose;
setKeepServerRunning(keepRunning).catch((error) => {
console.error('Failed to sync initial setting to Rust:', error);
});
}
}, []);
// Setup window close handler and auto-start server when running in Tauri (production only)
useEffect(() => {
@@ -43,25 +76,25 @@ function App() {
}
// Auto-start server in production
if (serverStarting) {
if (serverStartingRef.current) {
return;
}
serverStarting = true;
serverStartingRef.current = true;
console.log('Production mode: Starting bundled server...');
startServer(false)
.then(() => {
console.log('Server is ready');
.then((serverUrl) => {
console.log('Server is ready at:', serverUrl);
// Update the server URL in the store with the dynamically assigned port
useServerStore.getState().setServerUrl(serverUrl);
setServerReady(true);
// Mark that we started the server (so we know to stop it on close)
// @ts-expect-error - adding property to window
window.__voiceboxServerStartedByApp = true;
})
.catch((error) => {
console.error('Failed to auto-start server:', error);
serverStarting = false;
// @ts-expect-error - adding property to window
serverStartingRef.current = false;
window.__voiceboxServerStartedByApp = false;
});
@@ -69,72 +102,59 @@ function App() {
// Note: Window close is handled separately in Tauri Rust code
return () => {
// Window close event handles server shutdown based on setting
serverStarting = false;
serverStartingRef.current = false;
};
}, []);
// Cycle through loading messages every 3 seconds
useEffect(() => {
if (!isTauri() || serverReady) {
return;
}
const interval = setInterval(() => {
setLoadingMessageIndex((prev) => (prev + 1) % LOADING_MESSAGES.length);
}, 3000);
return () => clearInterval(interval);
}, [serverReady]);
// Show loading screen while server is starting in Tauri
if (isTauri() && !serverReady) {
return (
<div className="min-h-screen bg-background flex items-center justify-center">
<div className="text-center space-y-4">
<div className="animate-spin rounded-full h-12 w-12 border-b-2 border-primary mx-auto"></div>
<p className="text-muted-foreground">Starting server...</p>
<div
className={cn(
'min-h-screen bg-background flex items-center justify-center',
TOP_SAFE_AREA_PADDING,
)}
>
<TitleBarDragRegion />
<div className="text-center space-y-6">
<div className="flex justify-center relative">
<div className="absolute inset-0 flex items-center justify-center">
<div className="w-48 h-48 rounded-full bg-accent/20 blur-3xl" />
</div>
<img
src={voiceboxLogo}
alt="Voicebox"
className="w-48 h-48 object-contain animate-fade-in-scale relative z-10"
/>
</div>
<div className="animate-fade-in-delayed">
<ShinyText
text={LOADING_MESSAGES[loadingMessageIndex]}
className="text-lg font-medium text-muted-foreground"
speed={2}
color="hsl(var(--muted-foreground))"
shineColor="hsl(var(--foreground))"
/>
</div>
</div>
</div>
);
}
return (
<div className="h-screen bg-background flex flex-col overflow-hidden">
<div className="flex flex-1 min-h-0 overflow-hidden">
<Sidebar activeTab={activeTab} onTabChange={setActiveTab} />
<main className="flex-1 ml-20 overflow-hidden flex flex-col">
<div className="container mx-auto px-8 py-8 max-w-[1800px] h-full overflow-hidden flex flex-col">
<UpdateNotification />
{activeTab === 'settings' ? (
<div className="space-y-4 overflow-y-auto">
<div className="grid gap-4 md:grid-cols-2">
<ConnectionForm />
<ServerStatus />
</div>
{isTauri() && <UpdateStatus />}
<ModelManagement />
</div>
) : (
// Main view: Profiles top left, Generator bottom left, History right
<div className="grid grid-cols-1 lg:grid-cols-2 gap-6 h-full min-h-0 overflow-hidden">
{/* Left Column */}
<div className="flex flex-col gap-6 min-h-0 overflow-y-auto pb-32">
{/* Profiles - Top Left */}
<div className="shrink-0 flex flex-col">
<ProfileList />
</div>
{/* Generator - Bottom Left */}
<div className="shrink-0">
<GenerationForm />
</div>
</div>
{/* Right Column - History */}
<div className="flex flex-col min-h-0 overflow-hidden">
<HistoryTable />
</div>
</div>
)}
</div>
</main>
</div>
{/* Audio Player - always visible except on settings */}
{activeTab !== 'settings' && <AudioPlayer />}
<Toaster />
</div>
);
return <RouterProvider router={router} />;
}
export default App;
+35
View File
@@ -0,0 +1,35 @@
import { useRouterState } from '@tanstack/react-router';
import { TitleBarDragRegion } from '@/components/TitleBarDragRegion';
import { AudioPlayer } from '@/components/AudioPlayer/AudioPlayer';
import { StoryTrackEditor } from '@/components/StoriesTab/StoryTrackEditor';
import { TOP_SAFE_AREA_PADDING } from '@/lib/constants/ui';
import { cn } from '@/lib/utils/cn';
import { useStoryStore } from '@/stores/storyStore';
import { useStory } from '@/lib/hooks/useStories';
interface AppFrameProps {
children: React.ReactNode;
}
export function AppFrame({ children }: AppFrameProps) {
const routerState = useRouterState();
const isStoriesRoute = routerState.location.pathname === '/stories';
const selectedStoryId = useStoryStore((state) => state.selectedStoryId);
const { data: story } = useStory(selectedStoryId);
// Show track editor when on stories route with a selected story that has items
const showTrackEditor = isStoriesRoute && selectedStoryId && story && story.items.length > 0;
return (
<div className={cn('h-screen bg-background flex flex-col overflow-hidden', TOP_SAFE_AREA_PADDING)}>
<TitleBarDragRegion />
{children}
{showTrackEditor ? (
<StoryTrackEditor storyId={story.id} items={story.items} />
) : (
<AudioPlayer />
)}
</div>
);
}
+490 -49
View File
@@ -1,30 +1,75 @@
import { Pause, Play, Repeat, Volume2, VolumeX } from 'lucide-react';
import { useEffect, useRef, useState } from 'react';
import { useQuery } from '@tanstack/react-query';
import { invoke } from '@tauri-apps/api/core';
import { Pause, Play, Repeat, Volume2, VolumeX, X } from 'lucide-react';
import { useEffect, useMemo, useRef, useState } from 'react';
import WaveSurfer from 'wavesurfer.js';
import { Button } from '@/components/ui/button';
import { Slider } from '@/components/ui/slider';
import { apiClient } from '@/lib/api/client';
import { isTauri } from '@/lib/tauri';
import { formatAudioDuration } from '@/lib/utils/audio';
import { debug } from '@/lib/utils/debug';
import { usePlayerStore } from '@/stores/playerStore';
export function AudioPlayer() {
const {
audioUrl,
audioId,
profileId,
title,
isPlaying,
currentTime,
duration,
volume,
isLooping,
shouldRestart,
setIsPlaying,
setCurrentTime,
setDuration,
setVolume,
toggleLoop,
clearRestartFlag,
reset,
} = usePlayerStore();
// Check if profile has assigned channels (for native audio routing)
const { data: profileChannels } = useQuery({
queryKey: ['profile-channels', profileId],
queryFn: () => {
if (!profileId) return { channel_ids: [] };
return apiClient.getProfileChannels(profileId);
},
enabled: !!profileId && isTauri(),
});
const { data: channels } = useQuery({
queryKey: ['channels'],
queryFn: () => apiClient.listChannels(),
enabled: !!profileChannels && profileChannels.channel_ids.length > 0,
});
// Determine if we should use native playback
const useNativePlayback = useMemo(() => {
if (!isTauri() || !profileChannels || !channels) {
return false;
}
const assignedChannels = channels.filter((ch) => profileChannels.channel_ids.includes(ch.id));
// Use native playback if any assigned channel has non-default devices
const shouldUseNative = assignedChannels.some(
(ch) => ch.device_ids.length > 0 && !ch.is_default,
);
return shouldUseNative;
}, [profileChannels, channels, profileId]);
const waveformRef = useRef<HTMLDivElement>(null);
const wavesurferRef = useRef<WaveSurfer | null>(null);
const loadingRef = useRef(false);
const previousAudioIdRef = useRef<string | null>(null);
const hasInitializedRef = useRef(false);
const isUsingNativePlaybackRef = useRef(false);
const [isLoading, setIsLoading] = useState(false);
const [error, setError] = useState<string | null>(null);
@@ -36,11 +81,11 @@ export function AudioPlayer() {
}
if (wavesurferRef.current) {
console.log('WaveSurfer already initialized, skipping');
debug.log('WaveSurfer already initialized, skipping');
return;
}
console.log('Creating NEW WaveSurfer instance');
debug.log('Creating NEW WaveSurfer instance');
// Wait for container to be properly rendered
const initWaveSurfer = () => {
@@ -66,7 +111,7 @@ export function AudioPlayer() {
return;
}
console.log('Initializing WaveSurfer...', {
debug.log('Initializing WaveSurfer...', {
container,
width: rect.width,
height: rect.height,
@@ -99,9 +144,9 @@ export function AudioPlayer() {
});
wavesurferRef.current = wavesurfer;
console.log('WaveSurfer created successfully');
debug.log('WaveSurfer created successfully');
} catch (error) {
console.error('Failed to create WaveSurfer:', error);
debug.error('Failed to create WaveSurfer:', error);
setError(
`Failed to initialize waveform: ${error instanceof Error ? error.message : String(error)}`,
);
@@ -117,47 +162,243 @@ export function AudioPlayer() {
});
// Update store when duration is loaded
wavesurfer.on('ready', () => {
wavesurfer.on('ready', async () => {
const dur = wavesurfer.getDuration();
setDuration(dur);
loadingRef.current = false;
setIsLoading(false);
setError(null);
console.log('Audio ready, duration:', dur);
console.log('Waveform should be visible now');
debug.log('Audio ready, duration:', dur);
debug.log('Waveform should be visible now');
// Ensure volume is set
const currentVolume = usePlayerStore.getState().volume;
wavesurfer.setVolume(currentVolume);
// Get the underlying audio element and ensure it's not muted
// (unless we're using native playback, which will be set later)
const mediaElement = wavesurfer.getMediaElement();
if (mediaElement) {
if (mediaElement && !isUsingNativePlaybackRef.current) {
mediaElement.volume = currentVolume;
mediaElement.muted = false;
console.log('Audio element volume:', mediaElement.volume, 'muted:', mediaElement.muted);
debug.log('Audio element volume:', mediaElement.volume, 'muted:', mediaElement.muted);
}
// Auto-play when ready
// Use a small delay to ensure audio element is fully ready
setTimeout(() => {
wavesurfer.play().catch((error) => {
console.error('Failed to autoplay:', error);
// Don't show error for autoplay failures (browser restrictions)
});
}, 100);
// Auto-play when ready - check if we should use native playback
// Get current values from the store and queries at runtime (not captured closure values)
const currentAudioUrl = usePlayerStore.getState().audioUrl;
const currentProfileId = usePlayerStore.getState().profileId;
debug.log('Auto-play check - capturing runtime values...');
// Fetch profile channels at runtime (not using captured value)
let runtimeProfileChannels = null;
let runtimeChannels = null;
if (isTauri() && currentProfileId) {
try {
runtimeProfileChannels = await apiClient.getProfileChannels(currentProfileId);
debug.log('Runtime profileChannels:', runtimeProfileChannels);
if (runtimeProfileChannels && runtimeProfileChannels.channel_ids.length > 0) {
runtimeChannels = await apiClient.listChannels();
debug.log('Runtime channels:', runtimeChannels);
}
} catch (error) {
debug.error('Failed to fetch runtime channel data:', error);
}
}
debug.log('Auto-play check:', {
isTauri: isTauri(),
currentAudioUrl,
currentProfileId,
hasProfileChannels: !!runtimeProfileChannels,
hasChannels: !!runtimeChannels,
});
if (
isTauri() &&
currentAudioUrl &&
currentProfileId &&
runtimeProfileChannels &&
runtimeChannels
) {
debug.log('Attempting native audio playback...');
// Stop any existing native playback first
if (isUsingNativePlaybackRef.current) {
try {
await invoke('stop_audio_playback');
debug.log('Stopped existing native playback before starting new one');
} catch (error) {
debug.error('Failed to stop existing playback:', error);
}
}
try {
// Collect all device IDs from assigned channels
const assignedChannels = runtimeChannels.filter((ch: any) =>
runtimeProfileChannels.channel_ids.includes(ch.id),
);
debug.log('Assigned channels for playback:', assignedChannels);
// Check if any assigned channel has non-default devices
const shouldUseNative = assignedChannels.some(
(ch: any) => ch.device_ids.length > 0 && !ch.is_default,
);
debug.log('Should use native playback:', shouldUseNative);
if (!shouldUseNative) {
debug.log('No custom devices assigned, falling back to WaveSurfer');
// Reset native playback flag and unmute WaveSurfer
isUsingNativePlaybackRef.current = false;
const mediaElement = wavesurfer.getMediaElement();
if (mediaElement) {
const currentVolume = usePlayerStore.getState().volume;
mediaElement.volume = currentVolume;
mediaElement.muted = false;
debug.log(
'WaveSurfer unmuted for normal playback - volume:',
mediaElement.volume,
'muted:',
mediaElement.muted,
);
}
} else {
const deviceIds = assignedChannels.flatMap((ch: any) => ch.device_ids);
debug.log('Device IDs to play to:', deviceIds);
if (deviceIds.length > 0) {
debug.log('Fetching audio data from:', currentAudioUrl);
// Fetch audio data
const response = await fetch(currentAudioUrl);
const audioData = new Uint8Array(await response.arrayBuffer());
debug.log('Audio data size:', audioData.length);
// Play via native audio
debug.log('Invoking play_audio_to_devices...');
try {
const result = await invoke('play_audio_to_devices', {
audioData: Array.from(audioData),
deviceIds: deviceIds,
});
debug.log('play_audio_to_devices completed successfully, result:', result);
// Mark that we're using native playback
isUsingNativePlaybackRef.current = true;
// Mute WaveSurfer's audio element to prevent UI audio output
// Keep WaveSurfer running for visualization
const mediaElement = wavesurfer.getMediaElement();
if (mediaElement) {
mediaElement.volume = 0;
mediaElement.muted = true;
debug.log(
'WaveSurfer muted for native playback - volume:',
mediaElement.volume,
'muted:',
mediaElement.muted,
);
}
// Start WaveSurfer playback for visualization (muted)
wavesurfer.play().catch((error) => {
debug.error('Failed to start WaveSurfer visualization:', error);
});
setIsPlaying(true);
debug.log('Auto-playing via native audio routing - SUCCESS');
return;
} catch (invokeError) {
debug.error('play_audio_to_devices invoke failed:', invokeError);
throw invokeError;
}
} else {
debug.log('No device IDs found, falling back to WaveSurfer');
}
}
} catch (error) {
debug.error(
'Native playback failed during auto-play, falling back to WaveSurfer:',
error,
);
// Reset native playback flag and unmute WaveSurfer
isUsingNativePlaybackRef.current = false;
const mediaElement = wavesurfer.getMediaElement();
if (mediaElement) {
const currentVolume = usePlayerStore.getState().volume;
mediaElement.volume = currentVolume;
mediaElement.muted = false;
debug.log(
'WaveSurfer unmuted after native playback failure - volume:',
mediaElement.volume,
'muted:',
mediaElement.muted,
);
}
// Fall through to WaveSurfer playback
}
} else {
debug.log('Not using native playback, using WaveSurfer');
// Reset native playback flag and unmute WaveSurfer
isUsingNativePlaybackRef.current = false;
const mediaElement = wavesurfer.getMediaElement();
if (mediaElement) {
const currentVolume = usePlayerStore.getState().volume;
mediaElement.volume = currentVolume;
mediaElement.muted = false;
debug.log(
'WaveSurfer unmuted for normal playback - volume:',
mediaElement.volume,
'muted:',
mediaElement.muted,
);
}
}
// Only auto-play if shouldAutoPlay flag is set (user explicitly clicked to play)
const shouldAutoPlayNow = usePlayerStore.getState().shouldAutoPlay;
if (shouldAutoPlayNow) {
// Clear the flag first
usePlayerStore.getState().clearAutoPlayFlag();
// Use a small delay to ensure audio element is fully ready
setTimeout(() => {
wavesurfer.play().catch((error) => {
debug.error('Failed to autoplay:', error);
// Don't show error for autoplay failures (browser restrictions)
});
}, 100);
} else {
debug.log('Skipping auto-play - shouldAutoPlay is false');
}
});
// Handle play/pause
wavesurfer.on('play', () => {
setIsPlaying(true);
// Ensure audio element is not muted when playing
// Ensure audio element volume is set correctly
const mediaElement = wavesurfer.getMediaElement();
if (mediaElement) {
mediaElement.muted = false;
const currentVolume = usePlayerStore.getState().volume;
mediaElement.volume = currentVolume;
console.log('Playing - volume:', mediaElement.volume, 'muted:', mediaElement.muted);
// Double-check: if using native playback, keep WaveSurfer muted
// Otherwise, ensure it's unmuted
if (isUsingNativePlaybackRef.current) {
mediaElement.volume = 0;
mediaElement.muted = true;
debug.log('Playing (native mode) - WaveSurfer muted for visualization only');
} else {
// Ensure WaveSurfer is unmuted for normal playback
const currentVolume = usePlayerStore.getState().volume;
mediaElement.volume = currentVolume;
mediaElement.muted = false;
debug.log(
'Playing (normal mode) - volume:',
mediaElement.volume,
'muted:',
mediaElement.muted,
);
}
}
});
wavesurfer.on('pause', () => setIsPlaying(false));
@@ -169,12 +410,17 @@ export function AudioPlayer() {
wavesurfer.play();
} else {
setIsPlaying(false);
// Trigger finish callback if set
const onFinish = usePlayerStore.getState().onFinish;
if (onFinish) {
onFinish();
}
}
});
// Handle errors
wavesurfer.on('error', (error) => {
console.error('WaveSurfer error:', error);
debug.error('WaveSurfer error:', error);
setIsLoading(false);
setError(`Audio error: ${error instanceof Error ? error.message : String(error)}`);
});
@@ -189,7 +435,7 @@ export function AudioPlayer() {
// Load audio immediately if audioUrl is already set
if (audioUrl) {
console.log('WaveSurfer ready, loading audio:', audioUrl);
debug.log('WaveSurfer ready, loading audio:', audioUrl);
loadingRef.current = true;
setIsLoading(true);
// Stop any current playback before loading new audio
@@ -199,11 +445,11 @@ export function AudioPlayer() {
wavesurfer
.load(audioUrl)
.then(() => {
console.log('Audio loaded into WaveSurfer');
debug.log('Audio loaded into WaveSurfer');
loadingRef.current = false;
})
.catch((error) => {
console.error('Failed to load audio into WaveSurfer:', error);
debug.error('Failed to load audio into WaveSurfer:', error);
loadingRef.current = false;
setIsLoading(false);
setError(
@@ -228,12 +474,12 @@ export function AudioPlayer() {
});
return () => {
console.log('Cleaning up WaveSurfer initialization effect');
debug.log('Cleaning up WaveSurfer initialization effect');
if (rafId1) cancelAnimationFrame(rafId1);
if (rafId2) cancelAnimationFrame(rafId2);
if (timeoutId) clearTimeout(timeoutId);
if (wavesurferRef.current) {
console.log('Destroying WaveSurfer instance');
debug.log('Destroying WaveSurfer instance');
try {
const mediaElement = wavesurferRef.current.getMediaElement();
if (mediaElement) {
@@ -242,7 +488,7 @@ export function AudioPlayer() {
}
wavesurferRef.current.destroy();
} catch (error) {
console.error('Error destroying WaveSurfer:', error);
debug.error('Error destroying WaveSurfer:', error);
}
wavesurferRef.current = null;
}
@@ -263,36 +509,61 @@ export function AudioPlayer() {
setDuration(0);
setCurrentTime(0);
setError(null);
// Reset native playback flag
isUsingNativePlaybackRef.current = false;
}
return;
}
// Stop native playback if it was active
if (isUsingNativePlaybackRef.current && isTauri()) {
(async () => {
try {
await invoke('stop_audio_playback');
debug.log('Stopped native audio playback');
} catch (error) {
debug.error('Failed to stop native playback:', error);
}
})();
}
// Reset native playback flag when loading new audio
// Also unmute WaveSurfer if it was muted
if (isUsingNativePlaybackRef.current) {
const mediaElement = wavesurfer.getMediaElement();
if (mediaElement) {
mediaElement.muted = false;
mediaElement.volume = usePlayerStore.getState().volume;
}
}
isUsingNativePlaybackRef.current = false;
// CRITICAL: Force stop any current playback and cancel any pending loads
// This must happen BEFORE any early returns
console.log('Audio URL changed to:', audioUrl);
debug.log('Audio URL changed to:', audioUrl);
// COMPLETELY stop and destroy the current audio
try {
// First pause if playing
if (wavesurfer.isPlaying()) {
console.log('Pausing current playback');
debug.log('Pausing current playback');
wavesurfer.pause();
}
// Stop the media element explicitly
const mediaElement = wavesurfer.getMediaElement();
if (mediaElement) {
console.log('Stopping media element');
debug.log('Stopping media element');
mediaElement.pause();
mediaElement.currentTime = 0;
mediaElement.src = '';
}
// Use empty() to completely destroy the waveform and media element
console.log('Calling wavesurfer.empty() to destroy audio');
debug.log('Calling wavesurfer.empty() to destroy audio');
wavesurfer.empty();
} catch (error) {
console.error('Error stopping previous audio:', error);
debug.error('Error stopping previous audio:', error);
// Continue anyway to load new audio
}
@@ -307,16 +578,16 @@ export function AudioPlayer() {
setDuration(0);
// Load new audio
console.log('Starting new audio load for:', audioUrl);
debug.log('Starting new audio load for:', audioUrl);
wavesurfer
.load(audioUrl)
.then(() => {
console.log('Audio load promise resolved');
debug.log('Audio load promise resolved');
// Don't set loading to false here - wait for 'ready' event
})
.catch((error) => {
console.error('Failed to load audio:', error);
console.error('Audio URL:', audioUrl);
debug.error('Failed to load audio:', error);
debug.error('Audio URL:', audioUrl);
loadingRef.current = false;
setIsLoading(false);
setError(`Failed to load audio: ${error instanceof Error ? error.message : String(error)}`);
@@ -331,7 +602,7 @@ export function AudioPlayer() {
if (isPlaying && wavesurferRef.current.isPlaying() === false) {
// Only auto-play if audio is ready
wavesurferRef.current.play().catch((error) => {
console.error('Failed to play:', error);
debug.error('Failed to play:', error);
setIsPlaying(false);
setError(`Playback error: ${error instanceof Error ? error.message : String(error)}`);
});
@@ -347,33 +618,176 @@ export function AudioPlayer() {
// Also ensure the underlying audio element volume is set
const mediaElement = wavesurferRef.current.getMediaElement();
if (mediaElement) {
mediaElement.volume = volume;
mediaElement.muted = volume === 0;
console.log('Volume synced:', volume, 'muted:', mediaElement.muted);
// If using native playback, keep WaveSurfer muted regardless of volume setting
if (isUsingNativePlaybackRef.current) {
mediaElement.volume = 0;
mediaElement.muted = true;
debug.log('Volume sync: Using native playback, keeping WaveSurfer muted');
} else {
mediaElement.volume = volume;
mediaElement.muted = volume === 0;
debug.log('Volume synced:', volume, 'muted:', mediaElement.muted);
}
}
}
}, [volume]);
// Mark as initialized when audio is ready, reset when audioId changes
useEffect(() => {
if (duration > 0 && audioId) {
hasInitializedRef.current = true;
}
// Reset initialization flag when audioId changes to a new audio
if (audioId !== previousAudioIdRef.current && previousAudioIdRef.current !== null) {
hasInitializedRef.current = false;
}
if (audioId !== null) {
previousAudioIdRef.current = audioId;
}
}, [duration, audioId]);
// Handle restart flag - when history item is clicked again, restart from beginning
useEffect(() => {
const wavesurfer = wavesurferRef.current;
if (!wavesurfer || !shouldRestart || duration === 0) {
return;
}
// Reset to beginning and play
debug.log('Restarting current audio from beginning');
wavesurfer.seekTo(0);
wavesurfer.play().catch((error) => {
debug.error('Failed to play after restart:', error);
setIsPlaying(false);
setError(`Playback error: ${error instanceof Error ? error.message : String(error)}`);
});
// Clear the restart flag
clearRestartFlag();
}, [shouldRestart, duration, setIsPlaying, clearRestartFlag]);
// Handle shouldAutoPlay flag - for story mode auto-advance
const shouldAutoPlay = usePlayerStore((state) => state.shouldAutoPlay);
const clearAutoPlayFlag = usePlayerStore((state) => state.clearAutoPlayFlag);
useEffect(() => {
const wavesurfer = wavesurferRef.current;
if (!wavesurfer || !shouldAutoPlay || duration === 0) {
return;
}
// Auto-play the newly loaded audio
debug.log('Auto-playing next track in story mode');
wavesurfer.seekTo(0);
wavesurfer.play().catch((error) => {
debug.error('Failed to auto-play:', error);
setIsPlaying(false);
setError(`Playback error: ${error instanceof Error ? error.message : String(error)}`);
});
// Clear the auto-play flag
clearAutoPlayFlag();
}, [shouldAutoPlay, duration, setIsPlaying, clearAutoPlayFlag]);
// Handle loop - WaveSurfer handles this via the 'finish' event
const handlePlayPause = () => {
const handlePlayPause = async () => {
// Standard WaveSurfer playback (works for both normal and native playback modes)
// When using native playback, WaveSurfer is muted but still controls visualization
if (!wavesurferRef.current) {
console.error('WaveSurfer not initialized');
debug.error('WaveSurfer not initialized');
return;
}
// Check if audio is loaded
if (duration === 0 && !isLoading) {
console.error('Audio not loaded yet');
debug.error('Audio not loaded yet');
setError('Audio not loaded. Please wait...');
return;
}
// If using native playback
if (useNativePlayback && audioUrl && profileChannels && channels) {
if (isPlaying) {
// Pause: stop native playback and pause WaveSurfer visualization
try {
await invoke('stop_audio_playback');
debug.log('Stopped native audio playback');
} catch (error) {
debug.error('Failed to stop native playback:', error);
}
wavesurferRef.current.pause();
return;
}
// Play: trigger native playback
try {
// Stop any existing native playback first
try {
await invoke('stop_audio_playback');
} catch (_error) {
// Ignore errors when stopping (might not be playing)
debug.log('No existing playback to stop');
}
// Collect all device IDs from assigned channels
const assignedChannels = channels.filter((ch) =>
profileChannels.channel_ids.includes(ch.id),
);
const deviceIds = assignedChannels.flatMap((ch) => ch.device_ids);
if (deviceIds.length > 0) {
// Fetch audio data
const response = await fetch(audioUrl);
const audioData = new Uint8Array(await response.arrayBuffer());
// Play via native audio
await invoke('play_audio_to_devices', {
audioData: Array.from(audioData),
deviceIds: deviceIds,
});
// Mark that we're using native playback
isUsingNativePlaybackRef.current = true;
// Mute WaveSurfer and start it for visualization
const mediaElement = wavesurferRef.current.getMediaElement();
if (mediaElement) {
mediaElement.volume = 0;
mediaElement.muted = true;
}
// Start WaveSurfer for visualization (muted)
wavesurferRef.current.play().catch((error) => {
debug.error('Failed to start WaveSurfer visualization:', error);
setIsPlaying(false);
setError(`Playback error: ${error instanceof Error ? error.message : String(error)}`);
});
return;
}
} catch (error) {
debug.error('Native playback failed, falling back to WaveSurfer:', error);
// Fall through to WaveSurfer playback
isUsingNativePlaybackRef.current = false;
}
}
// Standard WaveSurfer playback (or fallback from native playback failure)
if (wavesurferRef.current.isPlaying()) {
wavesurferRef.current.pause();
} else {
// Ensure WaveSurfer is not muted if not using native playback
if (!isUsingNativePlaybackRef.current) {
const mediaElement = wavesurferRef.current.getMediaElement();
if (mediaElement) {
mediaElement.muted = false;
mediaElement.volume = volume;
}
}
wavesurferRef.current.play().catch((error) => {
console.error('Failed to play:', error);
debug.error('Failed to play:', error);
setIsPlaying(false);
setError(`Playback error: ${error instanceof Error ? error.message : String(error)}`);
});
@@ -390,6 +804,22 @@ export function AudioPlayer() {
setVolume(value[0] / 100);
};
const handleClose = () => {
// Stop any native playback
if (isUsingNativePlaybackRef.current && isTauri()) {
invoke('stop_audio_playback').catch((error) => {
debug.error('Failed to stop native playback:', error);
});
}
// Stop WaveSurfer
if (wavesurferRef.current) {
wavesurferRef.current.pause();
wavesurferRef.current.seekTo(0);
}
// Reset player state
reset();
};
// Don't render if no audio
if (!audioUrl) {
return null;
@@ -470,6 +900,17 @@ export function AudioPlayer() {
className="flex-1"
/>
</div>
{/* Close Button */}
<Button
variant="ghost"
size="icon"
onClick={handleClose}
className="shrink-0"
title="Close player"
>
<X className="h-5 w-5" />
</Button>
</div>
</div>
</div>
+672
View File
@@ -0,0 +1,672 @@
import { useMutation, useQuery, useQueryClient } from '@tanstack/react-query';
import { invoke } from '@tauri-apps/api/core';
import { Check, CheckCircle2, Edit, Plus, Speaker, Trash2 } from 'lucide-react';
import { useState } from 'react';
import { Badge } from '@/components/ui/badge';
import { Button } from '@/components/ui/button';
import {
Dialog,
DialogContent,
DialogDescription,
DialogFooter,
DialogHeader,
DialogTitle,
} from '@/components/ui/dialog';
import { Input } from '@/components/ui/input';
import { Label } from '@/components/ui/label';
import {
Select,
SelectContent,
SelectItem,
SelectTrigger,
SelectValue,
} from '@/components/ui/select';
import { apiClient } from '@/lib/api/client';
import { BOTTOM_SAFE_AREA_PADDING } from '@/lib/constants/ui';
import { isTauri } from '@/lib/tauri';
import { cn } from '@/lib/utils/cn';
import { usePlayerStore } from '@/stores/playerStore';
interface AudioDevice {
id: string;
name: string;
is_default: boolean;
}
export function AudioTab() {
const [createDialogOpen, setCreateDialogOpen] = useState(false);
const [editingChannel, setEditingChannel] = useState<string | null>(null);
const [selectedChannelId, setSelectedChannelId] = useState<string | null>(null);
const queryClient = useQueryClient();
const audioUrl = usePlayerStore((state) => state.audioUrl);
const isPlayerVisible = !!audioUrl;
const { data: channels, isLoading: channelsLoading } = useQuery({
queryKey: ['channels'],
queryFn: () => apiClient.listChannels(),
});
const { data: devices, isLoading: devicesLoading } = useQuery({
queryKey: ['audio-devices'],
queryFn: async () => {
if (!isTauri()) {
return [];
}
try {
const result = await invoke<AudioDevice[]>('list_audio_output_devices');
return result;
} catch (error) {
console.error('Failed to list audio devices:', error);
return [];
}
},
enabled: isTauri(),
});
const { data: profiles } = useQuery({
queryKey: ['profiles'],
queryFn: () => apiClient.listProfiles(),
});
const createChannel = useMutation({
mutationFn: (data: { name: string; device_ids: string[] }) => apiClient.createChannel(data),
onSuccess: () => {
queryClient.invalidateQueries({ queryKey: ['channels'] });
setCreateDialogOpen(false);
},
});
const updateChannel = useMutation({
mutationFn: ({
channelId,
data,
}: {
channelId: string;
data: { name?: string; device_ids?: string[] };
}) => apiClient.updateChannel(channelId, data),
onSuccess: () => {
queryClient.invalidateQueries({ queryKey: ['channels'] });
queryClient.invalidateQueries({ queryKey: ['profile-channels'] });
setEditingChannel(null);
},
});
const deleteChannel = useMutation({
mutationFn: (channelId: string) => apiClient.deleteChannel(channelId),
onSuccess: () => {
queryClient.invalidateQueries({ queryKey: ['channels'] });
queryClient.invalidateQueries({ queryKey: ['profile-channels'] });
},
});
const { data: channelVoices } = useQuery({
queryKey: ['channel-voices', editingChannel],
queryFn: async () => {
if (!editingChannel) return { profile_ids: [] };
return apiClient.getChannelVoices(editingChannel);
},
enabled: !!editingChannel,
});
const setChannelVoices = useMutation({
mutationFn: ({ channelId, profileIds }: { channelId: string; profileIds: string[] }) =>
apiClient.setChannelVoices(channelId, profileIds),
onSuccess: () => {
queryClient.invalidateQueries({ queryKey: ['channel-voices'] });
queryClient.invalidateQueries({ queryKey: ['profile-channels'] });
},
});
if (channelsLoading || devicesLoading) {
return (
<div className="flex items-center justify-center h-full">
<div className="text-muted-foreground">Loading...</div>
</div>
);
}
const allChannels = channels || [];
const allDevices = devices || [];
const selectedChannel = selectedChannelId
? allChannels.find((c) => c.id === selectedChannelId)
: null;
return (
<div className="h-full flex flex-col">
<div className="flex items-center justify-between mb-6 shrink-0">
<h2 className="text-2xl font-bold">Audio Channels</h2>
<Button onClick={() => setCreateDialogOpen(true)}>
<Plus className="h-4 w-4 mr-2" />
New Channel
</Button>
</div>
<div className="grid grid-cols-1 lg:grid-cols-2 gap-6 h-full min-h-0">
{/* Left Column - Channels */}
<div
className={cn(
'flex flex-col min-h-0 overflow-y-auto',
isPlayerVisible && BOTTOM_SAFE_AREA_PADDING,
)}
>
{allChannels.length === 0 ? (
<div className="flex flex-col items-center justify-center py-12 border-2 border-dashed border-muted rounded-md">
<Speaker className="h-12 w-12 text-muted-foreground mb-4" />
<p className="text-muted-foreground mb-4">
No audio channels yet. Create your first channel to route voices to specific
devices.
</p>
<Button onClick={() => setCreateDialogOpen(true)}>
<Plus className="h-4 w-4 mr-2" />
Create Channel
</Button>
</div>
) : (
<div className="space-y-3 p-2">
{allChannels.map((channel) => {
const isSelected = selectedChannelId === channel.id;
return (
<button
key={channel.id}
type="button"
className={cn(
'group border rounded-lg p-4 transition-colors cursor-pointer text-left w-full',
isSelected && 'ring-2 ring-primary bg-primary/5 border-primary',
)}
onClick={() => setSelectedChannelId(isSelected ? null : channel.id)}
>
<div className="flex items-start justify-between gap-4">
<div className="flex-1 min-w-0">
<div className="flex items-center gap-2 mb-3">
<div className="h-8 w-8 rounded-lg bg-muted flex items-center justify-center shrink-0">
<Speaker className="h-4 w-4 text-muted-foreground" />
</div>
<div className="flex items-center gap-2 min-w-0">
<h3 className="font-semibold text-base truncate">{channel.name}</h3>
</div>
</div>
<div className="space-y-2.5 ml-10">
<div>
<div className="text-xs font-medium text-muted-foreground mb-1">
Output Devices
</div>
<div className="flex flex-wrap gap-1.5">
{channel.device_ids.length > 0
? channel.device_ids.map((deviceId) => {
const device = allDevices.find((d) => d.id === deviceId);
return (
<Badge
key={deviceId}
variant="outline"
className="text-xs font-normal"
>
{device?.name || deviceId}
</Badge>
);
})
: (() => {
const defaultDevice = allDevices.find((d) => d.is_default);
return defaultDevice ? (
<Badge variant="outline" className="text-xs font-normal">
{defaultDevice.name}
</Badge>
) : null;
})()}
</div>
</div>
<div>
<div className="text-xs font-medium text-muted-foreground mb-1">
Assigned Voices
</div>
<ChannelVoicesList channelId={channel.id} />
</div>
</div>
</div>
{!channel.is_default && (
<div className="flex gap-1 shrink-0 opacity-0 group-hover:opacity-100 transition-opacity">
<Button
variant="ghost"
size="sm"
className="h-8 w-8 p-0"
onClick={(e) => {
e.stopPropagation();
setEditingChannel(channel.id);
}}
>
<Edit className="h-4 w-4" />
</Button>
<Button
variant="ghost"
size="sm"
className="h-8 w-8 p-0"
onClick={(e) => {
e.stopPropagation();
if (confirm('Delete this channel?')) {
deleteChannel.mutate(channel.id);
}
}}
>
<Trash2 className="h-4 w-4" />
</Button>
</div>
)}
</div>
</button>
);
})}
</div>
)}
</div>
{/* Right Column - Available Devices */}
<div
className={cn(
'flex flex-col min-h-0 overflow-y-auto',
isPlayerVisible && BOTTOM_SAFE_AREA_PADDING,
)}
>
<div className="shrink-0 mb-4">
<h3 className="text-lg font-semibold">Available Devices</h3>
<p className="text-sm text-muted-foreground mt-1">
{selectedChannelId
? selectedChannel?.is_default
? 'Default channel uses system default device'
: 'Click devices to add or remove them from the selected channel'
: 'Select a channel to assign devices'}
</p>
</div>
{allDevices.length > 0 ? (
<div className="space-y-2">
{allDevices.map((device) => {
const isConnected =
selectedChannelId &&
selectedChannel &&
(selectedChannel.device_ids.length === 0
? device.is_default
: selectedChannel.device_ids.includes(device.id));
const canToggle =
selectedChannelId && selectedChannel && !selectedChannel.is_default;
const handleDeviceClick = () => {
if (!canToggle || !selectedChannel) return;
const currentDeviceIds = selectedChannel.device_ids;
const newDeviceIds = isConnected
? currentDeviceIds.filter((id) => id !== device.id)
: [...currentDeviceIds, device.id];
updateChannel.mutate({
channelId: selectedChannelId,
data: { device_ids: newDeviceIds },
});
};
return (
<button
key={device.id}
type="button"
onClick={handleDeviceClick}
disabled={!canToggle}
className={cn(
'flex items-center gap-2 text-sm p-3 rounded-lg border transition-colors text-left w-full',
isConnected
? 'bg-primary/10 border-primary ring-1 ring-primary/20'
: 'hover:bg-muted/50',
!canToggle && 'cursor-default opacity-60',
canToggle && 'cursor-pointer',
)}
>
{canToggle ? (
<div
className={cn(
'h-4 w-4 rounded border-2 flex items-center justify-center shrink-0',
isConnected ? 'bg-accent border-accent' : 'border-muted-foreground/30',
)}
>
{isConnected && <Check className="h-3 w-3 text-accent-foreground" />}
</div>
) : device.is_default ? (
<CheckCircle2 className="h-4 w-4 text-primary shrink-0" />
) : null}
<span className={cn('truncate flex-1', device.is_default && 'font-medium')}>
{device.name}
</span>
</button>
);
})}
</div>
) : (
<div className="flex flex-col items-center justify-center py-12 border-2 border-dashed border-muted rounded-md">
<CheckCircle2 className="h-12 w-12 text-muted-foreground mb-4" />
<p className="text-muted-foreground text-center">
{isTauri() ? 'No audio devices found' : 'Audio device selection requires Tauri'}
</p>
</div>
)}
</div>
</div>
{/* Create Channel Dialog */}
<CreateChannelDialog
open={createDialogOpen}
onOpenChange={setCreateDialogOpen}
devices={devices || []}
onCreate={(name, deviceIds) => {
createChannel.mutate({ name, device_ids: deviceIds });
}}
/>
{/* Edit Channel Dialog */}
{editingChannel &&
(() => {
const channel = channels?.find((c) => c.id === editingChannel);
return channel ? (
<EditChannelDialog
open={!!editingChannel}
onOpenChange={(open) => !open && setEditingChannel(null)}
channel={channel}
devices={devices || []}
profiles={profiles || []}
channelVoices={channelVoices?.profile_ids || []}
onUpdate={(name, deviceIds) => {
updateChannel.mutate({
channelId: editingChannel,
data: { name, device_ids: deviceIds },
});
}}
onSetVoices={(profileIds) => {
setChannelVoices.mutate({
channelId: editingChannel,
profileIds,
});
}}
/>
) : null;
})()}
</div>
);
}
function ChannelVoicesList({ channelId }: { channelId: string }) {
const { data: voices } = useQuery({
queryKey: ['channel-voices', channelId],
queryFn: () => apiClient.getChannelVoices(channelId),
});
const { data: profiles } = useQuery({
queryKey: ['profiles'],
queryFn: () => apiClient.listProfiles(),
});
const voiceNames =
voices?.profile_ids.map((id) => profiles?.find((p) => p.id === id)?.name).filter(Boolean) || [];
return (
<div className="flex flex-wrap gap-1.5">
{voiceNames.length > 0 ? (
voiceNames.map((name) => (
<Badge key={name} variant="outline" className="text-xs font-normal">
{name}
</Badge>
))
) : (
<span className="text-sm text-muted-foreground">No voices assigned</span>
)}
</div>
);
}
interface CreateChannelDialogProps {
open: boolean;
onOpenChange: (open: boolean) => void;
devices: AudioDevice[];
onCreate: (name: string, deviceIds: string[]) => void;
}
function CreateChannelDialog({ open, onOpenChange, devices, onCreate }: CreateChannelDialogProps) {
const [name, setName] = useState('');
const [selectedDevices, setSelectedDevices] = useState<string[]>([]);
const handleSubmit = () => {
if (name.trim()) {
onCreate(name.trim(), selectedDevices);
setName('');
setSelectedDevices([]);
}
};
return (
<Dialog open={open} onOpenChange={onOpenChange}>
<DialogContent>
<DialogHeader>
<DialogTitle>Create Audio Channel</DialogTitle>
<DialogDescription>
Create a new audio channel (bus) to route voices to specific output devices.
</DialogDescription>
</DialogHeader>
<div className="space-y-4">
<div>
<Label htmlFor="channel-name">Channel Name</Label>
<Input
id="channel-name"
value={name}
onChange={(e) => setName(e.target.value)}
placeholder="e.g., Virtual Cable, Broadcast"
/>
</div>
<div>
<Label>Output Devices</Label>
<Select
value={selectedDevices[0] || ''}
onValueChange={(value) => {
if (value && !selectedDevices.includes(value)) {
setSelectedDevices([...selectedDevices, value]);
}
}}
>
<SelectTrigger>
<SelectValue placeholder="Select device" />
</SelectTrigger>
<SelectContent>
{devices.map((device) => (
<SelectItem key={device.id} value={device.id}>
{device.name} {device.is_default && '(default)'}
</SelectItem>
))}
</SelectContent>
</Select>
{selectedDevices.length > 0 && (
<div className="mt-2 space-y-1">
{selectedDevices.map((deviceId) => {
const device = devices.find((d) => d.id === deviceId);
return (
<div
key={deviceId}
className="flex items-center justify-between text-sm bg-muted p-2 rounded"
>
<span>{device?.name || deviceId}</span>
<Button
variant="ghost"
size="sm"
onClick={() =>
setSelectedDevices(selectedDevices.filter((id) => id !== deviceId))
}
>
<Trash2 className="h-3 w-3" />
</Button>
</div>
);
})}
</div>
)}
</div>
</div>
<DialogFooter>
<Button variant="outline" onClick={() => onOpenChange(false)}>
Cancel
</Button>
<Button onClick={handleSubmit} disabled={!name.trim()}>
Create
</Button>
</DialogFooter>
</DialogContent>
</Dialog>
);
}
interface EditChannelDialogProps {
open: boolean;
onOpenChange: (open: boolean) => void;
channel: {
id: string;
name: string;
device_ids: string[];
};
devices: AudioDevice[];
profiles: Array<{ id: string; name: string }>;
channelVoices: string[];
onUpdate: (name: string, deviceIds: string[]) => void;
onSetVoices: (profileIds: string[]) => void;
}
function EditChannelDialog({
open,
onOpenChange,
channel,
devices,
profiles,
channelVoices,
onUpdate,
onSetVoices,
}: EditChannelDialogProps) {
const [name, setName] = useState(channel.name);
const [selectedDevices, setSelectedDevices] = useState<string[]>(channel.device_ids);
const [selectedVoices, setSelectedVoices] = useState<string[]>(channelVoices);
const handleSubmit = () => {
if (name.trim()) {
onUpdate(name.trim(), selectedDevices);
onSetVoices(selectedVoices);
}
};
return (
<Dialog open={open} onOpenChange={onOpenChange}>
<DialogContent className="max-w-2xl">
<DialogHeader>
<DialogTitle>Edit Channel</DialogTitle>
<DialogDescription>Update channel settings and voice assignments.</DialogDescription>
</DialogHeader>
<div className="space-y-4">
<div>
<Label htmlFor="edit-channel-name">Channel Name</Label>
<Input id="edit-channel-name" value={name} onChange={(e) => setName(e.target.value)} />
</div>
<div>
<Label>Output Devices</Label>
<Select
value=""
onValueChange={(value) => {
if (value && !selectedDevices.includes(value)) {
setSelectedDevices([...selectedDevices, value]);
}
}}
>
<SelectTrigger>
<SelectValue placeholder="Add device" />
</SelectTrigger>
<SelectContent>
{devices.map((device) => (
<SelectItem key={device.id} value={device.id}>
{device.name} {device.is_default && '(default)'}
</SelectItem>
))}
</SelectContent>
</Select>
{selectedDevices.length > 0 && (
<div className="mt-2 space-y-1">
{selectedDevices.map((deviceId) => {
const device = devices.find((d) => d.id === deviceId);
return (
<div
key={deviceId}
className="flex items-center justify-between text-sm bg-muted p-2 rounded"
>
<span>{device?.name || deviceId}</span>
<Button
variant="ghost"
size="sm"
onClick={() =>
setSelectedDevices(selectedDevices.filter((id) => id !== deviceId))
}
>
<Trash2 className="h-3 w-3" />
</Button>
</div>
);
})}
</div>
)}
</div>
<div>
<Label>Assigned Voices</Label>
<Select
value=""
onValueChange={(value) => {
if (value && !selectedVoices.includes(value)) {
setSelectedVoices([...selectedVoices, value]);
}
}}
>
<SelectTrigger>
<SelectValue placeholder="Add voice" />
</SelectTrigger>
<SelectContent>
{profiles.map((profile) => (
<SelectItem key={profile.id} value={profile.id}>
{profile.name}
</SelectItem>
))}
</SelectContent>
</Select>
{selectedVoices.length > 0 && (
<div className="mt-2 space-y-1">
{selectedVoices.map((profileId) => {
const profile = profiles.find((p) => p.id === profileId);
return (
<div
key={profileId}
className="flex items-center justify-between text-sm bg-muted p-2 rounded"
>
<span>{profile?.name || profileId}</span>
<Button
variant="ghost"
size="sm"
onClick={() =>
setSelectedVoices(selectedVoices.filter((id) => id !== profileId))
}
>
<Trash2 className="h-3 w-3" />
</Button>
</div>
);
})}
</div>
)}
</div>
</div>
<DialogFooter>
<Button variant="outline" onClick={() => onOpenChange(false)}>
Cancel
</Button>
<Button onClick={handleSubmit} disabled={!name.trim()}>
Save
</Button>
</DialogFooter>
</DialogContent>
</Dialog>
);
}
-1
View File
@@ -1 +0,0 @@
# Voice generation components
@@ -0,0 +1,423 @@
import { useMatchRoute } from '@tanstack/react-router';
import { AnimatePresence, motion } from 'framer-motion';
import { Loader2, MessageSquare, Sparkles } from 'lucide-react';
import { useEffect, useRef, useState } from 'react';
import { Button } from '@/components/ui/button';
import { Form, FormControl, FormField, FormItem, FormMessage } from '@/components/ui/form';
import {
Select,
SelectContent,
SelectItem,
SelectTrigger,
SelectValue,
} from '@/components/ui/select';
import { Textarea } from '@/components/ui/textarea';
import { useToast } from '@/components/ui/use-toast';
import { LANGUAGE_OPTIONS } from '@/lib/constants/languages';
import { useGenerationForm } from '@/lib/hooks/useGenerationForm';
import { useProfile, useProfiles } from '@/lib/hooks/useProfiles';
import { useAddStoryItem, useStory } from '@/lib/hooks/useStories';
import { cn } from '@/lib/utils/cn';
import { useStoryStore } from '@/stores/storyStore';
import { useUIStore } from '@/stores/uiStore';
interface FloatingGenerateBoxProps {
isPlayerOpen?: boolean;
showVoiceSelector?: boolean;
}
export function FloatingGenerateBox({
isPlayerOpen = false,
showVoiceSelector = false,
}: FloatingGenerateBoxProps) {
const selectedProfileId = useUIStore((state) => state.selectedProfileId);
const setSelectedProfileId = useUIStore((state) => state.setSelectedProfileId);
const { data: selectedProfile } = useProfile(selectedProfileId || '');
const { data: profiles } = useProfiles();
const [isExpanded, setIsExpanded] = useState(false);
const [isInstructMode, setIsInstructMode] = useState(false);
const containerRef = useRef<HTMLDivElement>(null);
const textareaRef = useRef<HTMLTextAreaElement | null>(null);
const matchRoute = useMatchRoute();
const isStoriesRoute = matchRoute({ to: '/stories' });
const selectedStoryId = useStoryStore((state) => state.selectedStoryId);
const trackEditorHeight = useStoryStore((state) => state.trackEditorHeight);
const { data: currentStory } = useStory(selectedStoryId);
const addStoryItem = useAddStoryItem();
const { toast } = useToast();
// Calculate if track editor is visible (on stories route with items)
const hasTrackEditor = isStoriesRoute && currentStory && currentStory.items.length > 0;
const { form, handleSubmit, isPending } = useGenerationForm({
onSuccess: async (generationId) => {
setIsExpanded(false);
// If on stories route and a story is selected, add generation to story
if (isStoriesRoute && selectedStoryId && generationId) {
try {
await addStoryItem.mutateAsync({
storyId: selectedStoryId,
data: { generation_id: generationId },
});
toast({
title: 'Added to story',
description: `Generation added to "${currentStory?.name || 'story'}"`,
});
} catch (error) {
toast({
title: 'Failed to add to story',
description:
error instanceof Error ? error.message : 'Could not add generation to story',
variant: 'destructive',
});
}
}
},
});
// Click away handler to collapse the box
useEffect(() => {
function handleClickOutside(event: MouseEvent) {
const target = event.target as HTMLElement;
// Don't collapse if clicking inside the container
if (containerRef.current?.contains(target)) {
return;
}
// Don't collapse if clicking on a Select dropdown (which renders in a portal)
if (
target.closest('[role="listbox"]') ||
target.closest('[data-radix-popper-content-wrapper]')
) {
return;
}
setIsExpanded(false);
}
if (isExpanded) {
document.addEventListener('mousedown', handleClickOutside);
}
return () => {
document.removeEventListener('mousedown', handleClickOutside);
};
}, [isExpanded]);
// Set first voice as default if none selected
useEffect(() => {
if (!selectedProfileId && profiles && profiles.length > 0) {
setSelectedProfileId(profiles[0].id);
}
}, [selectedProfileId, profiles, setSelectedProfileId]);
// Auto-resize textarea based on content (only when expanded)
useEffect(() => {
if (!isExpanded) {
// Reset textarea height after collapse animation completes
const timeoutId = setTimeout(() => {
const textarea = textareaRef.current;
if (textarea) {
textarea.style.height = '32px';
textarea.style.overflowY = 'hidden';
}
}, 200); // Wait for animation to complete
return () => clearTimeout(timeoutId);
}
const textarea = textareaRef.current;
if (!textarea) return;
const adjustHeight = () => {
textarea.style.height = 'auto';
const scrollHeight = textarea.scrollHeight;
const minHeight = 100; // Expanded minimum
const maxHeight = 300; // Max height in pixels
const targetHeight = Math.max(minHeight, Math.min(scrollHeight, maxHeight));
textarea.style.height = `${targetHeight}px`;
// Show scrollbar if content exceeds max height
if (scrollHeight > maxHeight) {
textarea.style.overflowY = 'auto';
} else {
textarea.style.overflowY = 'hidden';
}
};
// Small delay to let framer animation complete
const timeoutId = setTimeout(() => {
adjustHeight();
}, 200);
// Adjust on mount and when value changes
adjustHeight();
// Watch for input changes
textarea.addEventListener('input', adjustHeight);
return () => {
clearTimeout(timeoutId);
textarea.removeEventListener('input', adjustHeight);
};
}, [isExpanded]);
async function onSubmit(data: Parameters<typeof handleSubmit>[0]) {
await handleSubmit(data, selectedProfileId);
}
return (
<motion.div
ref={containerRef}
className={cn(
'fixed right-auto',
isStoriesRoute
? // Position aligned with story list: after sidebar + padding, width 360px
'left-[calc(5rem+2rem)] w-[360px]'
: 'left-[calc(5rem+2rem)] w-[calc((100%-5rem-4rem)/2-1rem)]',
)}
style={{
// On stories route: offset by track editor height when visible
// On other routes: offset by audio player height when visible
bottom: hasTrackEditor
? `${trackEditorHeight + 24}px`
: isPlayerOpen
? 'calc(7rem + 1.5rem)'
: '1.5rem',
}}
>
<motion.div
className="bg-background/30 backdrop-blur-2xl border border-accent/20 rounded-[2rem] shadow-2xl hover:bg-background/40 hover:border-accent/20 transition-all duration-300 overflow-hidden p-3"
transition={{ duration: 0.6, ease: 'easeInOut' }}
>
<Form {...form}>
<form onSubmit={form.handleSubmit(onSubmit)}>
<div className="flex gap-2">
<motion.div
className={cn('flex-1', isExpanded && 'mr-12')}
transition={{ duration: 0.3, ease: 'easeOut' }}
>
{/* Text field - hidden when in instruct mode */}
<div style={{ display: isInstructMode ? 'none' : 'block' }}>
<FormField
control={form.control}
name="text"
render={({ field }) => (
<FormItem>
<FormControl>
<motion.div
animate={{
height: isExpanded ? 'auto' : '32px',
}}
transition={{ duration: 0.15, ease: 'easeOut' }}
style={{ overflow: 'hidden' }}
>
<Textarea
{...field}
ref={(node: HTMLTextAreaElement | null) => {
// Store ref for auto-resize (only for active field)
if (!isInstructMode) {
textareaRef.current = node;
}
// Forward ref to react-hook-form
if (typeof field.ref === 'function') {
field.ref(node);
}
}}
placeholder={
isStoriesRoute && currentStory
? `Generate speech for "${currentStory.name}"...`
: selectedProfile
? `Generate speech using ${selectedProfile.name}...`
: 'Select a voice profile above...'
}
className="resize-none bg-transparent border-none focus-visible:ring-0 focus-visible:ring-offset-0 focus:outline-none focus:ring-0 outline-none ring-0 rounded-2xl text-sm placeholder:text-muted-foreground/60 w-full"
style={{
minHeight: isExpanded ? '100px' : '32px',
maxHeight: '300px',
}}
disabled={!selectedProfileId}
onClick={() => setIsExpanded(true)}
onFocus={() => setIsExpanded(true)}
/>
</motion.div>
</FormControl>
<FormMessage className="text-xs" />
</FormItem>
)}
/>
</div>
{/* Instruct field - hidden when in text mode */}
<div style={{ display: isInstructMode ? 'block' : 'none' }}>
<FormField
control={form.control}
name="instruct"
render={({ field }) => (
<FormItem>
<FormControl>
<motion.div
animate={{
height: isExpanded ? 'auto' : '32px',
}}
transition={{ duration: 0.15, ease: 'easeOut' }}
style={{ overflow: 'hidden' }}
>
<Textarea
{...field}
ref={(node: HTMLTextAreaElement | null) => {
// Store ref for auto-resize (only for active field)
if (isInstructMode) {
textareaRef.current = node;
}
// Forward ref to react-hook-form
if (typeof field.ref === 'function') {
field.ref(node);
}
}}
placeholder="Add delivery instructions..."
className="resize-none bg-transparent border-none focus-visible:ring-0 focus-visible:ring-offset-0 focus:outline-none focus:ring-0 outline-none ring-0 rounded-2xl text-sm placeholder:text-muted-foreground/60 w-full"
style={{
minHeight: isExpanded ? '100px' : '32px',
maxHeight: '300px',
}}
disabled={!selectedProfileId}
onClick={() => setIsExpanded(true)}
onFocus={() => setIsExpanded(true)}
/>
</motion.div>
</FormControl>
<FormMessage className="text-xs" />
</FormItem>
)}
/>
</div>
</motion.div>
<div className="relative shrink-0">
<Button
type="submit"
disabled={isPending || !selectedProfileId}
className="h-10 w-10 rounded-full bg-accent hover:bg-accent/90 hover:scale-105 text-accent-foreground shadow-lg hover:shadow-accent/50 transition-all duration-200"
size="icon"
>
{isPending ? (
<Loader2 className="h-4 w-4 animate-spin" />
) : (
<Sparkles className="h-4 w-4" />
)}
</Button>
<AnimatePresence>
{isExpanded && (
<motion.div
initial={{ opacity: 0, scale: 0.8 }}
animate={{ opacity: 1, scale: 1 }}
exit={{ opacity: 0, scale: 0.8 }}
transition={{ duration: 0.2 }}
className="absolute top-0 right-[calc(100%+0.5rem)]"
>
<Button
type="button"
variant="ghost"
size="icon"
onClick={() => setIsInstructMode(!isInstructMode)}
className={cn(
'h-10 w-10 rounded-full transition-all duration-200',
isInstructMode
? 'bg-accent text-accent-foreground border border-accent hover:bg-accent/90'
: 'bg-card border border-border hover:bg-background/50',
)}
>
<MessageSquare className="h-4 w-4" />
</Button>
</motion.div>
)}
</AnimatePresence>
</div>
</div>
<AnimatePresence>
<motion.div
initial={{ height: 0, opacity: 0 }}
animate={{ height: 'auto', opacity: 1 }}
exit={{ height: 0, opacity: 0 }}
transition={{ duration: 0.3, ease: 'easeOut' }}
className=" mt-3"
>
<div className="flex items-center gap-2">
{showVoiceSelector && (
<div className="flex-1">
<Select
value={selectedProfileId || ''}
onValueChange={(value) => setSelectedProfileId(value || null)}
>
<SelectTrigger className="h-8 text-xs bg-card border-border rounded-full hover:bg-background/50 transition-all w-full">
<SelectValue placeholder="Select a voice..." />
</SelectTrigger>
<SelectContent>
{profiles?.map((profile) => (
<SelectItem key={profile.id} value={profile.id} className="text-xs">
{profile.name}
</SelectItem>
))}
</SelectContent>
</Select>
</div>
)}
<FormField
control={form.control}
name="language"
render={({ field }) => (
<FormItem className="flex-1 space-y-0">
<Select onValueChange={field.onChange} defaultValue={field.value}>
<FormControl>
<SelectTrigger className="h-8 text-xs bg-card border-border rounded-full hover:bg-background/50 transition-all">
<SelectValue />
</SelectTrigger>
</FormControl>
<SelectContent>
{LANGUAGE_OPTIONS.map((lang) => (
<SelectItem key={lang.value} value={lang.value} className="text-xs">
{lang.label}
</SelectItem>
))}
</SelectContent>
</Select>
<FormMessage className="text-xs" />
</FormItem>
)}
/>
<FormField
control={form.control}
name="modelSize"
render={({ field }) => (
<FormItem className="flex-1 space-y-0">
<Select onValueChange={field.onChange} defaultValue={field.value}>
<FormControl>
<SelectTrigger className="h-8 text-xs bg-card border-border rounded-full hover:bg-background/50 transition-all">
<SelectValue />
</SelectTrigger>
</FormControl>
<SelectContent>
<SelectItem value="1.7B" className="text-xs text-muted-foreground">
Qwen3-TTS 1.7B
</SelectItem>
<SelectItem value="0.6B" className="text-xs text-muted-foreground">
Qwen3-TTS 0.6B
</SelectItem>
</SelectContent>
</Select>
<FormMessage className="text-xs" />
</FormItem>
)}
/>
</div>
</motion.div>
</AnimatePresence>
</form>
</Form>
</motion.div>
</motion.div>
);
}
@@ -1,8 +1,4 @@
import { zodResolver } from '@hookform/resolvers/zod';
import { Loader2, Mic } from 'lucide-react';
import { useForm } from 'react-hook-form';
import * as z from 'zod';
import { Badge } from '@/components/ui/badge';
import { Button } from '@/components/ui/button';
import { Card, CardContent, CardHeader, CardTitle } from '@/components/ui/card';
import {
@@ -23,71 +19,19 @@ import {
SelectValue,
} from '@/components/ui/select';
import { Textarea } from '@/components/ui/textarea';
import { useToast } from '@/components/ui/use-toast';
import { useGeneration } from '@/lib/hooks/useGeneration';
import { LANGUAGE_OPTIONS } from '@/lib/constants/languages';
import { useGenerationForm } from '@/lib/hooks/useGenerationForm';
import { useProfile } from '@/lib/hooks/useProfiles';
import { useUIStore } from '@/stores/uiStore';
const generationSchema = z.object({
text: z.string().min(1, 'Text is required').max(5000),
language: z.enum(['en', 'zh']),
seed: z.number().int().optional(),
modelSize: z.enum(['1.7B', '0.6B']).optional(),
instruct: z.string().max(500).optional(),
});
type GenerationFormValues = z.infer<typeof generationSchema>;
export function GenerationForm() {
const selectedProfileId = useUIStore((state) => state.selectedProfileId);
const { data: selectedProfile } = useProfile(selectedProfileId || '');
const generation = useGeneration();
const { toast } = useToast();
const form = useForm<GenerationFormValues>({
resolver: zodResolver(generationSchema),
defaultValues: {
text: '',
language: 'en',
seed: undefined,
modelSize: '1.7B',
instruct: '',
},
});
const { form, handleSubmit, isPending } = useGenerationForm();
async function onSubmit(data: GenerationFormValues) {
if (!selectedProfileId) {
toast({
title: 'No profile selected',
description: 'Please select a voice profile from the cards above.',
variant: 'destructive',
});
return;
}
try {
const result = await generation.mutateAsync({
profile_id: selectedProfileId,
text: data.text,
language: data.language,
seed: data.seed,
model_size: data.modelSize,
instruct: data.instruct || undefined,
});
toast({
title: 'Generation complete!',
description: `Audio generated (${result.duration.toFixed(2)}s)`,
});
form.reset();
} catch (error) {
toast({
title: 'Generation failed',
description: error instanceof Error ? error.message : 'Failed to generate audio',
variant: 'destructive',
});
}
async function onSubmit(data: Parameters<typeof handleSubmit>[0]) {
await handleSubmit(data, selectedProfileId);
}
return (
@@ -104,7 +48,7 @@ export function GenerationForm() {
<div className="mt-2 p-3 border rounded-md bg-muted/50 flex items-center gap-2">
<Mic className="h-4 w-4 text-muted-foreground" />
<span className="font-medium">{selectedProfile.name}</span>
<Badge variant="outline">{selectedProfile.language}</Badge>
<span className="text-sm text-muted-foreground">{selectedProfile.language}</span>
</div>
) : (
<div className="mt-2 p-3 border border-dashed rounded-md text-sm text-muted-foreground">
@@ -168,8 +112,11 @@ export function GenerationForm() {
</SelectTrigger>
</FormControl>
<SelectContent>
<SelectItem value="en">English</SelectItem>
<SelectItem value="zh">Chinese</SelectItem>
{LANGUAGE_OPTIONS.map((lang) => (
<SelectItem key={lang.value} value={lang.value}>
{lang.label}
</SelectItem>
))}
</SelectContent>
</Select>
<FormMessage />
@@ -226,9 +173,9 @@ export function GenerationForm() {
<Button
type="submit"
className="w-full"
disabled={generation.isPending || !selectedProfileId}
disabled={isPending || !selectedProfileId}
>
{generation.isPending ? (
{isPending ? (
<>
<Loader2 className="mr-2 h-4 w-4 animate-spin" />
Generating...
-1
View File
@@ -1 +0,0 @@
# Generation history components
+271 -123
View File
@@ -1,30 +1,48 @@
import { Download, MoreHorizontal, Play, Trash2 } from 'lucide-react';
import { useState } from 'react';
import { Badge } from '@/components/ui/badge';
import { AudioWaveform, Download, FileArchive, MoreHorizontal, Play, Trash2 } from 'lucide-react';
import { useEffect, useRef, useState } from 'react';
import { Button } from '@/components/ui/button';
import {
Dialog,
DialogContent,
DialogDescription,
DialogFooter,
DialogHeader,
DialogTitle,
} from '@/components/ui/dialog';
import {
DropdownMenu,
DropdownMenuContent,
DropdownMenuItem,
DropdownMenuTrigger,
} from '@/components/ui/dropdown-menu';
import {
Table,
TableBody,
TableCell,
TableHead,
TableHeader,
TableRow,
} from '@/components/ui/table';
import { Textarea } from '@/components/ui/textarea';
import { useToast } from '@/components/ui/use-toast';
import { apiClient } from '@/lib/api/client';
import { useDeleteGeneration, useHistory } from '@/lib/hooks/useHistory';
import { BOTTOM_SAFE_AREA_PADDING } from '@/lib/constants/ui';
import {
useDeleteGeneration,
useExportGeneration,
useExportGenerationAudio,
useHistory,
useImportGeneration,
} from '@/lib/hooks/useHistory';
import { cn } from '@/lib/utils/cn';
import { formatDate, formatDuration } from '@/lib/utils/format';
import { usePlayerStore } from '@/stores/playerStore';
// OLD TABLE-BASED COMPONENT - REMOVED (can be found in git history)
// This is the new alternate history view with fixed height rows
// NEW ALTERNATE HISTORY VIEW - FIXED HEIGHT ROWS
export function HistoryTable() {
const [page, setPage] = useState(0);
const [page, _setPage] = useState(0);
const [isScrolled, setIsScrolled] = useState(false);
const scrollRef = useRef<HTMLDivElement>(null);
const fileInputRef = useRef<HTMLInputElement>(null);
const [importDialogOpen, setImportDialogOpen] = useState(false);
const [selectedFile, setSelectedFile] = useState<File | null>(null);
const limit = 20;
const { toast } = useToast();
const { data: historyData, isLoading } = useHistory({
limit,
@@ -32,144 +50,274 @@ export function HistoryTable() {
});
const deleteGeneration = useDeleteGeneration();
const setAudio = usePlayerStore((state) => state.setAudio);
const exportGeneration = useExportGeneration();
const exportGenerationAudio = useExportGenerationAudio();
const importGeneration = useImportGeneration();
const setAudioWithAutoPlay = usePlayerStore((state) => state.setAudioWithAutoPlay);
const restartCurrentAudio = usePlayerStore((state) => state.restartCurrentAudio);
const currentAudioId = usePlayerStore((state) => state.audioId);
const isPlaying = usePlayerStore((state) => state.isPlaying);
const audioUrl = usePlayerStore((state) => state.audioUrl);
const isPlayerVisible = !!audioUrl;
const handlePlay = (audioId: string, text: string) => {
const audioUrl = apiClient.getAudioUrl(audioId);
// If clicking the same audio that's playing, it will be handled by the player
setAudio(audioUrl, audioId, text.substring(0, 50));
useEffect(() => {
const scrollEl = scrollRef.current;
if (!scrollEl) return;
const handleScroll = () => {
setIsScrolled(scrollEl.scrollTop > 0);
};
scrollEl.addEventListener('scroll', handleScroll);
return () => scrollEl.removeEventListener('scroll', handleScroll);
}, []);
const handlePlay = (audioId: string, text: string, profileId: string) => {
// If clicking the same audio, restart it from the beginning
if (currentAudioId === audioId) {
restartCurrentAudio();
} else {
// Otherwise, load the new audio and auto-play it
const audioUrl = apiClient.getAudioUrl(audioId);
setAudioWithAutoPlay(audioUrl, audioId, profileId, text.substring(0, 50));
}
};
const handleDownload = (audioId: string, text: string) => {
const audioUrl = apiClient.getAudioUrl(audioId);
const filename = `${text.substring(0, 30).replace(/[^a-z0-9]/gi, '_')}.wav`;
const link = document.createElement('a');
link.href = audioUrl;
link.download = filename;
document.body.appendChild(link);
link.click();
document.body.removeChild(link);
const handleDownloadAudio = (generationId: string, text: string) => {
exportGenerationAudio.mutate(
{ generationId, text },
{
onError: (error) => {
toast({
title: 'Failed to download audio',
description: error.message,
variant: 'destructive',
});
},
},
);
};
const handleExportPackage = (generationId: string, text: string) => {
exportGeneration.mutate(
{ generationId, text },
{
onError: (error) => {
toast({
title: 'Failed to export generation',
description: error.message,
variant: 'destructive',
});
},
},
);
};
const _handleImportClick = () => {
file_handleImportClickk.click();
};
const _handleFileChange = (_e: React.ChangeEvent<HTMLInputElement>) => {
cons_handleFileChangeet.files?.[0];
if (file) {
// Validate file extension
if (!file.name.endsWith('.voicebox.zip')) {
toast({
title: 'Invalid file type',
description: 'Please select a valid .voicebox.zip file',
variant: 'destructive',
});
return;
}
setSelectedFile(file);
setImportDialogOpen(true);
}
};
const handleImportConfirm = () => {
if (selectedFile) {
importGeneration.mutate(selectedFile, {
onSuccess: (data) => {
setImportDialogOpen(false);
setSelectedFile(null);
if (fileInputRef.current) {
fileInputRef.current.value = '';
}
toast({
title: 'Generation imported',
description: data.message || 'Generation imported successfully',
});
},
onError: (error) => {
toast({
title: 'Failed to import generation',
description: error.message,
variant: 'destructive',
});
},
});
}
};
if (isLoading) {
return (
<div className="flex items-center justify-center p-8">
<div className="text-muted-foreground">Loading history...</div>
</div>
);
return null;
}
const history = historyData?.items || [];
const total = historyData?.total || 0;
const hasMore = history.length === limit && (page + 1) * limit < total;
const _hasMore = history.length === limit && (page + 1) * limit < total;
return (
<div className="flex flex-col h-full min-h-0">
<div className="flex flex-col h-full min-h-0 relative">
{history.length === 0 ? (
<div className="text-center py-12 text-muted-foreground flex-1 flex items-center justify-center">
No generation history yet. Generate your first audio to see it here.
<div className="text-center py-12 px-5 border-2 border-dashed mb-5 border-muted rounded-md text-muted-foreground flex-1 flex items-center justify-center">
No voice generations, yet...
</div>
) : (
<>
{isScrolled && (
<div className="absolute top-0 left-0 right-0 h-16 bg-gradient-to-b from-background to-transparent z-10 pointer-events-none" />
)}
<div
ref={scrollRef}
className={cn(
'flex-1 min-h-0 overflow-y-auto border rounded-md overflow-x-hidden',
isPlayerVisible && 'max-h-[calc(100vh-220px)]',
'flex-1 min-h-0 overflow-y-auto space-y-2 pb-4',
isPlayerVisible && BOTTOM_SAFE_AREA_PADDING,
)}
>
<Table className="w-full table-fixed">
<TableHeader className="sticky top-0 bg-background z-10">
<TableRow>
<TableHead className="w-[38%]">Input</TableHead>
<TableHead className="w-[13%]">Voice</TableHead>
<TableHead className="w-[9%]">Lang</TableHead>
<TableHead className="w-[9%]">Length</TableHead>
<TableHead className="w-[13%]">Date</TableHead>
<TableHead className="w-[8%] text-right"></TableHead>
</TableRow>
</TableHeader>
<TableBody>
{history.map((gen) => {
const isCurrentlyPlaying = currentAudioId === gen.id && isPlaying;
return (
<TableRow
key={gen.id}
className={cn(isCurrentlyPlaying && 'bg-muted/50', 'cursor-pointer')}
onClick={() => handlePlay(gen.id, gen.text)}
>
<TableCell className="truncate">{gen.text}</TableCell>
<TableCell className="truncate">{gen.profile_name}</TableCell>
<TableCell>
<Badge variant="outline" className="text-xs text-muted-foreground">
{gen.language}
</Badge>
</TableCell>
<TableCell className="text-sm">{formatDuration(gen.duration)}</TableCell>
<TableCell className="text-xs text-muted-foreground/60">
{formatDate(gen.created_at)}
</TableCell>
<TableCell className="text-right">
<div className="flex justify-end" onClick={(e) => e.stopPropagation()}>
<DropdownMenu>
<DropdownMenuTrigger asChild>
<Button
variant="ghost"
size="icon"
className="h-7 w-7 rounded-full"
aria-label="Actions"
>
<MoreHorizontal className="h-3.5 w-3.5" />
</Button>
</DropdownMenuTrigger>
<DropdownMenuContent align="end">
<DropdownMenuItem onClick={() => handlePlay(gen.id, gen.text)}>
<Play className="mr-2 h-4 w-4" />
Play
</DropdownMenuItem>
<DropdownMenuItem onClick={() => handleDownload(gen.id, gen.text)}>
<Download className="mr-2 h-4 w-4" />
Download
</DropdownMenuItem>
<DropdownMenuItem
onClick={() => deleteGeneration.mutate(gen.id)}
disabled={deleteGeneration.isPending}
className="text-destructive focus:text-destructive"
>
<Trash2 className="mr-2 h-4 w-4" />
Delete
</DropdownMenuItem>
</DropdownMenuContent>
</DropdownMenu>
</div>
</TableCell>
</TableRow>
);
})}
</TableBody>
</Table>
</div>
{history.map((gen) => {
const isCurrentlyPlaying = currentAudioId === gen.id && isPlaying;
return (
<div
key={gen.id}
className={cn(
'flex items-stretch gap-4 h-26 border rounded-md p-3 bg-card hover:bg-muted/70 transition-colors text-left w-full',
isCurrentlyPlaying && 'bg-muted/70',
)}
onMouseDown={(e) => {
// Don't trigger play if clicking on textarea or if text is selected
const target = e.target as HTMLElement;
if (target.closest('textarea') || window.getSelection()?.toString()) {
return;
}
handlePlay(gen.id, gen.text, gen.profile_id);
}}
>
{/* Waveform icon */}
<div className="flex items-center shrink-0">
<AudioWaveform className="h-5 w-5 text-muted-foreground" />
</div>
<div className="flex justify-between items-center mt-4 shrink-0">
<Button
variant="outline"
onClick={() => setPage((p) => Math.max(0, p - 1))}
disabled={page === 0}
>
Previous
</Button>
<div className="text-sm text-muted-foreground">
Page {page + 1} • {total} total
</div>
<Button variant="outline" onClick={() => setPage((p) => p + 1)} disabled={!hasMore}>
Next
</Button>
{/* Left side - Meta information */}
<div className="flex flex-col gap-1.5 w-48 shrink-0 justify-center">
<div className="font-medium text-sm truncate" title={gen.profile_name}>
{gen.profile_name}
</div>
<div className="flex items-center gap-2">
<span className="text-xs text-muted-foreground">{gen.language}</span>
<span className="text-xs text-muted-foreground">
{formatDuration(gen.duration)}
</span>
</div>
<div className="text-xs text-muted-foreground">
{formatDate(gen.created_at)}
</div>
</div>
{/* Right side - Transcript textarea */}
<div className="flex-1 min-w-0 flex">
<Textarea
value={gen.text}
className="flex-1 resize-none text-sm text-muted-foreground select-text"
/>
</div>
{/* Far right - Ellipsis actions */}
<div
className="w-10 shrink-0 flex justify-end"
onMouseDown={(e) => e.stopPropagation()}
onClick={(e) => e.stopPropagation()}
>
<DropdownMenu>
<DropdownMenuTrigger asChild>
<Button
variant="ghost"
size="icon"
className="h-8 w-8"
aria-label="Actions"
>
<MoreHorizontal className="h-4 w-4" />
</Button>
</DropdownMenuTrigger>
<DropdownMenuContent align="end">
<DropdownMenuItem
onClick={() => handlePlay(gen.id, gen.text, gen.profile_id)}
>
<Play className="mr-2 h-4 w-4" />
Play
</DropdownMenuItem>
<DropdownMenuItem
onClick={() => handleDownloadAudio(gen.id, gen.text)}
disabled={exportGenerationAudio.isPending}
>
<Download className="mr-2 h-4 w-4" />
Export Audio
</DropdownMenuItem>
<DropdownMenuItem
onClick={() => handleExportPackage(gen.id, gen.text)}
disabled={exportGeneration.isPending}
>
<FileArchive className="mr-2 h-4 w-4" />
Export Package
</DropdownMenuItem>
<DropdownMenuItem
onClick={() => deleteGeneration.mutate(gen.id)}
disabled={deleteGeneration.isPending}
className="text-destructive focus:text-destructive"
>
<Trash2 className="mr-2 h-4 w-4" />
Delete
</DropdownMenuItem>
</DropdownMenuContent>
</DropdownMenu>
</div>
</div>
);
})}
</div>
</>
)}
<Dialog open={importDialogOpen} onOpenChange={setImportDialogOpen}>
<DialogContent>
<DialogHeader>
<DialogTitle>Import Generation</DialogTitle>
<DialogDescription>
Import the generation from "{selectedFile?.name}". This will add it to your history.
</DialogDescription>
</DialogHeader>
<DialogFooter>
<Button
variant="outline"
onClick={() => {
setImportDialogOpen(false);
setSelectedFile(null);
if (fileInputRef.current) {
fileInputRef.current.value = '';
}
}}
>
Cancel
</Button>
<Button
onClick={handleImportConfirm}
disabled={importGeneration.isPending || !selectedFile}
>
{importGeneration.isPending ? 'Importing...' : 'Import'}
</Button>
</DialogFooter>
</DialogContent>
</Dialog>
</div>
);
}
@@ -0,0 +1,168 @@
import { Sparkles, Upload } from 'lucide-react';
import { useRef, useState } from 'react';
import { FloatingGenerateBox } from '@/components/Generation/FloatingGenerateBox';
import { HistoryTable } from '@/components/History/HistoryTable';
import { Button } from '@/components/ui/button';
import {
Dialog,
DialogContent,
DialogDescription,
DialogFooter,
DialogHeader,
DialogTitle,
} from '@/components/ui/dialog';
import { useToast } from '@/components/ui/use-toast';
import { ProfileList } from '@/components/VoiceProfiles/ProfileList';
import { BOTTOM_SAFE_AREA_PADDING } from '@/lib/constants/ui';
import { useImportProfile } from '@/lib/hooks/useProfiles';
import { cn } from '@/lib/utils/cn';
import { usePlayerStore } from '@/stores/playerStore';
import { useUIStore } from '@/stores/uiStore';
export function MainEditor() {
const audioUrl = usePlayerStore((state) => state.audioUrl);
const isPlayerVisible = !!audioUrl;
const scrollRef = useRef<HTMLDivElement>(null);
const setDialogOpen = useUIStore((state) => state.setProfileDialogOpen);
const importProfile = useImportProfile();
const fileInputRef = useRef<HTMLInputElement>(null);
const [importDialogOpen, setImportDialogOpen] = useState(false);
const [selectedFile, setSelectedFile] = useState<File | null>(null);
const { toast } = useToast();
const handleImportClick = () => {
fileInputRef.current?.click();
};
const handleFileChange = (e: React.ChangeEvent<HTMLInputElement>) => {
const file = e.target.files?.[0];
if (file) {
if (!file.name.endsWith('.voicebox.zip')) {
toast({
title: 'Invalid file type',
description: 'Please select a valid .voicebox.zip file',
variant: 'destructive',
});
return;
}
setSelectedFile(file);
setImportDialogOpen(true);
}
};
const handleImportConfirm = () => {
if (selectedFile) {
importProfile.mutate(selectedFile, {
onSuccess: () => {
setImportDialogOpen(false);
setSelectedFile(null);
if (fileInputRef.current) {
fileInputRef.current.value = '';
}
toast({
title: 'Profile imported',
description: 'Voice profile imported successfully',
});
},
onError: (error) => {
toast({
title: 'Failed to import profile',
description: error.message,
variant: 'destructive',
});
},
});
}
};
return (
// Main view: Profiles top left, Generator bottom left, History right
<div className="grid grid-cols-1 lg:grid-cols-2 gap-6 h-full min-h-0 overflow-hidden relative">
{/* Left Column */}
<div className="flex flex-col min-h-0 overflow-hidden relative">
{/* Scroll Mask - Always visible, behind content */}
<div className="absolute top-0 left-0 right-0 h-16 bg-gradient-to-b from-background to-transparent z-0 pointer-events-none" />
{/* Fixed Header */}
<div className="absolute top-0 left-0 right-0 z-10">
<div className="flex items-center justify-between mb-4 px-1">
<h2 className="text-2xl font-bold">Voicebox</h2>
<div className="flex gap-2">
<Button variant="outline" onClick={handleImportClick}>
<Upload className="mr-2 h-4 w-4" />
Import Voice
</Button>
<input
ref={fileInputRef}
type="file"
accept=".voicebox.zip"
onChange={handleFileChange}
className="hidden"
/>
<Button onClick={() => setDialogOpen(true)}>
<Sparkles className="mr-2 h-4 w-4" />
Create Voice
</Button>
</div>
</div>
</div>
{/* Scrollable Content */}
<div
ref={scrollRef}
className={cn(
'flex-1 min-h-0 overflow-y-auto pt-14',
isPlayerVisible ? BOTTOM_SAFE_AREA_PADDING : 'pb-4',
)}
>
<div className="flex flex-col gap-6">
<div className="shrink-0 flex flex-col">
<ProfileList />
</div>
</div>
</div>
</div>
{/* Right Column - History */}
<div className="flex flex-col min-h-0 overflow-hidden">
<HistoryTable />
</div>
{/* Floating Generate Box */}
<FloatingGenerateBox isPlayerOpen={!!audioUrl} />
{/* Import Dialog */}
<Dialog open={importDialogOpen} onOpenChange={setImportDialogOpen}>
<DialogContent>
<DialogHeader>
<DialogTitle>Import Profile</DialogTitle>
<DialogDescription>
Import the profile from "{selectedFile?.name}". This will create a new profile with
all samples.
</DialogDescription>
</DialogHeader>
<DialogFooter>
<Button
variant="outline"
onClick={() => {
setImportDialogOpen(false);
setSelectedFile(null);
if (fileInputRef.current) {
fileInputRef.current.value = '';
}
}}
>
Cancel
</Button>
<Button
onClick={handleImportConfirm}
disabled={importProfile.isPending || !selectedFile}
>
{importProfile.isPending ? 'Importing...' : 'Import'}
</Button>
</DialogFooter>
</DialogContent>
</Dialog>
</div>
);
}
@@ -0,0 +1,9 @@
import { ModelManagement } from '@/components/ServerSettings/ModelManagement';
export function ModelsTab() {
return (
<div className="space-y-4 overflow-y-auto flex flex-col">
<ModelManagement />
</div>
);
}
@@ -1 +0,0 @@
# Server settings and connection components
@@ -1,6 +1,6 @@
import { zodResolver } from '@hookform/resolvers/zod';
import { useForm } from 'react-hook-form';
import { useEffect } from 'react';
import { useForm } from 'react-hook-form';
import * as z from 'zod';
import { Button } from '@/components/ui/button';
import { Card, CardContent, CardHeader, CardTitle } from '@/components/ui/card';
@@ -17,6 +17,7 @@ import { Input } from '@/components/ui/input';
import { Checkbox } from '@/components/ui/checkbox';
import { useToast } from '@/components/ui/use-toast';
import { useServerStore } from '@/stores/serverStore';
import { setKeepServerRunning } from '@/lib/tauri';
const connectionSchema = z.object({
serverUrl: z.string().url('Please enter a valid URL'),
@@ -69,7 +70,7 @@ export function ConnectionForm() {
<FormItem>
<FormLabel>Server URL</FormLabel>
<FormControl>
<Input placeholder="http://localhost:8000" {...field} />
<Input placeholder="http://127.0.0.1:17493" {...field} />
</FormControl>
<FormDescription>Enter the URL of your voicebox backend server</FormDescription>
<FormMessage />
@@ -88,6 +89,9 @@ export function ConnectionForm() {
checked={keepServerRunningOnClose}
onCheckedChange={(checked: boolean) => {
setKeepServerRunningOnClose(checked);
setKeepServerRunning(checked).catch((error) => {
console.error('Failed to sync setting to Rust:', error);
});
toast({
title: 'Setting updated',
description: checked
@@ -1,17 +1,29 @@
import { useQuery, useMutation, useQueryClient } from '@tanstack/react-query';
import { useMutation, useQuery, useQueryClient } from '@tanstack/react-query';
import { Download, Loader2, Trash2 } from 'lucide-react';
import { useState } from 'react';
import { apiClient } from '@/lib/api/client';
import { Card, CardContent, CardHeader, CardTitle, CardDescription } from '@/components/ui/card';
import { Button } from '@/components/ui/button';
import {
AlertDialog,
AlertDialogAction,
AlertDialogCancel,
AlertDialogContent,
AlertDialogDescription,
AlertDialogFooter,
AlertDialogHeader,
AlertDialogTitle,
} from '@/components/ui/alert-dialog';
import { Badge } from '@/components/ui/badge';
import { Loader2, Download, CheckCircle2 } from 'lucide-react';
import { ModelProgress } from './ModelProgress';
import { Button } from '@/components/ui/button';
import { Card, CardContent, CardDescription, CardHeader, CardTitle } from '@/components/ui/card';
import { useToast } from '@/components/ui/use-toast';
import { apiClient } from '@/lib/api/client';
import { useModelDownloadToast } from '@/lib/hooks/useModelDownloadToast';
import { ModelProgress } from './ModelProgress';
export function ModelManagement() {
const { toast } = useToast();
const queryClient = useQueryClient();
const [downloadingModel, setDownloadingModel] = useState<string | null>(null);
const [downloadingDisplayName, setDownloadingDisplayName] = useState<string | null>(null);
const { data: modelStatus, isLoading } = useQuery({
queryKey: ['modelStatus'],
@@ -19,34 +31,63 @@ export function ModelManagement() {
refetchInterval: 5000, // Refresh every 5 seconds
});
// Use progress toast hook for the downloading model
useModelDownloadToast({
modelName: downloadingModel || '',
displayName: downloadingDisplayName || '',
enabled: !!downloadingModel && !!downloadingDisplayName,
});
const [deleteDialogOpen, setDeleteDialogOpen] = useState(false);
const [modelToDelete, setModelToDelete] = useState<{
name: string;
displayName: string;
sizeMb?: number;
} | null>(null);
const downloadMutation = useMutation({
mutationFn: (modelName: string) => {
setDownloadingModel(modelName);
// Find display name from model status
const model = modelStatus?.models.find((m) => m.model_name === modelName);
setDownloadingDisplayName(model?.display_name || modelName);
return apiClient.triggerModelDownload(modelName);
},
onSuccess: (_, modelName) => {
toast({
title: 'Download started',
description: `Downloading ${modelName}...`,
});
// Refetch status after a delay to see progress
setTimeout(() => {
queryClient.invalidateQueries({ queryKey: ['modelStatus'] });
}, 1000);
onSuccess: () => {
// Download completed - clear state and refetch status
setDownloadingModel(null);
setDownloadingDisplayName(null);
queryClient.invalidateQueries({ queryKey: ['modelStatus'] });
},
onError: (error: Error) => {
setDownloadingModel(null);
setDownloadingDisplayName(null);
toast({
title: 'Download failed',
description: error.message,
variant: 'destructive',
});
},
onSettled: () => {
// Clear downloading state after a delay to allow progress to show
setTimeout(() => {
setDownloadingModel(null);
}, 2000);
});
const deleteMutation = useMutation({
mutationFn: (modelName: string) => apiClient.deleteModel(modelName),
onSuccess: () => {
toast({
title: 'Model deleted',
description: `${modelToDelete?.displayName || 'Model'} has been deleted successfully.`,
});
setDeleteDialogOpen(false);
setModelToDelete(null);
// Refetch status to update UI
queryClient.invalidateQueries({ queryKey: ['modelStatus'] });
},
onError: (error: Error) => {
toast({
title: 'Delete failed',
description: error.message,
variant: 'destructive',
});
},
});
@@ -84,6 +125,14 @@ export function ModelManagement() {
key={model.model_name}
model={model}
onDownload={() => downloadMutation.mutate(model.model_name)}
onDelete={() => {
setModelToDelete({
name: model.model_name,
displayName: model.display_name,
sizeMb: model.size_mb,
});
setDeleteDialogOpen(true);
}}
isDownloading={downloadingModel === model.model_name}
formatSize={formatSize}
/>
@@ -104,6 +153,14 @@ export function ModelManagement() {
key={model.model_name}
model={model}
onDownload={() => downloadMutation.mutate(model.model_name)}
onDelete={() => {
setModelToDelete({
name: model.model_name,
displayName: model.display_name,
sizeMb: model.size_mb,
});
setDeleteDialogOpen(true);
}}
isDownloading={downloadingModel === model.model_name}
formatSize={formatSize}
/>
@@ -129,6 +186,46 @@ export function ModelManagement() {
</div>
) : null}
</CardContent>
{/* Delete Confirmation Dialog */}
<AlertDialog open={deleteDialogOpen} onOpenChange={setDeleteDialogOpen}>
<AlertDialogContent>
<AlertDialogHeader>
<AlertDialogTitle>Delete Model</AlertDialogTitle>
<AlertDialogDescription>
Are you sure you want to delete <strong>{modelToDelete?.displayName}</strong>?
{modelToDelete?.sizeMb && (
<>
{' '}
This will free up {formatSize(modelToDelete.sizeMb)} of disk space. The model will
need to be re-downloaded if you want to use it again.
</>
)}
</AlertDialogDescription>
</AlertDialogHeader>
<AlertDialogFooter>
<AlertDialogCancel>Cancel</AlertDialogCancel>
<AlertDialogAction
onClick={() => {
if (modelToDelete) {
deleteMutation.mutate(modelToDelete.name);
}
}}
disabled={deleteMutation.isPending}
className="bg-destructive text-destructive-foreground hover:bg-destructive/90"
>
{deleteMutation.isPending ? (
<>
<Loader2 className="h-4 w-4 mr-2 animate-spin" />
Deleting...
</>
) : (
'Delete'
)}
</AlertDialogAction>
</AlertDialogFooter>
</AlertDialogContent>
</AlertDialog>
</Card>
);
}
@@ -142,11 +239,12 @@ interface ModelItemProps {
loaded: boolean;
};
onDownload: () => void;
onDelete: () => void;
isDownloading: boolean;
formatSize: (sizeMb?: number) => string;
}
function ModelItem({ model, onDownload, isDownloading, formatSize }: ModelItemProps) {
function ModelItem({ model, onDownload, onDelete, isDownloading, formatSize }: ModelItemProps) {
return (
<div className="flex items-center justify-between p-3 border rounded-lg">
<div className="flex-1">
@@ -171,9 +269,19 @@ function ModelItem({ model, onDownload, isDownloading, formatSize }: ModelItemPr
</div>
<div className="flex items-center gap-2">
{model.downloaded ? (
<div className="flex items-center gap-1 text-sm text-muted-foreground">
<CheckCircle2 className="h-4 w-4 text-green-500" />
<span>Ready</span>
<div className="flex items-center gap-2">
<div className="flex items-center gap-1 text-sm text-muted-foreground">
<span>Ready</span>
</div>
<Button
size="sm"
onClick={onDelete}
variant="outline"
disabled={model.loaded}
title={model.loaded ? 'Unload model before deleting' : 'Delete model'}
>
<Trash2 className="h-4 w-4" />
</Button>
</div>
) : (
<Button size="sm" onClick={onDownload} disabled={isDownloading} variant="outline">
@@ -1,9 +1,9 @@
import { Loader2, XCircle } from 'lucide-react';
import { useEffect, useState } from 'react';
import { Progress } from '@/components/ui/progress';
import { Card, CardContent, CardHeader, CardTitle } from '@/components/ui/card';
import { useServerStore } from '@/stores/serverStore';
import { Progress } from '@/components/ui/progress';
import type { ModelProgress as ModelProgressType } from '@/lib/api/types';
import { Loader2, CheckCircle2, XCircle } from 'lucide-react';
import { useServerStore } from '@/stores/serverStore';
interface ModelProgressProps {
modelName: string;
@@ -12,11 +12,10 @@ interface ModelProgressProps {
export function ModelProgress({ modelName, displayName }: ModelProgressProps) {
const [progress, setProgress] = useState<ModelProgressType | null>(null);
const [isSubscribed, setIsSubscribed] = useState(false);
const serverUrl = useServerStore((state) => state.serverUrl);
useEffect(() => {
if (!serverUrl || isSubscribed) return;
if (!serverUrl) return;
// Subscribe to progress updates via Server-Sent Events
const eventSource = new EventSource(`${serverUrl}/models/progress/${modelName}`);
@@ -29,7 +28,6 @@ export function ModelProgress({ modelName, displayName }: ModelProgressProps) {
// Close connection if complete or error
if (data.status === 'complete' || data.status === 'error') {
eventSource.close();
setIsSubscribed(false);
}
} catch (error) {
console.error('Error parsing progress event:', error);
@@ -39,16 +37,12 @@ export function ModelProgress({ modelName, displayName }: ModelProgressProps) {
eventSource.onerror = (error) => {
console.error('SSE error:', error);
eventSource.close();
setIsSubscribed(false);
};
setIsSubscribed(true);
return () => {
eventSource.close();
setIsSubscribed(false);
};
}, [serverUrl, modelName, isSubscribed]);
}, [serverUrl, modelName]);
// Don't render if no progress or if complete/error and some time has passed
if (
@@ -63,13 +57,11 @@ export function ModelProgress({ modelName, displayName }: ModelProgressProps) {
const k = 1024;
const sizes = ['B', 'KB', 'MB', 'GB'];
const i = Math.floor(Math.log(bytes) / Math.log(k));
return `${(bytes / Math.pow(k, i)).toFixed(1)} ${sizes[i]}`;
return `${(bytes / k ** i).toFixed(1)} ${sizes[i]}`;
};
const getStatusIcon = () => {
switch (progress.status) {
case 'complete':
return <CheckCircle2 className="h-4 w-4 text-green-500" />;
case 'error':
return <XCircle className="h-4 w-4 text-destructive" />;
case 'downloading':
@@ -1,4 +1,4 @@
import { CheckCircle2, Loader2, XCircle } from 'lucide-react';
import { Loader2, XCircle } from 'lucide-react';
import { Badge } from '@/components/ui/badge';
import { Card, CardContent, CardHeader, CardTitle } from '@/components/ui/card';
import { useServerHealth } from '@/lib/hooks/useServer';
@@ -43,7 +43,6 @@ export function ServerStatus() {
) : health ? (
<div className="space-y-2">
<div className="flex items-center gap-2">
<CheckCircle2 className="h-4 w-4 text-green-500" />
<span className="text-sm">Connected</span>
</div>
<div className="flex flex-wrap gap-2">
@@ -1,14 +1,14 @@
import { useState, useEffect } from 'react';
import { RefreshCw, Download, CheckCircle2, AlertCircle } from 'lucide-react';
import { getVersion } from '@tauri-apps/api/app';
import { AlertCircle, Download, RefreshCw } from 'lucide-react';
import { useEffect, useState } from 'react';
import { Badge } from '@/components/ui/badge';
import { Button } from '@/components/ui/button';
import { Card, CardContent, CardHeader, CardTitle } from '@/components/ui/card';
import { Badge } from '@/components/ui/badge';
import { Progress } from '@/components/ui/progress';
import { useAutoUpdater } from '@/hooks/useAutoUpdater';
import { getVersion } from '@tauri-apps/api/app';
export function UpdateStatus() {
const { status, checkForUpdates, downloadAndInstall } = useAutoUpdater(false);
const { status, checkForUpdates, downloadAndInstall, restartAndInstall } = useAutoUpdater(false);
const [currentVersion, setCurrentVersion] = useState<string>('');
useEffect(() => {
@@ -30,7 +30,7 @@ export function UpdateStatus() {
</div>
<Button
onClick={checkForUpdates}
disabled={status.checking || status.downloading || status.installing}
disabled={status.checking || status.downloading || status.readyToInstall}
variant="outline"
size="sm"
>
@@ -53,7 +53,7 @@ export function UpdateStatus() {
</div>
)}
{status.available && !status.downloading && !status.installing && (
{status.available && !status.downloading && !status.readyToInstall && (
<div className="space-y-3 p-4 border rounded-lg bg-primary/5">
<div className="flex items-center justify-between">
<div>
@@ -64,34 +64,57 @@ export function UpdateStatus() {
</div>
<Button onClick={downloadAndInstall} className="w-full" size="sm">
<Download className="h-4 w-4 mr-2" />
Install Update
Download Update
</Button>
</div>
)}
{status.downloading && (
<div className="space-y-2">
<div className="flex items-center gap-2 text-sm">
<Download className="h-4 w-4" />
Downloading update...
<div className="flex items-center justify-between text-sm">
<div className="flex items-center gap-2">
<Download className="h-4 w-4" />
Downloading update...
</div>
{status.downloadProgress !== undefined && (
<span className="text-muted-foreground">{status.downloadProgress}%</span>
)}
</div>
<Progress />
<Progress value={status.downloadProgress} />
{status.downloadedBytes !== undefined &&
status.totalBytes !== undefined &&
status.totalBytes > 0 && (
<div className="text-xs text-muted-foreground">
{(status.downloadedBytes / 1024 / 1024).toFixed(1)} MB /{' '}
{(status.totalBytes / 1024 / 1024).toFixed(1)} MB
</div>
)}
</div>
)}
{status.installing && (
<div className="space-y-2">
<div className="flex items-center gap-2 text-sm">
<RefreshCw className="h-4 w-4 animate-spin" />
Installing update...
{status.readyToInstall && (
<div className="space-y-3 p-4 border rounded-lg bg-accent/30 border-accent/50">
<div className="flex items-center gap-2">
<div>
<div className="font-semibold">Update Ready to Install</div>
<div className="text-sm text-muted-foreground">
Version {status.version} has been downloaded
</div>
</div>
</div>
<div className="text-xs text-muted-foreground">App will restart automatically</div>
<div className="text-sm text-muted-foreground">
The app needs to restart to complete the installation. You can do this now or later at
your convenience.
</div>
<Button onClick={restartAndInstall} className="w-full" size="sm">
<RefreshCw className="h-4 w-4 mr-2" />
Restart Now
</Button>
</div>
)}
{!status.available && !status.checking && !status.error && status.checking === false && (
<div className="flex items-center gap-2 text-sm text-muted-foreground">
<CheckCircle2 className="h-4 w-4 text-green-500" />
You're up to date
</div>
)}
@@ -0,0 +1,27 @@
import { ConnectionForm } from '@/components/ServerSettings/ConnectionForm';
import { ServerStatus } from '@/components/ServerSettings/ServerStatus';
import { UpdateStatus } from '@/components/ServerSettings/UpdateStatus';
import { isTauri } from '@/lib/tauri';
export function ServerTab() {
return (
<div className="space-y-4 overflow-y-auto flex flex-col">
<div className="grid gap-4 md:grid-cols-2">
<ConnectionForm />
<ServerStatus />
</div>
{isTauri() && <UpdateStatus />}
<div className="py-8 text-center text-sm text-muted-foreground">
Created by{' '}
<a
href="https://github.com/jamiepine"
target="_blank"
rel="noopener noreferrer"
className="text-accent hover:underline"
>
Jamie Pine
</a>
</div>
</div>
);
}
+134
View File
@@ -0,0 +1,134 @@
import { motion, useAnimationFrame, useMotionValue, useTransform } from 'motion/react';
import type React from 'react';
import { useCallback, useEffect, useRef, useState } from 'react';
interface ShinyTextProps {
text: string;
disabled?: boolean;
speed?: number;
className?: string;
color?: string;
shineColor?: string;
spread?: number;
yoyo?: boolean;
pauseOnHover?: boolean;
direction?: 'left' | 'right';
delay?: number;
}
const ShinyText: React.FC<ShinyTextProps> = ({
text,
disabled = false,
speed = 2,
className = '',
color = '#b5b5b5',
shineColor = '#ffffff',
spread = 120,
yoyo = false,
pauseOnHover = false,
direction = 'left',
delay = 0,
}) => {
const [isPaused, setIsPaused] = useState(false);
const progress = useMotionValue(0);
const elapsedRef = useRef(0);
const lastTimeRef = useRef<number | null>(null);
const directionRef = useRef(direction === 'left' ? 1 : -1);
const animationDuration = speed * 1000;
const delayDuration = delay * 1000;
useAnimationFrame((time) => {
if (disabled || isPaused) {
lastTimeRef.current = null;
return;
}
if (lastTimeRef.current === null) {
lastTimeRef.current = time;
return;
}
const deltaTime = time - lastTimeRef.current;
lastTimeRef.current = time;
elapsedRef.current += deltaTime;
// Animation goes from 0 to 100
if (yoyo) {
const cycleDuration = animationDuration + delayDuration;
const fullCycle = cycleDuration * 2;
const cycleTime = elapsedRef.current % fullCycle;
if (cycleTime < animationDuration) {
// Forward animation: 0 -> 100
const p = (cycleTime / animationDuration) * 100;
progress.set(directionRef.current === 1 ? p : 100 - p);
} else if (cycleTime < cycleDuration) {
// Delay at end
progress.set(directionRef.current === 1 ? 100 : 0);
} else if (cycleTime < cycleDuration + animationDuration) {
// Reverse animation: 100 -> 0
const reverseTime = cycleTime - cycleDuration;
const p = 100 - (reverseTime / animationDuration) * 100;
progress.set(directionRef.current === 1 ? p : 100 - p);
} else {
// Delay at start
progress.set(directionRef.current === 1 ? 0 : 100);
}
} else {
const cycleDuration = animationDuration + delayDuration;
const cycleTime = elapsedRef.current % cycleDuration;
if (cycleTime < animationDuration) {
// Animation phase: 0 -> 100
const p = (cycleTime / animationDuration) * 100;
progress.set(directionRef.current === 1 ? p : 100 - p);
} else {
// Delay phase - hold at end (shine off-screen)
progress.set(directionRef.current === 1 ? 100 : 0);
}
}
});
useEffect(() => {
directionRef.current = direction === 'left' ? 1 : -1;
elapsedRef.current = 0;
progress.set(0);
// eslint-d, progress.setisable-next-line react-hooks/exhaustive-deps
}, [direction]);
// Transform: p=0 -> 150% (shine off right), p=100 -> -50% (shine off left)
const backgroundPosition = useTransform(progress, (p) => `${150 - p * 2}% center`);
const handleMouseEnter = useCallback(() => {
if (pauseOnHover) setIsPaused(true);
}, [pauseOnHover]);
const handleMouseLeave = useCallback(() => {
if (pauseOnHover) setIsPaused(false);
}, [pauseOnHover]);
const gradientStyle: React.CSSProperties = {
backgroundImage: `linear-gradient(${spread}deg, ${color} 0%, ${color} 35%, ${shineColor} 50%, ${color} 65%, ${color} 100%)`,
backgroundSize: '200% auto',
WebkitBackgroundClip: 'text',
backgroundClip: 'text',
WebkitTextFillColor: 'transparent',
};
return (
<motion.span
className={`inline-block ${className}`}
style={{ ...gradientStyle, backgroundPosition }}
onMouseEnter={handleMouseEnter}
onMouseLeave={handleMouseLeave}
>
{text}
</motion.span>
);
};
export default ShinyText;
// plugins: [],
// };
+50 -20
View File
@@ -1,53 +1,83 @@
import { Home, Settings } from 'lucide-react';
import { cn } from '@/lib/utils/cn';
import { Link, useMatchRoute } from '@tanstack/react-router';
import { Box, BookOpen, Loader2, Mic, Server, Speaker, Volume2 } from 'lucide-react';
import voiceboxLogo from '@/assets/voicebox-logo.png';
import { cn } from '@/lib/utils/cn';
import { useGenerationStore } from '@/stores/generationStore';
import { usePlayerStore } from '@/stores/playerStore';
interface SidebarProps {
activeTab: string;
onTabChange: (tab: string) => void;
isMacOS?: boolean;
}
const tabs = [
{ id: 'main', icon: Home, label: 'Main' },
{ id: 'settings', icon: Settings, label: 'Settings' },
{ id: 'main', path: '/', icon: Volume2, label: 'Generate' },
{ id: 'stories', path: '/stories', icon: BookOpen, label: 'Stories' },
{ id: 'voices', path: '/voices', icon: Mic, label: 'Voices' },
{ id: 'audio', path: '/audio', icon: Speaker, label: 'Audio' },
{ id: 'models', path: '/models', icon: Box, label: 'Models' },
{ id: 'server', path: '/server', icon: Server, label: 'Server' },
];
export function Sidebar({ activeTab, onTabChange }: SidebarProps) {
export function Sidebar({ isMacOS }: SidebarProps) {
const isGenerating = useGenerationStore((state) => state.isGenerating);
const audioUrl = usePlayerStore((state) => state.audioUrl);
const isPlayerVisible = !!audioUrl;
const matchRoute = useMatchRoute();
return (
<div className="fixed left-0 top-0 h-full w-20 bg-sidebar border-r border-border flex flex-col items-center py-6 gap-6">
<div
className={cn(
'fixed left-0 top-0 h-full w-20 bg-sidebar border-r border-border flex flex-col items-center py-6 gap-6',
isMacOS && 'pt-14',
)}
>
{/* Logo */}
<div className="mb-2">
<img
src={voiceboxLogo}
alt="Voicebox"
className="w-12 h-12 object-contain"
/>
<img src={voiceboxLogo} alt="Voicebox" className="w-12 h-12 object-contain" />
</div>
{/* Navigation Buttons */}
<div className="flex flex-col gap-3">
{tabs.map((tab) => {
const Icon = tab.icon;
const isActive = activeTab === tab.id;
// For index route, use exact match; for others, use default matching
const isActive =
tab.path === '/'
? matchRoute({ to: '/', exact: true })
: matchRoute({ to: tab.path });
return (
<button
<Link
key={tab.id}
type="button"
onClick={() => onTabChange(tab.id)}
to={tab.path}
className={cn(
'w-12 h-12 rounded-full flex items-center justify-center transition-all duration-200',
'hover:bg-accent hover:text-accent-foreground',
isActive ? 'bg-accent text-accent-foreground shadow-lg' : 'text-muted-foreground',
'hover:bg-muted/50',
isActive ? 'bg-muted/50 text-foreground shadow-lg' : 'text-muted-foreground',
)}
title={tab.label}
aria-label={tab.label}
>
<Icon className="h-5 w-5" />
</button>
</Link>
);
})}
</div>
{/* Spacer to push loader to bottom */}
<div className="flex-1" />
{/* Generation Loader */}
{isGenerating && (
<div
className={cn(
'w-full flex items-center justify-center transition-all duration-200',
isPlayerVisible ? 'mb-[120px]' : 'mb-0',
)}
>
<Loader2 className="h-6 w-6 text-accent animate-spin" />
</div>
)}
</div>
);
}
@@ -0,0 +1,25 @@
import { FloatingGenerateBox } from '@/components/Generation/FloatingGenerateBox';
import { StoryContent } from './StoryContent';
import { StoryList } from './StoryList';
export function StoriesTab() {
return (
<div className="flex flex-col h-full min-h-0 overflow-hidden">
{/* Main content area */}
<div className="flex-1 min-h-0 flex gap-6 overflow-hidden relative">
{/* Left Column - Story List */}
<div className="flex flex-col min-h-0 overflow-hidden w-full max-w-[360px] shrink-0">
<StoryList />
</div>
{/* Right Column - Story Content */}
<div className="flex flex-col min-h-0 overflow-hidden flex-1">
<StoryContent />
</div>
{/* Floating Generate Box - position is managed via storyStore.trackEditorHeight */}
<FloatingGenerateBox showVoiceSelector />
</div>
</div>
);
}
@@ -0,0 +1,166 @@
import { useSortable } from '@dnd-kit/sortable';
import { CSS } from '@dnd-kit/utilities';
import { GripVertical, Mic, MoreHorizontal, Play, Trash2 } from 'lucide-react';
import { useState } from 'react';
import { Button } from '@/components/ui/button';
import {
DropdownMenu,
DropdownMenuContent,
DropdownMenuItem,
DropdownMenuTrigger,
} from '@/components/ui/dropdown-menu';
import { Textarea } from '@/components/ui/textarea';
import type { StoryItemDetail } from '@/lib/api/types';
import { cn } from '@/lib/utils/cn';
import { useStoryStore } from '@/stores/storyStore';
import { useServerStore } from '@/stores/serverStore';
interface StoryChatItemProps {
item: StoryItemDetail;
storyId: string;
index: number;
onRemove: () => void;
currentTimeMs: number;
isPlaying: boolean;
dragHandleProps?: React.HTMLAttributes<HTMLButtonElement>;
isDragging?: boolean;
}
export function StoryChatItem({
item,
onRemove,
currentTimeMs,
isPlaying,
dragHandleProps,
isDragging,
}: StoryChatItemProps) {
const seek = useStoryStore((state) => state.seek);
const serverUrl = useServerStore((state) => state.serverUrl);
const [avatarError, setAvatarError] = useState(false);
const avatarUrl = `${serverUrl}/profiles/${item.profile_id}/avatar`;
// Check if this item is currently playing based on timecode
const itemStartMs = item.start_time_ms;
const itemEndMs = item.start_time_ms + item.duration * 1000;
const isCurrentlyPlaying = isPlaying && currentTimeMs >= itemStartMs && currentTimeMs < itemEndMs;
const handlePlay = () => {
// Seek to the start of this item
seek(itemStartMs);
};
const formatTime = (ms: number): string => {
const totalSeconds = Math.floor(ms / 1000);
const minutes = Math.floor(totalSeconds / 60);
const seconds = totalSeconds % 60;
const milliseconds = Math.floor((ms % 1000) / 100);
return `${minutes}:${seconds.toString().padStart(2, '0')}.${milliseconds}`;
};
return (
<div
className={cn(
'flex items-start gap-3 p-4 rounded-lg border transition-colors',
isCurrentlyPlaying && 'bg-muted/70 border-primary',
!isCurrentlyPlaying && 'hover:bg-muted/50',
isDragging && 'opacity-50 shadow-lg',
)}
>
{/* Drag Handle */}
{dragHandleProps && (
<button
type="button"
className="shrink-0 cursor-grab active:cursor-grabbing touch-none text-muted-foreground hover:text-foreground transition-colors"
{...dragHandleProps}
>
<GripVertical className="h-5 w-5" />
</button>
)}
{/* Voice Avatar */}
<div className="shrink-0">
<div className="h-10 w-10 rounded-full bg-muted flex items-center justify-center overflow-hidden">
{!avatarError ? (
<img
src={avatarUrl}
alt={`${item.profile_name} avatar`}
className={cn(
'h-full w-full object-cover transition-all duration-200',
!isCurrentlyPlaying && 'grayscale'
)}
onError={() => setAvatarError(true)}
/>
) : (
<Mic className="h-5 w-5 text-muted-foreground" />
)}
</div>
</div>
{/* Content */}
<div className="flex-1 min-w-0">
<div className="flex items-center gap-2 mb-2">
<span className="font-medium text-sm">{item.profile_name}</span>
<span className="text-xs text-muted-foreground">{item.language}</span>
<span className="text-xs text-muted-foreground tabular-nums ml-auto">
{formatTime(itemStartMs)}
</span>
</div>
<Textarea
value={item.text}
className="flex-1 resize-none text-sm text-muted-foreground select-text bg-card cursor-text"
readOnly
onDoubleClick={handlePlay}
/>
</div>
{/* Actions */}
<div className="shrink-0">
<DropdownMenu>
<DropdownMenuTrigger asChild>
<Button variant="ghost" size="icon" className="h-8 w-8" aria-label="Actions">
<MoreHorizontal className="h-4 w-4" />
</Button>
</DropdownMenuTrigger>
<DropdownMenuContent align="end">
<DropdownMenuItem onClick={handlePlay}>
<Play className="mr-2 h-4 w-4" />
Play from here
</DropdownMenuItem>
<DropdownMenuItem onClick={onRemove} className="text-destructive focus:text-destructive">
<Trash2 className="mr-2 h-4 w-4" />
Remove from Story
</DropdownMenuItem>
</DropdownMenuContent>
</DropdownMenu>
</div>
</div>
);
}
// Sortable wrapper component
export function SortableStoryChatItem(props: Omit<StoryChatItemProps, 'dragHandleProps' | 'isDragging'>) {
const {
attributes,
listeners,
setNodeRef,
transform,
transition,
isDragging,
} = useSortable({ id: props.item.generation_id });
const style = {
transform: CSS.Transform.toString(transform),
transition,
};
return (
<div ref={setNodeRef} style={style} {...attributes}>
<StoryChatItem
{...props}
dragHandleProps={listeners}
isDragging={isDragging}
/>
</div>
);
}
@@ -0,0 +1,376 @@
import {
closestCenter,
DndContext,
type DragEndEvent,
KeyboardSensor,
PointerSensor,
useSensor,
useSensors,
} from '@dnd-kit/core';
import {
arrayMove,
SortableContext,
sortableKeyboardCoordinates,
verticalListSortingStrategy,
} from '@dnd-kit/sortable';
import { Download, Plus } from 'lucide-react';
import { useEffect, useMemo, useRef, useState } from 'react';
import { Button } from '@/components/ui/button';
import { Input } from '@/components/ui/input';
import { Popover, PopoverContent, PopoverTrigger } from '@/components/ui/popover';
import { useToast } from '@/components/ui/use-toast';
import { useHistory } from '@/lib/hooks/useHistory';
import {
useAddStoryItem,
useExportStoryAudio,
useRemoveStoryItem,
useReorderStoryItems,
useStory,
} from '@/lib/hooks/useStories';
import { useStoryPlayback } from '@/lib/hooks/useStoryPlayback';
import { useStoryStore } from '@/stores/storyStore';
import { SortableStoryChatItem } from './StoryChatItem';
export function StoryContent() {
const selectedStoryId = useStoryStore((state) => state.selectedStoryId);
const { data: story, isLoading } = useStory(selectedStoryId);
const removeItem = useRemoveStoryItem();
const reorderItems = useReorderStoryItems();
const exportAudio = useExportStoryAudio();
const addStoryItem = useAddStoryItem();
const { toast } = useToast();
const scrollRef = useRef<HTMLDivElement>(null);
// Add generation popover state
const [searchQuery, setSearchQuery] = useState('');
const [isAddOpen, setIsAddOpen] = useState(false);
const { data: historyData } = useHistory();
// Filter generations not in story and matching search
const availableGenerations = useMemo(() => {
if (!historyData?.items || !story) return [];
const storyGenerationIds = new Set(story.items.map((i) => i.generation_id));
const query = searchQuery.toLowerCase();
return historyData.items.filter(
(gen) =>
!storyGenerationIds.has(gen.id) &&
(gen.text.toLowerCase().includes(query) ||
gen.profile_name.toLowerCase().includes(query)),
);
}, [historyData, story, searchQuery]);
// Get track editor height from store for dynamic padding
const trackEditorHeight = useStoryStore((state) => state.trackEditorHeight);
// Track editor is shown when story has items
const hasBottomBar = story && story.items.length > 0;
// Calculate dynamic bottom padding: track editor + gap
const bottomPadding = hasBottomBar ? trackEditorHeight + 24 : 0;
// Drag and drop sensors
const sensors = useSensors(
useSensor(PointerSensor, {
activationConstraint: {
distance: 8,
},
}),
useSensor(KeyboardSensor, {
coordinateGetter: sortableKeyboardCoordinates,
}),
);
// Playback state (for auto-scroll and item highlighting)
const isPlaying = useStoryStore((state) => state.isPlaying);
const currentTimeMs = useStoryStore((state) => state.currentTimeMs);
const playbackStoryId = useStoryStore((state) => state.playbackStoryId);
// Refs for auto-scrolling to playing item
const itemRefsMap = useRef<Map<string, HTMLDivElement>>(new Map());
const lastScrolledItemRef = useRef<string | null>(null);
// Use playback hook
useStoryPlayback(story?.items);
// Sort items by start_time_ms
const sortedItems = useMemo(() => {
if (!story?.items) return [];
return [...story.items].sort((a, b) => a.start_time_ms - b.start_time_ms);
}, [story?.items]);
// Find the currently playing item based on timecode
const currentlyPlayingItemId = useMemo(() => {
if (!isPlaying || playbackStoryId !== story?.id || !sortedItems.length) {
return null;
}
const playingItem = sortedItems.find((item) => {
const itemStart = item.start_time_ms;
const itemEnd = item.start_time_ms + item.duration * 1000;
return currentTimeMs >= itemStart && currentTimeMs < itemEnd;
});
return playingItem?.generation_id ?? null;
}, [isPlaying, playbackStoryId, story?.id, sortedItems, currentTimeMs]);
// Auto-scroll to the currently playing item
useEffect(() => {
if (!currentlyPlayingItemId || currentlyPlayingItemId === lastScrolledItemRef.current) {
return;
}
const element = itemRefsMap.current.get(currentlyPlayingItemId);
if (element && scrollRef.current) {
element.scrollIntoView({ behavior: 'smooth', block: 'start' });
lastScrolledItemRef.current = currentlyPlayingItemId;
}
}, [currentlyPlayingItemId]);
// Reset last scrolled item when playback stops
useEffect(() => {
if (!isPlaying) {
lastScrolledItemRef.current = null;
}
}, [isPlaying]);
const handleRemoveItem = (itemId: string) => {
if (!story) return;
removeItem.mutate(
{
storyId: story.id,
itemId,
},
{
onError: (error) => {
toast({
title: 'Failed to remove item',
description: error.message,
variant: 'destructive',
});
},
},
);
};
const handleDragEnd = (event: DragEndEvent) => {
const { active, over } = event;
if (!story || !over || active.id === over.id) return;
const oldIndex = sortedItems.findIndex((item) => item.generation_id === active.id);
const newIndex = sortedItems.findIndex((item) => item.generation_id === over.id);
if (oldIndex === -1 || newIndex === -1) return;
// Calculate the new order
const newOrder = arrayMove(sortedItems, oldIndex, newIndex);
const generationIds = newOrder.map((item) => item.generation_id);
// Send reorder request to backend
reorderItems.mutate(
{
storyId: story.id,
data: { generation_ids: generationIds },
},
{
onError: (error) => {
toast({
title: 'Failed to reorder items',
description: error.message,
variant: 'destructive',
});
},
},
);
};
const handleExportAudio = () => {
if (!story) return;
exportAudio.mutate(
{
storyId: story.id,
storyName: story.name,
},
{
onError: (error) => {
toast({
title: 'Failed to export audio',
description: error.message,
variant: 'destructive',
});
},
},
);
};
const handleAddGeneration = (generationId: string) => {
if (!story) return;
addStoryItem.mutate(
{
storyId: story.id,
data: { generation_id: generationId },
},
{
onSuccess: () => {
setIsAddOpen(false);
setSearchQuery('');
},
onError: (error) => {
toast({
title: 'Failed to add generation',
description: error.message,
variant: 'destructive',
});
},
},
);
};
if (!selectedStoryId) {
return (
<div className="flex items-center justify-center h-full text-muted-foreground">
<div className="text-center">
<p className="text-lg font-medium mb-2">Select a story</p>
<p className="text-sm">Choose a story from the list to view its content</p>
</div>
</div>
);
}
if (isLoading) {
return (
<div className="flex items-center justify-center h-full">
<div className="text-muted-foreground">Loading story...</div>
</div>
);
}
if (!story) {
return (
<div className="flex items-center justify-center h-full text-muted-foreground">
<div className="text-center">
<p className="text-lg font-medium mb-2">Story not found</p>
<p className="text-sm">The selected story could not be loaded</p>
</div>
</div>
);
}
return (
<div className="flex flex-col h-full min-h-0">
{/* Header */}
<div className="flex items-center justify-between mb-4 px-1">
<div>
<h2 className="text-2xl font-bold">{story.name}</h2>
{story.description && (
<p className="text-sm text-muted-foreground mt-1">{story.description}</p>
)}
</div>
<div className="flex gap-2">
<Popover open={isAddOpen} onOpenChange={setIsAddOpen}>
<PopoverTrigger asChild>
<Button variant="outline" size="sm">
<Plus className="mr-2 h-4 w-4" />
Add
</Button>
</PopoverTrigger>
<PopoverContent className="w-80 p-0" align="end">
<div className="p-2 border-b">
<Input
placeholder="Search by name or transcript..."
value={searchQuery}
onChange={(e) => setSearchQuery(e.target.value)}
autoFocus
/>
</div>
<div className="max-h-60 overflow-y-auto">
{availableGenerations.length === 0 ? (
<div className="p-4 text-center text-sm text-muted-foreground">
{searchQuery
? 'No matching generations found'
: 'No available generations'}
</div>
) : (
availableGenerations.map((gen) => (
<button
key={gen.id}
type="button"
className="w-full text-left px-3 py-2 hover:bg-muted transition-colors border-b last:border-b-0"
onClick={() => handleAddGeneration(gen.id)}
>
<div className="font-medium text-sm">{gen.profile_name}</div>
<div className="text-xs text-muted-foreground truncate">
{gen.text.length > 50 ? `${gen.text.substring(0, 50)}...` : gen.text}
</div>
</button>
))
)}
</div>
</PopoverContent>
</Popover>
{story.items.length > 0 && (
<Button
variant="outline"
size="sm"
onClick={handleExportAudio}
disabled={exportAudio.isPending}
>
<Download className="mr-2 h-4 w-4" />
Export Audio
</Button>
)}
</div>
</div>
{/* Content */}
<div
ref={scrollRef}
className="flex-1 min-h-0 overflow-y-auto space-y-3"
style={{ paddingBottom: bottomPadding > 0 ? `${bottomPadding}px` : undefined }}
>
{sortedItems.length === 0 ? (
<div className="text-center py-12 px-5 border-2 border-dashed border-muted rounded-md text-muted-foreground">
<p className="text-sm">No items in this story</p>
<p className="text-xs mt-2">Generate speech using the box below to add items</p>
</div>
) : (
<DndContext
sensors={sensors}
collisionDetection={closestCenter}
onDragEnd={handleDragEnd}
>
<SortableContext
items={sortedItems.map((item) => item.generation_id)}
strategy={verticalListSortingStrategy}
>
<div className="space-y-3">
{sortedItems.map((item, index) => (
<div
key={item.id}
ref={(el) => {
if (el) {
itemRefsMap.current.set(item.generation_id, el);
} else {
itemRefsMap.current.delete(item.generation_id);
}
}}
>
<SortableStoryChatItem
item={item}
storyId={story.id}
index={index}
onRemove={() => handleRemoveItem(item.id)}
currentTimeMs={currentTimeMs}
isPlaying={isPlaying && playbackStoryId === story.id}
/>
</div>
))}
</div>
</SortableContext>
</DndContext>
)}
</div>
</div>
);
}
+369
View File
@@ -0,0 +1,369 @@
import { Plus, BookOpen, MoreHorizontal, Pencil, Trash2 } from 'lucide-react';
import { useState } from 'react';
import {
AlertDialog,
AlertDialogAction,
AlertDialogCancel,
AlertDialogContent,
AlertDialogDescription,
AlertDialogFooter,
AlertDialogHeader,
AlertDialogTitle,
} from '@/components/ui/alert-dialog';
import { Button } from '@/components/ui/button';
import {
Dialog,
DialogContent,
DialogDescription,
DialogFooter,
DialogHeader,
DialogTitle,
} from '@/components/ui/dialog';
import {
DropdownMenu,
DropdownMenuContent,
DropdownMenuItem,
DropdownMenuTrigger,
} from '@/components/ui/dropdown-menu';
import { Input } from '@/components/ui/input';
import { Label } from '@/components/ui/label';
import { Textarea } from '@/components/ui/textarea';
import { useToast } from '@/components/ui/use-toast';
import { useStories, useCreateStory, useUpdateStory, useDeleteStory } from '@/lib/hooks/useStories';
import { cn } from '@/lib/utils/cn';
import { formatDate } from '@/lib/utils/format';
import { useStoryStore } from '@/stores/storyStore';
export function StoryList() {
const { data: stories, isLoading } = useStories();
const selectedStoryId = useStoryStore((state) => state.selectedStoryId);
const setSelectedStoryId = useStoryStore((state) => state.setSelectedStoryId);
const createStory = useCreateStory();
const updateStory = useUpdateStory();
const deleteStory = useDeleteStory();
const [createDialogOpen, setCreateDialogOpen] = useState(false);
const [editDialogOpen, setEditDialogOpen] = useState(false);
const [deleteDialogOpen, setDeleteDialogOpen] = useState(false);
const [editingStory, setEditingStory] = useState<{
id: string;
name: string;
description?: string;
} | null>(null);
const [deletingStoryId, setDeletingStoryId] = useState<string | null>(null);
const [newStoryName, setNewStoryName] = useState('');
const [newStoryDescription, setNewStoryDescription] = useState('');
const { toast } = useToast();
const handleCreateStory = () => {
if (!newStoryName.trim()) {
toast({
title: 'Name required',
description: 'Please enter a story name',
variant: 'destructive',
});
return;
}
createStory.mutate(
{
name: newStoryName.trim(),
description: newStoryDescription.trim() || undefined,
},
{
onSuccess: (story) => {
setSelectedStoryId(story.id);
setCreateDialogOpen(false);
setNewStoryName('');
setNewStoryDescription('');
toast({
title: 'Story created',
description: `"${story.name}" has been created`,
});
},
onError: (error) => {
toast({
title: 'Failed to create story',
description: error.message,
variant: 'destructive',
});
},
},
);
};
const handleEditClick = (story: { id: string; name: string; description?: string }) => {
setEditingStory(story);
setNewStoryName(story.name);
setNewStoryDescription(story.description || '');
setEditDialogOpen(true);
};
const handleUpdateStory = () => {
if (!editingStory || !newStoryName.trim()) {
toast({
title: 'Name required',
description: 'Please enter a story name',
variant: 'destructive',
});
return;
}
updateStory.mutate(
{
storyId: editingStory.id,
data: {
name: newStoryName.trim(),
description: newStoryDescription.trim() || undefined,
},
},
{
onSuccess: () => {
setEditDialogOpen(false);
setEditingStory(null);
setNewStoryName('');
setNewStoryDescription('');
},
onError: (error) => {
toast({
title: 'Failed to update story',
description: error.message,
variant: 'destructive',
});
},
},
);
};
const handleDeleteClick = (storyId: string) => {
setDeletingStoryId(storyId);
setDeleteDialogOpen(true);
};
const handleDeleteConfirm = () => {
if (!deletingStoryId) return;
deleteStory.mutate(deletingStoryId, {
onSuccess: () => {
// Clear selection if deleting the currently selected story
if (selectedStoryId === deletingStoryId) {
setSelectedStoryId(null);
}
setDeleteDialogOpen(false);
setDeletingStoryId(null);
},
onError: (error) => {
toast({
title: 'Failed to delete story',
description: error.message,
variant: 'destructive',
});
},
});
};
if (isLoading) {
return (
<div className="flex items-center justify-center h-full">
<div className="text-muted-foreground">Loading stories...</div>
</div>
);
}
const storyList = stories || [];
return (
<div className="flex flex-col h-full min-h-0">
{/* Header */}
<div className="flex items-center justify-between mb-4 px-1">
<h2 className="text-2xl font-bold">Stories</h2>
<Button onClick={() => setCreateDialogOpen(true)} size="sm">
<Plus className="mr-2 h-4 w-4" />
New Story
</Button>
</div>
{/* Story List */}
<div className="flex-1 min-h-0 overflow-y-auto space-y-2">
{storyList.length === 0 ? (
<div className="text-center py-12 px-5 border-2 border-dashed border-muted rounded-2xl text-muted-foreground">
<BookOpen className="h-12 w-12 mx-auto mb-4 opacity-50" />
<p className="text-sm">No stories yet</p>
<p className="text-xs mt-2">Create your first story to get started</p>
</div>
) : (
storyList.map((story) => (
<div
key={story.id}
className={cn(
'h-24 p-4 border rounded-2xl transition-colors group flex items-center',
selectedStoryId === story.id && 'bg-muted border-primary',
)}
>
<div className="flex items-start justify-between gap-2 w-full min-w-0">
<button
type="button"
className="flex-1 min-w-0 text-left cursor-pointer overflow-hidden"
onClick={() => setSelectedStoryId(story.id)}
>
<h3 className="font-medium truncate">{story.name}</h3>
{story.description && (
<p className="text-sm text-muted-foreground mt-1 truncate">
{story.description}
</p>
)}
<div className="flex items-center gap-3 mt-2 text-xs text-muted-foreground">
<span>
{story.item_count} {story.item_count === 1 ? 'item' : 'items'}
</span>
<span>•</span>
<span>{formatDate(story.updated_at)}</span>
</div>
</button>
<DropdownMenu>
<DropdownMenuTrigger asChild>
<Button
variant="ghost"
size="icon"
className="h-8 w-8 opacity-0 group-hover:opacity-100 transition-opacity"
onClick={(e) => e.stopPropagation()}
>
<MoreHorizontal className="h-4 w-4" />
</Button>
</DropdownMenuTrigger>
<DropdownMenuContent align="end">
<DropdownMenuItem onClick={() => handleEditClick(story)}>
<Pencil className="mr-2 h-4 w-4" />
Edit
</DropdownMenuItem>
<DropdownMenuItem
onClick={() => handleDeleteClick(story.id)}
className="text-destructive focus:text-destructive"
>
<Trash2 className="mr-2 h-4 w-4" />
Delete
</DropdownMenuItem>
</DropdownMenuContent>
</DropdownMenu>
</div>
</div>
))
)}
</div>
{/* Create Story Dialog */}
<Dialog open={createDialogOpen} onOpenChange={setCreateDialogOpen}>
<DialogContent>
<DialogHeader>
<DialogTitle>Create New Story</DialogTitle>
<DialogDescription>
Create a new story to organize your voice generations into conversations.
</DialogDescription>
</DialogHeader>
<div className="space-y-4 py-4">
<div className="space-y-2">
<Label htmlFor="story-name">Name</Label>
<Input
id="story-name"
placeholder="My Story"
value={newStoryName}
onChange={(e) => setNewStoryName(e.target.value)}
onKeyDown={(e) => {
if (e.key === 'Enter') {
handleCreateStory();
}
}}
/>
</div>
<div className="space-y-2">
<Label htmlFor="story-description">Description (optional)</Label>
<Textarea
id="story-description"
placeholder="A conversation between..."
value={newStoryDescription}
onChange={(e) => setNewStoryDescription(e.target.value)}
rows={3}
/>
</div>
</div>
<DialogFooter>
<Button variant="outline" onClick={() => setCreateDialogOpen(false)}>
Cancel
</Button>
<Button onClick={handleCreateStory} disabled={createStory.isPending}>
{createStory.isPending ? 'Creating...' : 'Create'}
</Button>
</DialogFooter>
</DialogContent>
</Dialog>
{/* Edit Story Dialog */}
<Dialog open={editDialogOpen} onOpenChange={setEditDialogOpen}>
<DialogContent>
<DialogHeader>
<DialogTitle>Edit Story</DialogTitle>
<DialogDescription>Update the story name and description.</DialogDescription>
</DialogHeader>
<div className="space-y-4 py-4">
<div className="space-y-2">
<Label htmlFor="edit-story-name">Name</Label>
<Input
id="edit-story-name"
placeholder="My Story"
value={newStoryName}
onChange={(e) => setNewStoryName(e.target.value)}
onKeyDown={(e) => {
if (e.key === 'Enter') {
handleUpdateStory();
}
}}
/>
</div>
<div className="space-y-2">
<Label htmlFor="edit-story-description">Description (optional)</Label>
<Textarea
id="edit-story-description"
placeholder="A conversation between..."
value={newStoryDescription}
onChange={(e) => setNewStoryDescription(e.target.value)}
rows={3}
/>
</div>
</div>
<DialogFooter>
<Button variant="outline" onClick={() => setEditDialogOpen(false)}>
Cancel
</Button>
<Button onClick={handleUpdateStory} disabled={updateStory.isPending}>
{updateStory.isPending ? 'Saving...' : 'Save'}
</Button>
</DialogFooter>
</DialogContent>
</Dialog>
{/* Delete Story Confirmation Dialog */}
<AlertDialog open={deleteDialogOpen} onOpenChange={setDeleteDialogOpen}>
<AlertDialogContent>
<AlertDialogHeader>
<AlertDialogTitle>Are you sure?</AlertDialogTitle>
<AlertDialogDescription>
This will permanently delete the story and all its items. This action cannot be
undone.
</AlertDialogDescription>
</AlertDialogHeader>
<AlertDialogFooter>
<AlertDialogCancel>Cancel</AlertDialogCancel>
<AlertDialogAction asChild>
<Button
onClick={handleDeleteConfirm}
disabled={deleteStory.isPending}
className="bg-destructive text-destructive-foreground hover:bg-destructive/90"
>
{deleteStory.isPending ? 'Deleting...' : 'Delete'}
</Button>
</AlertDialogAction>
</AlertDialogFooter>
</AlertDialogContent>
</AlertDialog>
</div>
);
}
@@ -0,0 +1,988 @@
import {
Copy,
GripHorizontal,
Minus,
Pause,
Play,
Plus,
Scissors,
Square,
Trash2,
} from 'lucide-react';
import { useCallback, useEffect, useMemo, useRef, useState } from 'react';
import WaveSurfer from 'wavesurfer.js';
import { Button } from '@/components/ui/button';
import { useToast } from '@/components/ui/use-toast';
import { apiClient } from '@/lib/api/client';
import type { StoryItemDetail } from '@/lib/api/types';
import {
useDuplicateStoryItem,
useMoveStoryItem,
useRemoveStoryItem,
useSplitStoryItem,
useTrimStoryItem,
} from '@/lib/hooks/useStories';
import { cn } from '@/lib/utils/cn';
import { useStoryStore } from '@/stores/storyStore';
// Clip waveform component with trim support
function ClipWaveform({
generationId,
width,
trimStartMs,
trimEndMs,
duration,
}: {
generationId: string;
width: number;
trimStartMs: number;
trimEndMs: number;
duration: number;
}) {
const waveformRef = useRef<HTMLDivElement>(null);
const wavesurferRef = useRef<WaveSurfer | null>(null);
// Calculate the full waveform width based on the original duration
// The visible portion (width) represents the effective duration after trimming
const effectiveDurationMs = duration * 1000 - trimStartMs - trimEndMs;
const fullWaveformWidth =
effectiveDurationMs > 0 ? (width / effectiveDurationMs) * (duration * 1000) : width;
// Calculate how much to offset the waveform to hide the trimmed start
const offsetX =
effectiveDurationMs > 0 ? (trimStartMs / (duration * 1000)) * fullWaveformWidth : 0;
useEffect(() => {
if (!waveformRef.current || fullWaveformWidth < 20) return;
// Get CSS colors
const root = document.documentElement;
const getCSSVar = (varName: string) => {
const value = getComputedStyle(root).getPropertyValue(varName).trim();
return value ? `hsl(${value})` : '';
};
const waveColor = getCSSVar('--accent-foreground');
const wavesurfer = WaveSurfer.create({
container: waveformRef.current,
waveColor,
progressColor: waveColor,
cursorWidth: 0,
barWidth: 1,
barRadius: 1,
barGap: 1,
height: 28,
normalize: true,
interact: false,
});
wavesurferRef.current = wavesurfer;
const audioUrl = apiClient.getAudioUrl(generationId);
wavesurfer.load(audioUrl).catch(() => {
// Ignore load errors
});
return () => {
wavesurfer.destroy();
wavesurferRef.current = null;
};
}, [generationId, fullWaveformWidth]);
return (
<div className="w-full h-full opacity-60 overflow-hidden">
{/* Inner container that holds the full waveform, offset to show only visible portion */}
<div
ref={waveformRef}
style={{
width: `${fullWaveformWidth}px`,
transform: `translateX(-${offsetX}px)`,
}}
className="h-full"
/>
</div>
);
}
interface StoryTrackEditorProps {
storyId: string;
items: StoryItemDetail[];
}
const TRACK_HEIGHT = 48;
const TIME_RULER_HEIGHT = 24; // h-6 = 1.5rem = 24px
const MIN_PIXELS_PER_SECOND = 10;
const MAX_PIXELS_PER_SECOND = 200;
const DEFAULT_PIXELS_PER_SECOND = 50;
const DEFAULT_TRACKS = [1, 0, -1]; // Default 3 tracks
const MIN_EDITOR_HEIGHT = 120;
const MAX_EDITOR_HEIGHT = 500;
export function StoryTrackEditor({ storyId, items }: StoryTrackEditorProps) {
const [pixelsPerSecond, setPixelsPerSecond] = useState(DEFAULT_PIXELS_PER_SECOND);
const [draggingItem, setDraggingItem] = useState<string | null>(null);
const [dragOffset, setDragOffset] = useState({ x: 0, y: 0 });
const [dragPosition, setDragPosition] = useState({ x: 0, y: 0 });
const [isResizing, setIsResizing] = useState(false);
const [containerWidth, setContainerWidth] = useState(0);
const containerRef = useRef<HTMLDivElement>(null);
const tracksRef = useRef<HTMLDivElement>(null);
const resizeStartY = useRef(0);
const resizeStartHeight = useRef(0);
const moveItem = useMoveStoryItem();
const trimItem = useTrimStoryItem();
const splitItem = useSplitStoryItem();
const duplicateItem = useDuplicateStoryItem();
const removeItem = useRemoveStoryItem();
const { toast } = useToast();
// Selection state
const selectedClipId = useStoryStore((state) => state.selectedClipId);
const setSelectedClipId = useStoryStore((state) => state.setSelectedClipId);
// Trim state
const [trimmingItem, setTrimmingItem] = useState<string | null>(null);
const [trimSide, setTrimSide] = useState<'start' | 'end' | null>(null);
const [trimStartX, setTrimStartX] = useState(0);
const [tempTrimValues, setTempTrimValues] = useState<{
trim_start_ms: number;
trim_end_ms: number;
} | null>(null);
// Track editor height from store (shared with FloatingGenerateBox)
const editorHeight = useStoryStore((state) => state.trackEditorHeight);
const setEditorHeight = useStoryStore((state) => state.setTrackEditorHeight);
// Playback state
const isPlaying = useStoryStore((state) => state.isPlaying);
const currentTimeMs = useStoryStore((state) => state.currentTimeMs);
const playbackStoryId = useStoryStore((state) => state.playbackStoryId);
const play = useStoryStore((state) => state.play);
const pause = useStoryStore((state) => state.pause);
const stop = useStoryStore((state) => state.stop);
const seek = useStoryStore((state) => state.seek);
const setActiveStory = useStoryStore((state) => state.setActiveStory);
const isActiveStory = playbackStoryId === storyId;
const isCurrentlyPlaying = isPlaying && isActiveStory;
// Auto-activate this story when the editor is shown so playhead is visible
useEffect(() => {
if (items.length > 0 && !isActiveStory) {
const totalDuration = Math.max(
...items.map((item) => {
const trimStart = item.trim_start_ms || 0;
const trimEnd = item.trim_end_ms || 0;
const effectiveDuration = item.duration * 1000 - trimStart - trimEnd;
return item.start_time_ms + effectiveDuration;
}),
0,
);
setActiveStory(storyId, items, totalDuration);
}
}, [storyId, items, isActiveStory, setActiveStory]);
// Sort items by start time for play
const sortedItems = useMemo(() => {
return [...items].sort((a, b) => a.start_time_ms - b.start_time_ms);
}, [items]);
const handlePlayPause = () => {
if (isCurrentlyPlaying) {
pause();
} else {
play(storyId, sortedItems);
}
};
const handleStop = () => {
stop();
};
// Calculate unique tracks from items, always showing at least 3 default tracks
const tracks = useMemo(() => {
const trackSet = new Set([...DEFAULT_TRACKS, ...items.map((item) => item.track)]);
return Array.from(trackSet).sort((a, b) => b - a); // Higher tracks on top
}, [items]);
// Track container width for full-width minimum
useEffect(() => {
const container = tracksRef.current;
if (!container) return;
const observer = new ResizeObserver((entries) => {
for (const entry of entries) {
setContainerWidth(entry.contentRect.width);
}
});
observer.observe(container);
// Set initial width
setContainerWidth(container.clientWidth);
return () => observer.disconnect();
}, []);
// Calculate effective duration (accounting for trims)
const getEffectiveDuration = (item: StoryItemDetail) => {
return item.duration * 1000 - (item.trim_start_ms || 0) - (item.trim_end_ms || 0);
};
// Calculate total duration (using effective durations)
const totalDurationMs = useMemo(() => {
if (items.length === 0) return 10000; // Default 10 seconds
return Math.max(...items.map((item) => item.start_time_ms + getEffectiveDuration(item)), 10000);
}, [items, getEffectiveDuration]);
// Calculate timeline width - at least full container width
const contentWidth = (totalDurationMs / 1000) * pixelsPerSecond + 200; // Content width with padding
const timelineWidth = Math.max(contentWidth, containerWidth);
// Generate time markers
const timeMarkers = useMemo(() => {
const markers: number[] = [];
// Determine interval based on zoom level
let intervalMs = 5000; // 5 seconds
if (pixelsPerSecond > 100) intervalMs = 1000;
else if (pixelsPerSecond > 50) intervalMs = 2000;
else if (pixelsPerSecond < 20) intervalMs = 10000;
for (let ms = 0; ms <= totalDurationMs + intervalMs; ms += intervalMs) {
markers.push(ms);
}
return markers;
}, [totalDurationMs, pixelsPerSecond]);
const formatTime = (ms: number): string => {
const totalSeconds = Math.floor(ms / 1000);
const minutes = Math.floor(totalSeconds / 60);
const seconds = totalSeconds % 60;
return `${minutes}:${seconds.toString().padStart(2, '0')}`;
};
const msToPixels = useCallback((ms: number) => (ms / 1000) * pixelsPerSecond, [pixelsPerSecond]);
const pixelsToMs = useCallback((px: number) => (px / pixelsPerSecond) * 1000, [pixelsPerSecond]);
const handleZoomIn = () => {
setPixelsPerSecond((prev) => Math.min(prev * 1.5, MAX_PIXELS_PER_SECOND));
};
const handleZoomOut = () => {
setPixelsPerSecond((prev) => Math.max(prev / 1.5, MIN_PIXELS_PER_SECOND));
};
// Resize handlers
const handleResizeStart = useCallback(
(e: React.MouseEvent) => {
e.preventDefault();
setIsResizing(true);
resizeStartY.current = e.clientY;
resizeStartHeight.current = editorHeight;
},
[editorHeight],
);
const handleResizeMove = useCallback(
(e: MouseEvent) => {
if (!isResizing) return;
const deltaY = resizeStartY.current - e.clientY;
const newHeight = Math.min(
MAX_EDITOR_HEIGHT,
Math.max(MIN_EDITOR_HEIGHT, resizeStartHeight.current + deltaY),
);
setEditorHeight(newHeight);
},
[isResizing, setEditorHeight],
);
const handleResizeEnd = useCallback(() => {
setIsResizing(false);
}, []);
// Add global mouse listeners for resizing
useEffect(() => {
if (isResizing) {
window.addEventListener('mousemove', handleResizeMove);
window.addEventListener('mouseup', handleResizeEnd);
return () => {
window.removeEventListener('mousemove', handleResizeMove);
window.removeEventListener('mouseup', handleResizeEnd);
};
}
}, [isResizing, handleResizeMove, handleResizeEnd]);
const handleTimelineClick = (e: React.MouseEvent<HTMLDivElement>) => {
if (!tracksRef.current || draggingItem || trimmingItem) return;
const rect = tracksRef.current.getBoundingClientRect();
const x = e.clientX - rect.left + tracksRef.current.scrollLeft;
const timeMs = Math.max(0, pixelsToMs(x));
seek(timeMs);
// Deselect clip when clicking on timeline
setSelectedClipId(null);
};
const handleClipClick = (e: React.MouseEvent, item: StoryItemDetail) => {
e.stopPropagation();
if (draggingItem || trimmingItem) return;
setSelectedClipId(item.id);
};
const handleTrimStart = (e: React.MouseEvent, item: StoryItemDetail, side: 'start' | 'end') => {
e.stopPropagation();
if (!tracksRef.current) return;
setTrimmingItem(item.id);
setTrimSide(side);
setSelectedClipId(item.id);
setTrimStartX(e.clientX);
trimStartItemRef.current = {
item,
initialTrimStart: item.trim_start_ms || 0,
initialTrimEnd: item.trim_end_ms || 0,
};
};
const trimStartItemRef = useRef<{
item: StoryItemDetail;
initialTrimStart: number;
initialTrimEnd: number;
} | null>(null);
const handleTrimMove = useCallback(
(e: MouseEvent) => {
if (!trimmingItem || !trimSide || !trimStartItemRef.current) return;
const deltaX = e.clientX - trimStartX;
const deltaMs = pixelsToMs(deltaX); // Signed delta in milliseconds
const { item, initialTrimStart, initialTrimEnd } = trimStartItemRef.current;
const originalDurationMs = item.duration * 1000;
let newTrimStart = initialTrimStart;
let newTrimEnd = initialTrimEnd;
if (trimSide === 'start') {
// Moving right increases trim_start (trims more from start)
// Moving left decreases trim_start (restores from start)
newTrimStart = Math.round(
Math.max(
0,
Math.min(initialTrimStart + deltaMs, originalDurationMs - initialTrimEnd - 100),
),
);
} else {
// Moving right decreases trim_end (restores from end)
// Moving left increases trim_end (trims more from end)
newTrimEnd = Math.round(
Math.max(
0,
Math.min(initialTrimEnd - deltaMs, originalDurationMs - initialTrimStart - 100),
),
);
}
// Validate that we don't exceed duration
if (newTrimStart + newTrimEnd >= originalDurationMs - 100) {
return; // Don't allow trimming to less than 100ms
}
// Update temporary trim values for visual feedback
setTempTrimValues({
trim_start_ms: newTrimStart,
trim_end_ms: newTrimEnd,
});
},
[trimmingItem, trimSide, trimStartX, pixelsToMs],
);
const handleTrimEnd = useCallback(() => {
if (!trimmingItem || !trimSide || !trimStartItemRef.current) {
setTrimmingItem(null);
setTrimSide(null);
setTempTrimValues(null);
trimStartItemRef.current = null;
return;
}
const { initialTrimStart, initialTrimEnd } = trimStartItemRef.current;
// Use temporary trim values if available, otherwise use initial values
// Ensure values are integers for the backend
const finalTrimStart = Math.round(tempTrimValues?.trim_start_ms ?? initialTrimStart);
const finalTrimEnd = Math.round(tempTrimValues?.trim_end_ms ?? initialTrimEnd);
// Only update if values changed
if (finalTrimStart !== initialTrimStart || finalTrimEnd !== initialTrimEnd) {
trimItem.mutate(
{
storyId,
itemId: trimmingItem,
data: {
trim_start_ms: finalTrimStart,
trim_end_ms: finalTrimEnd,
},
},
{
onError: (error) => {
toast({
title: 'Failed to trim clip',
description: error instanceof Error ? error.message : String(error),
variant: 'destructive',
});
},
},
);
}
setTrimmingItem(null);
setTrimSide(null);
setTempTrimValues(null);
trimStartItemRef.current = null;
}, [trimmingItem, trimSide, tempTrimValues, storyId, trimItem, toast]);
const handleSplit = useCallback(() => {
if (!selectedClipId) return;
const item = items.find((i) => i.id === selectedClipId);
if (!item) return;
const splitTimeMs = currentTimeMs - item.start_time_ms;
const effectiveDuration = getEffectiveDuration(item);
if (splitTimeMs <= 0 || splitTimeMs >= effectiveDuration) {
toast({
title: 'Invalid split point',
description: 'Playhead must be within the selected clip',
variant: 'destructive',
});
return;
}
splitItem.mutate(
{
storyId,
itemId: selectedClipId,
data: { split_time_ms: splitTimeMs },
},
{
onSuccess: () => {
setSelectedClipId(null);
},
onError: (error) => {
toast({
title: 'Failed to split clip',
description: error instanceof Error ? error.message : String(error),
variant: 'destructive',
});
},
},
);
}, [
selectedClipId,
items,
currentTimeMs,
getEffectiveDuration,
storyId,
splitItem,
toast,
setSelectedClipId,
]);
const handleDuplicate = useCallback(() => {
if (!selectedClipId) return;
duplicateItem.mutate(
{
storyId,
itemId: selectedClipId,
},
{
onError: (error) => {
toast({
title: 'Failed to duplicate clip',
description: error instanceof Error ? error.message : String(error),
variant: 'destructive',
});
},
},
);
}, [selectedClipId, storyId, duplicateItem, toast]);
const handleDelete = useCallback(() => {
if (!selectedClipId) return;
removeItem.mutate(
{
storyId,
itemId: selectedClipId,
},
{
onSuccess: () => {
setSelectedClipId(null);
},
onError: (error) => {
toast({
title: 'Failed to delete clip',
description: error instanceof Error ? error.message : String(error),
variant: 'destructive',
});
},
},
);
}, [selectedClipId, storyId, removeItem, toast, setSelectedClipId]);
// Keyboard shortcuts
useEffect(() => {
const handleKeyDown = (e: KeyboardEvent) => {
// Only handle shortcuts when editor is focused or no input is focused
if (e.target instanceof HTMLInputElement || e.target instanceof HTMLTextAreaElement) {
return;
}
if (e.key === ' ') {
e.preventDefault();
handlePlayPause();
} else if (e.key === 'Escape') {
setSelectedClipId(null);
} else if (e.key === 's' || e.key === 'S') {
if (selectedClipId) {
e.preventDefault();
handleSplit();
}
} else if (e.key === 'd' || e.key === 'D') {
if (selectedClipId && (e.metaKey || e.ctrlKey)) {
e.preventDefault();
handleDuplicate();
}
} else if (e.key === 'Delete' || e.key === 'Backspace') {
if (selectedClipId) {
e.preventDefault();
handleDelete();
}
}
};
window.addEventListener('keydown', handleKeyDown);
return () => window.removeEventListener('keydown', handleKeyDown);
}, [
selectedClipId,
handleSplit,
handleDuplicate,
handleDelete,
setSelectedClipId,
handlePlayPause,
]);
// Add global mouse listeners for trimming
useEffect(() => {
if (trimmingItem) {
window.addEventListener('mousemove', handleTrimMove);
window.addEventListener('mouseup', handleTrimEnd);
return () => {
window.removeEventListener('mousemove', handleTrimMove);
window.removeEventListener('mouseup', handleTrimEnd);
};
}
}, [trimmingItem, handleTrimMove, handleTrimEnd]);
const handleDragStart = (e: React.MouseEvent, item: StoryItemDetail) => {
e.stopPropagation();
if (!tracksRef.current) return;
const rect = e.currentTarget.getBoundingClientRect();
setDragOffset({
x: e.clientX - rect.left,
y: e.clientY - rect.top,
});
setDragPosition({
x: rect.left - tracksRef.current.getBoundingClientRect().left + tracksRef.current.scrollLeft,
// Subtract ruler height since clips are positioned relative to tracks area, not the scrollable container
y: rect.top - tracksRef.current.getBoundingClientRect().top - TIME_RULER_HEIGHT,
});
setDraggingItem(item.id);
};
const handleDragMove = useCallback(
(e: React.MouseEvent) => {
if (!draggingItem || !tracksRef.current) return;
const rect = tracksRef.current.getBoundingClientRect();
const x = e.clientX - rect.left + tracksRef.current.scrollLeft - dragOffset.x;
// Subtract ruler height since clips are positioned relative to tracks area
const y = e.clientY - rect.top - dragOffset.y - TIME_RULER_HEIGHT;
setDragPosition({ x: Math.max(0, x), y });
},
[draggingItem, dragOffset],
);
const handleDragEnd = useCallback(() => {
if (!draggingItem || !tracksRef.current) {
setDraggingItem(null);
return;
}
const item = items.find((i) => i.id === draggingItem);
if (!item) {
setDraggingItem(null);
return;
}
// Calculate new time from x position
const newTimeMs = Math.max(0, Math.round(pixelsToMs(dragPosition.x)));
// Calculate new track from y position
const trackIndex = Math.floor(dragPosition.y / TRACK_HEIGHT);
const clampedTrackIndex = Math.max(0, Math.min(trackIndex, tracks.length - 1));
const newTrack = tracks[clampedTrackIndex] ?? 0;
// Check if position changed
if (newTimeMs !== item.start_time_ms || newTrack !== item.track) {
moveItem.mutate(
{
storyId,
itemId: item.id,
data: {
start_time_ms: newTimeMs,
track: newTrack,
},
},
{
onError: (error) => {
toast({
title: 'Failed to move item',
description: error instanceof Error ? error.message : String(error),
variant: 'destructive',
});
},
},
);
}
setDraggingItem(null);
}, [draggingItem, dragPosition, items, tracks, pixelsToMs, storyId, moveItem, toast]);
// Get track index for rendering
const getTrackIndex = (trackNumber: number) => tracks.indexOf(trackNumber);
// Calculate clip position and dimensions
const getClipStyle = (item: StoryItemDetail) => {
const isDragging = draggingItem === item.id;
const trackIndex = getTrackIndex(item.track);
const effectiveDuration = getEffectiveDuration(item);
const width = msToPixels(effectiveDuration);
const left = isDragging ? dragPosition.x : msToPixels(item.start_time_ms);
const top = isDragging ? dragPosition.y : trackIndex * TRACK_HEIGHT;
return {
width: `${width}px`,
left: `${left}px`,
top: `${top}px`,
height: `${TRACK_HEIGHT - 4}px`,
};
};
// Playhead position
const playheadLeft = msToPixels(currentTimeMs);
// Auto-scroll timeline to follow playhead during playback
useEffect(() => {
if (!isCurrentlyPlaying || !tracksRef.current) return;
const container = tracksRef.current;
const containerWidth = container.clientWidth;
const scrollLeft = container.scrollLeft;
const halfwayPoint = scrollLeft + containerWidth / 2;
// If playhead is past the halfway point, scroll to keep it centered
if (playheadLeft > halfwayPoint) {
const targetScroll = playheadLeft - containerWidth / 2;
container.scrollLeft = targetScroll;
}
}, [isCurrentlyPlaying, playheadLeft]);
// Calculate tracks area height
const tracksAreaHeight = tracks.length * TRACK_HEIGHT;
const timelineContainerHeight = editorHeight - 40; // Subtract toolbar height
if (items.length === 0) {
return null;
}
return (
<div className="fixed bottom-0 left-0 right-0 border-t bg-background/95 backdrop-blur supports-backdrop-filter:bg-background/60 z-50">
<div
className="border-t bg-background/30 backdrop-blur-2xl overflow-hidden relative"
ref={containerRef}
>
{/* Resize handle at top */}
<button
type="button"
className="absolute top-0 left-0 right-0 h-2 cursor-ns-resize flex items-center justify-center hover:bg-muted/50 transition-colors z-20 group"
onMouseDown={handleResizeStart}
aria-label="Resize track editor"
>
<GripHorizontal className="h-3 w-3 text-muted-foreground/50 group-hover:text-muted-foreground" />
</button>
{/* Toolbar */}
<div className="flex items-center justify-between px-3 py-2 border-b bg-muted/30 mt-2">
{/* Play controls - left side */}
<div className="flex items-center gap-2">
<Button
variant="ghost"
size="icon"
className="h-7 w-7"
onClick={handlePlayPause}
title="Play/Pause (Space)"
>
{isCurrentlyPlaying ? <Pause className="h-4 w-4" /> : <Play className="h-4 w-4" />}
</Button>
<Button
variant="ghost"
size="icon"
className="h-7 w-7"
onClick={handleStop}
disabled={!isCurrentlyPlaying}
>
<Square className="h-3 w-3" />
</Button>
<span className="text-xs text-muted-foreground tabular-nums ml-2">
{formatTime(currentTimeMs)} / {formatTime(totalDurationMs)}
</span>
</div>
{/* Clip editing controls - center */}
{selectedClipId && (
<div className="flex items-center gap-1">
<Button
variant="ghost"
size="icon"
className="h-7 w-7"
onClick={handleSplit}
title="Split at playhead (S)"
>
<Scissors className="h-4 w-4" />
</Button>
<Button
variant="ghost"
size="icon"
className="h-7 w-7"
onClick={handleDuplicate}
title="Duplicate (Cmd/Ctrl+D)"
>
<Copy className="h-4 w-4" />
</Button>
<Button
variant="ghost"
size="icon"
className="h-7 w-7"
onClick={handleDelete}
title="Delete (Delete/Backspace)"
>
<Trash2 className="h-4 w-4" />
</Button>
</div>
)}
{/* Zoom controls - right side */}
<div className="flex items-center gap-2">
<span className="text-xs text-muted-foreground">Zoom:</span>
<Button variant="ghost" size="icon" className="h-6 w-6" onClick={handleZoomOut}>
<Minus className="h-3 w-3" />
</Button>
<Button variant="ghost" size="icon" className="h-6 w-6" onClick={handleZoomIn}>
<Plus className="h-3 w-3" />
</Button>
</div>
</div>
{/* Timeline container with track labels sidebar */}
<div className="flex" style={{ height: `${timelineContainerHeight}px` }}>
{/* Track labels sidebar - fixed width */}
<div className="w-16 shrink-0 border-r bg-muted/20 overflow-hidden">
{/* Spacer for time ruler */}
<div className="h-6 border-b bg-muted/30" />
{/* Track labels */}
<div style={{ height: `${tracksAreaHeight}px` }}>
{tracks.map((trackNumber, index) => (
<div
key={trackNumber}
className={cn(
'border-b flex items-center justify-center',
index % 2 === 0 ? 'bg-background' : 'bg-muted/10',
)}
style={{ height: `${TRACK_HEIGHT}px` }}
>
<span className="text-[10px] text-muted-foreground select-none">
{trackNumber}
</span>
</div>
))}
</div>
</div>
{/* Scrollable timeline area */}
{/* biome-ignore lint/a11y/noStaticElementInteractions: Container handles drag events for child clips */}
<div
ref={tracksRef}
className="overflow-auto relative flex-1"
onMouseMove={draggingItem ? handleDragMove : undefined}
onMouseUp={draggingItem ? handleDragEnd : undefined}
onMouseLeave={draggingItem ? handleDragEnd : undefined}
>
{/* Time ruler - clickable to seek */}
<button
type="button"
className="h-6 border-b bg-muted/20 sticky top-0 z-10 cursor-pointer text-left"
style={{ width: `${timelineWidth}px` }}
onClick={handleTimelineClick}
aria-label="Seek timeline"
>
{timeMarkers.map((ms) => (
<div
key={ms}
className="absolute top-0 h-full flex flex-col justify-end pointer-events-none"
style={{ left: `${msToPixels(ms)}px` }}
>
<div className="h-2 w-px bg-border" />
<span className="text-[10px] text-muted-foreground ml-1 select-none">
{formatTime(ms)}
</span>
</div>
))}
</button>
{/* Tracks area */}
<div
className="relative"
style={{ width: `${timelineWidth}px`, height: `${tracksAreaHeight}px` }}
>
{/* Track backgrounds - pointer-events-none to allow clicks to pass through */}
{tracks.map((trackNumber, index) => (
<div
key={trackNumber}
className={cn(
'absolute left-0 right-0 border-b pointer-events-none',
index % 2 === 0 ? 'bg-background' : 'bg-muted/10',
)}
style={{
top: `${index * TRACK_HEIGHT}px`,
height: `${TRACK_HEIGHT}px`,
}}
/>
))}
{/* Click area for seeking - z-index lower than clips */}
<button
type="button"
className="absolute inset-0 z-0 cursor-pointer"
onClick={handleTimelineClick}
aria-label="Seek timeline"
/>
{/* Audio clips */}
{items.map((item) => {
const isDragging = draggingItem === item.id;
const isSelected = selectedClipId === item.id;
const isTrimming = trimmingItem === item.id;
// Use temporary trim values during trimming for visual feedback
const displayTrimStart =
isTrimming && tempTrimValues
? tempTrimValues.trim_start_ms
: item.trim_start_ms || 0;
const displayTrimEnd =
isTrimming && tempTrimValues ? tempTrimValues.trim_end_ms : item.trim_end_ms || 0;
const effectiveDuration = item.duration * 1000 - displayTrimStart - displayTrimEnd;
const style = getClipStyle({
...item,
trim_start_ms: displayTrimStart,
trim_end_ms: displayTrimEnd,
});
const clipWidth = msToPixels(effectiveDuration);
return (
<div
key={item.id}
className={cn(
'absolute rounded select-none overflow-visible z-10',
isSelected && 'ring-2 ring-primary ring-offset-1',
isTrimming && 'ring-2 ring-accent',
)}
style={style}
>
<button
type="button"
className={cn(
'w-full h-full rounded cursor-move overflow-hidden',
'bg-accent/80 hover:bg-accent border border-accent-foreground/20',
'flex flex-col justify-center',
isDragging && 'opacity-80 shadow-lg z-20',
!isDragging && 'transition-all duration-100',
)}
onClick={(e) => handleClipClick(e, item)}
onMouseDown={(e) => {
// Only start drag if not clicking on trim handles
if (!(e.target as HTMLElement).closest('.trim-handle')) {
handleDragStart(e, item);
}
}}
>
{/* Clip label */}
<div className="absolute top-0 left-1 right-1 z-10">
<p className="text-[9px] font-medium text-accent-foreground truncate">
{item.profile_name}
</p>
</div>
{/* Waveform */}
<div className="absolute inset-0 top-3">
<ClipWaveform
generationId={item.generation_id}
width={clipWidth}
trimStartMs={displayTrimStart}
trimEndMs={displayTrimEnd}
duration={item.duration}
/>
</div>
</button>
{/* Trim handles */}
{isSelected && (
<>
{/* Left trim handle */}
<button
type="button"
className="trim-handle absolute left-0 top-0 bottom-0 w-2 cursor-ew-resize hover:bg-primary/30 bg-primary/20 z-30 rounded-l"
onMouseDown={(e) => handleTrimStart(e, item, 'start')}
aria-label="Trim start"
/>
{/* Right trim handle */}
<button
type="button"
className="trim-handle absolute right-0 top-0 bottom-0 w-2 cursor-ew-resize hover:bg-primary/30 bg-primary/20 z-30 rounded-r"
onMouseDown={(e) => handleTrimStart(e, item, 'end')}
aria-label="Trim end"
/>
</>
)}
</div>
);
})}
{/* Playhead - always visible */}
<div
className="absolute top-0 bottom-0 w-1 bg-accent z-30 pointer-events-none rounded-full"
style={{ left: `${playheadLeft}px` }}
>
<div className="absolute -top-1 left-1/2 -translate-x-1/2 w-3 h-3 bg-accent rounded-full" />
</div>
</div>
</div>
</div>
</div>
</div>
);
}
@@ -0,0 +1,8 @@
export function TitleBarDragRegion() {
return (
<div
data-tauri-drag-region
className="fixed top-0 left-0 right-0 h-12 z-[9999]"
/>
);
}
-69
View File
@@ -1,69 +0,0 @@
import { useAutoUpdater } from '../hooks/useAutoUpdater';
import { Button } from './ui/button';
import { Card } from './ui/card';
import { Progress } from './ui/progress';
export function UpdateNotification() {
const { status, checkForUpdates, downloadAndInstall } = useAutoUpdater(true);
if (status.error) {
return null;
}
if (!status.available && !status.checking) {
return null;
}
if (status.checking) {
return (
<Card className="p-4 mb-4">
<div className="flex items-center gap-3">
<div className="animate-spin h-4 w-4 border-2 border-primary border-t-transparent rounded-full" />
<span className="text-sm">Checking for updates...</span>
</div>
</Card>
);
}
if (status.available) {
return (
<Card className="p-4 mb-4 border-primary">
<div className="space-y-3">
<div>
<h3 className="font-semibold">Update Available</h3>
<p className="text-sm text-muted-foreground">
Version {status.version} is ready to install
</p>
</div>
{status.downloading && (
<div className="space-y-2">
<p className="text-sm">Downloading update...</p>
<Progress />
</div>
)}
{status.installing && (
<div className="space-y-2">
<p className="text-sm">Installing update...</p>
<p className="text-xs text-muted-foreground">App will restart automatically</p>
</div>
)}
{!status.downloading && !status.installing && (
<div className="flex gap-2">
<Button onClick={downloadAndInstall} size="sm">
Install Now
</Button>
<Button onClick={() => window.location.reload()} variant="outline" size="sm">
Later
</Button>
</div>
)}
</div>
</Card>
);
}
return null;
}
@@ -1 +0,0 @@
# Voice profile management components
@@ -0,0 +1,171 @@
import { Mic, Pause, Play, Square } from 'lucide-react';
import { memo, useEffect, useState } from 'react';
import { Visualizer } from 'react-sound-visualizer';
import { Button } from '@/components/ui/button';
import { FormControl, FormItem, FormMessage } from '@/components/ui/form';
import { formatAudioDuration } from '@/lib/utils/audio';
const MemoizedWaveform = memo(function MemoizedWaveform({
audioStream,
}: {
audioStream: MediaStream;
}) {
return (
<div className="absolute inset-0 pointer-events-none flex items-center justify-center opacity-30">
<Visualizer audio={audioStream} autoStart strokeColor="#b39a3d">
{({ canvasRef }) => (
<canvas
ref={canvasRef}
width={500}
height={150}
className="w-full h-full"
/>
)}
</Visualizer>
</div>
);
});
interface AudioSampleRecordingProps {
file: File | null | undefined;
isRecording: boolean;
duration: number;
onStart: () => void;
onStop: () => void;
onCancel: () => void;
onTranscribe: () => void;
onPlayPause: () => void;
isPlaying: boolean;
isTranscribing?: boolean;
showWaveform?: boolean;
}
export function AudioSampleRecording({
file,
isRecording,
duration,
onStart,
onStop,
onCancel,
onTranscribe,
onPlayPause,
isPlaying,
isTranscribing = false,
showWaveform = true,
}: AudioSampleRecordingProps) {
const [audioStream, setAudioStream] = useState<MediaStream | null>(null);
// Request microphone access when component mounts
useEffect(() => {
if (!showWaveform) return;
let stream: MediaStream | null = null;
navigator.mediaDevices
.getUserMedia({ audio: true, video: false })
.then((s) => {
stream = s;
setAudioStream(s);
})
.catch((err) => {
console.warn('Could not access microphone for visualization:', err);
});
return () => {
if (stream) {
stream.getTracks().forEach((track) => {
track.stop();
});
}
};
}, [showWaveform]);
return (
<FormItem>
<FormControl>
<div className="space-y-4">
{!isRecording && !file && (
<div className="relative flex flex-col items-center justify-center gap-4 p-4 border-2 border-dashed rounded-lg min-h-[180px] overflow-hidden">
{showWaveform && audioStream && (
<MemoizedWaveform audioStream={audioStream} />
)}
<Button
type="button"
onClick={onStart}
size="lg"
className="relative z-10 flex items-center gap-2"
>
<Mic className="h-5 w-5" />
Start Recording
</Button>
<p className="relative z-10 text-sm text-muted-foreground text-center">
Click to start recording. Maximum duration: 30 seconds.
</p>
</div>
)}
{isRecording && (
<div className="relative flex flex-col items-center justify-center gap-4 p-4 border-2 border-accent rounded-lg bg-accent/5 min-h-[180px] overflow-hidden">
{showWaveform && audioStream && (
<MemoizedWaveform audioStream={audioStream} />
)}
<div className="relative z-10 flex items-center gap-4">
<div className="flex items-center gap-2">
<div className="h-3 w-3 rounded-full bg-accent animate-pulse" />
<span className="text-lg font-mono font-semibold">
{formatAudioDuration(duration)}
</span>
</div>
</div>
<Button
type="button"
onClick={onStop}
className="relative z-10 flex items-center gap-2 bg-accent text-accent-foreground hover:bg-accent/90"
>
<Square className="h-4 w-4" />
Stop Recording
</Button>
<p className="relative z-10 text-sm text-muted-foreground text-center">
{formatAudioDuration(30 - duration)} remaining
</p>
</div>
)}
{file && !isRecording && (
<div className="flex flex-col items-center justify-center gap-4 p-4 border-2 border-primary rounded-lg bg-primary/5 min-h-[180px]">
<div className="flex items-center gap-2">
<Mic className="h-5 w-5 text-primary" />
<span className="font-medium">Recording complete</span>
</div>
<p className="text-sm text-muted-foreground text-center">File: {file.name}</p>
<div className="flex gap-2">
<Button type="button" size="icon" variant="outline" onClick={onPlayPause}>
{isPlaying ? <Pause className="h-4 w-4" /> : <Play className="h-4 w-4" />}
</Button>
<Button
type="button"
variant="outline"
onClick={onTranscribe}
disabled={isTranscribing}
className="flex items-center gap-2"
>
<Mic className="h-4 w-4" />
{isTranscribing ? 'Transcribing...' : 'Transcribe'}
</Button>
<Button
type="button"
variant="outline"
onClick={onCancel}
className="flex items-center gap-2"
>
Record Again
</Button>
</div>
</div>
)}
</div>
</FormControl>
<FormMessage />
</FormItem>
);
}
@@ -0,0 +1,109 @@
import { Mic, Monitor, Pause, Play, Square } from 'lucide-react';
import { Button } from '@/components/ui/button';
import { FormControl, FormItem, FormMessage } from '@/components/ui/form';
import { formatAudioDuration } from '@/lib/utils/audio';
interface AudioSampleSystemProps {
file: File | null | undefined;
isRecording: boolean;
duration: number;
onStart: () => void;
onStop: () => void;
onCancel: () => void;
onTranscribe: () => void;
onPlayPause: () => void;
isPlaying: boolean;
isTranscribing?: boolean;
}
export function AudioSampleSystem({
file,
isRecording,
duration,
onStart,
onStop,
onCancel,
onTranscribe,
onPlayPause,
isPlaying,
isTranscribing = false,
}: AudioSampleSystemProps) {
return (
<FormItem>
<FormControl>
<div className="space-y-4">
{!isRecording && !file && (
<div className="flex flex-col items-center justify-center gap-4 p-4 border-2 border-dashed rounded-lg min-h-[180px]">
<Button type="button" onClick={onStart} size="lg" className="flex items-center gap-2">
<Monitor className="h-5 w-5" />
Start Capture
</Button>
<p className="text-sm text-muted-foreground text-center">
Capture audio from your system. Maximum duration: 30 seconds.
</p>
</div>
)}
{isRecording && (
<div className="flex flex-col items-center justify-center gap-4 p-4 border-2 border-destructive rounded-lg bg-destructive/5 min-h-[180px]">
<div className="flex items-center gap-4">
<div className="flex items-center gap-2">
<div className="h-3 w-3 rounded-full bg-destructive animate-pulse" />
<span className="text-lg font-mono font-semibold">
{formatAudioDuration(duration)}
</span>
</div>
</div>
<Button
type="button"
onClick={onStop}
variant="destructive"
className="flex items-center gap-2"
>
<Square className="h-4 w-4" />
Stop Capture
</Button>
<p className="text-sm text-muted-foreground text-center">
{formatAudioDuration(30 - duration)} remaining
</p>
</div>
)}
{file && !isRecording && (
<div className="flex flex-col items-center justify-center gap-4 p-4 border-2 border-primary rounded-lg bg-primary/5 min-h-[180px]">
<div className="flex items-center gap-2">
<Monitor className="h-5 w-5 text-primary" />
<span className="font-medium">Capture complete</span>
</div>
<p className="text-sm text-muted-foreground text-center">File: {file.name}</p>
<div className="flex gap-2">
<Button type="button" size="icon" variant="outline" onClick={onPlayPause}>
{isPlaying ? <Pause className="h-4 w-4" /> : <Play className="h-4 w-4" />}
</Button>
<Button
type="button"
variant="outline"
onClick={onTranscribe}
disabled={isTranscribing}
className="flex items-center gap-2"
>
<Mic className="h-4 w-4" />
{isTranscribing ? 'Transcribing...' : 'Transcribe'}
</Button>
<Button
type="button"
variant="outline"
onClick={onCancel}
className="flex items-center gap-2"
>
Capture Again
</Button>
</div>
</div>
)}
</div>
</FormControl>
<FormMessage />
</FormItem>
);
}
@@ -0,0 +1,147 @@
import { Mic, Pause, Play, Upload } from 'lucide-react';
import { useRef, useState } from 'react';
import { Button } from '@/components/ui/button';
import { FormControl, FormItem, FormMessage } from '@/components/ui/form';
interface AudioSampleUploadProps {
file: File | null | undefined;
onFileChange: (file: File | undefined) => void;
onTranscribe: () => void;
onPlayPause: () => void;
isPlaying: boolean;
isValidating?: boolean;
isTranscribing?: boolean;
isDisabled?: boolean;
fieldName: string;
}
export function AudioSampleUpload({
file,
onFileChange,
onTranscribe,
onPlayPause,
isPlaying,
isValidating = false,
isTranscribing = false,
isDisabled = false,
fieldName,
}: AudioSampleUploadProps) {
const [isDragging, setIsDragging] = useState(false);
const fileInputRef = useRef<HTMLInputElement>(null);
return (
<FormItem>
<FormControl>
<div className="flex flex-col gap-2">
<input
type="file"
accept="audio/*"
name={fieldName}
ref={fileInputRef}
onChange={(e) => {
const selectedFile = e.target.files?.[0];
if (selectedFile) {
onFileChange(selectedFile);
} else {
onFileChange(undefined);
}
}}
className="hidden"
/>
<div
role="button"
tabIndex={0}
onDragOver={(e) => {
e.preventDefault();
setIsDragging(true);
}}
onDragLeave={(e) => {
e.preventDefault();
setIsDragging(false);
}}
onDrop={(e) => {
e.preventDefault();
setIsDragging(false);
const droppedFile = e.dataTransfer.files?.[0];
if (droppedFile?.type.startsWith('audio/')) {
onFileChange(droppedFile);
}
}}
onKeyDown={(e) => {
if (e.key === 'Enter' || e.key === ' ') {
e.preventDefault();
fileInputRef.current?.click();
}
}}
className={`flex flex-col items-center justify-center gap-4 p-4 border-2 rounded-lg transition-colors min-h-[180px] ${
file
? 'border-primary bg-primary/5'
: isDragging
? 'border-primary bg-primary/5'
: 'border-dashed border-muted-foreground/25 hover:border-muted-foreground/50'
}`}
>
{!file ? (
<>
<Button
type="button"
size="lg"
onClick={() => fileInputRef.current?.click()}
className="flex items-center gap-2"
>
<Upload className="h-5 w-5" />
Choose File
</Button>
<p className="text-sm text-muted-foreground text-center">
Click to choose a file or drag and drop. Maximum duration: 30 seconds.
</p>
</>
) : (
<>
<div className="flex items-center gap-2">
<Upload className="h-5 w-5 text-primary" />
<span className="font-medium">File uploaded</span>
</div>
<p className="text-sm text-muted-foreground text-center">File: {file.name}</p>
<div className="flex gap-2">
<Button
type="button"
size="icon"
variant="outline"
onClick={onPlayPause}
disabled={isValidating}
>
{isPlaying ? <Pause className="h-4 w-4" /> : <Play className="h-4 w-4" />}
</Button>
<Button
type="button"
variant="outline"
onClick={onTranscribe}
disabled={isTranscribing || isValidating || isDisabled}
className="flex items-center gap-2"
>
<Mic className="h-4 w-4" />
{isTranscribing ? 'Transcribing...' : 'Transcribe'}
</Button>
<Button
type="button"
variant="outline"
onClick={() => {
onFileChange(undefined);
if (fileInputRef.current) {
fileInputRef.current.value = '';
}
}}
>
Remove
</Button>
</div>
</>
)}
</div>
</div>
</FormControl>
<FormMessage />
</FormItem>
);
}
@@ -1,5 +1,6 @@
import { Edit, Eye, Mic, Trash2 } from 'lucide-react';
import { Download, Edit, Mic, Trash2 } from 'lucide-react';
import { useState } from 'react';
import { useServerStore } from '@/stores/serverStore';
import { Badge } from '@/components/ui/badge';
import { Button } from '@/components/ui/button';
import { Card, CardContent, CardHeader, CardTitle } from '@/components/ui/card';
@@ -13,26 +14,31 @@ import {
DialogTitle,
} from '@/components/ui/dialog';
import type { VoiceProfileResponse } from '@/lib/api/types';
import { useDeleteProfile } from '@/lib/hooks/useProfiles';
import { useDeleteProfile, useExportProfile } from '@/lib/hooks/useProfiles';
import { cn } from '@/lib/utils/cn';
import { useUIStore } from '@/stores/uiStore';
import { ProfileDetail } from './ProfileDetail';
interface ProfileCardProps {
profile: VoiceProfileResponse;
}
export function ProfileCard({ profile }: ProfileCardProps) {
const [detailOpen, setDetailOpen] = useState(false);
const [deleteDialogOpen, setDeleteDialogOpen] = useState(false);
const [avatarError, setAvatarError] = useState(false);
const deleteProfile = useDeleteProfile();
const exportProfile = useExportProfile();
const setEditingProfileId = useUIStore((state) => state.setEditingProfileId);
const setProfileDialogOpen = useUIStore((state) => state.setProfileDialogOpen);
const selectedProfileId = useUIStore((state) => state.selectedProfileId);
const setSelectedProfileId = useUIStore((state) => state.setSelectedProfileId);
const serverUrl = useServerStore((state) => state.serverUrl);
const isSelected = selectedProfileId === profile.id;
const avatarUrl = profile.avatar_path
? `${serverUrl}/profiles/${profile.id}/avatar`
: null;
const handleSelect = () => {
setSelectedProfileId(isSelected ? null : profile.id);
};
@@ -52,6 +58,11 @@ export function ProfileCard({ profile }: ProfileCardProps) {
setDeleteDialogOpen(false);
};
const handleExport = (e: React.MouseEvent) => {
e.stopPropagation();
exportProfile.mutate(profile.id);
};
return (
<>
<Card
@@ -63,8 +74,20 @@ export function ProfileCard({ profile }: ProfileCardProps) {
>
<CardHeader className="p-3 pb-2">
<CardTitle className="flex items-center gap-1.5 text-base font-medium">
<div className="h-6 w-6 rounded-full bg-muted flex items-center justify-center shrink-0">
<Mic className="h-3.5 w-3.5 text-muted-foreground" />
<div className="h-6 w-6 rounded-full bg-muted flex items-center justify-center shrink-0 overflow-hidden">
{avatarUrl && !avatarError ? (
<img
src={avatarUrl}
alt={`${profile.name} avatar`}
className={cn(
'h-full w-full object-cover transition-all duration-200',
!isSelected && 'grayscale'
)}
onError={() => setAvatarError(true)}
/>
) : (
<Mic className="h-3.5 w-3.5 text-muted-foreground" />
)}
</div>
<span className="break-words">{profile.name}</span>
</CardTitle>
@@ -80,12 +103,10 @@ export function ProfileCard({ profile }: ProfileCardProps) {
</div>
<div className="flex gap-0.5 justify-end items-end mt-auto">
<CircleButton
icon={Eye}
onClick={(e) => {
e.stopPropagation();
setDetailOpen(true);
}}
aria-label="View details"
icon={Download}
onClick={handleExport}
disabled={exportProfile.isPending}
aria-label="Export profile"
/>
<CircleButton
icon={Edit}
@@ -105,8 +126,6 @@ export function ProfileCard({ profile }: ProfileCardProps) {
</CardContent>
</Card>
<ProfileDetail profileId={profile.id} open={detailOpen} onOpenChange={setDetailOpen} />
<Dialog open={deleteDialogOpen} onOpenChange={setDeleteDialogOpen}>
<DialogContent>
<DialogHeader>
@@ -1,66 +0,0 @@
import { Badge } from '@/components/ui/badge';
import {
Dialog,
DialogContent,
DialogDescription,
DialogHeader,
DialogTitle,
} from '@/components/ui/dialog';
import { useProfile } from '@/lib/hooks/useProfiles';
import { formatDate } from '@/lib/utils/format';
import { SampleList } from './SampleList';
interface ProfileDetailProps {
profileId: string;
open: boolean;
onOpenChange: (open: boolean) => void;
}
export function ProfileDetail({ profileId, open, onOpenChange }: ProfileDetailProps) {
const { data: profile, isLoading } = useProfile(profileId);
if (isLoading) {
return (
<Dialog open={open} onOpenChange={onOpenChange}>
<DialogContent>
<div className="text-muted-foreground">Loading profile...</div>
</DialogContent>
</Dialog>
);
}
if (!profile) {
return null;
}
return (
<Dialog open={open} onOpenChange={onOpenChange}>
<DialogContent className="max-w-3xl max-h-[90vh] overflow-y-auto">
<DialogHeader>
<DialogTitle>{profile.name}</DialogTitle>
<DialogDescription>Manage samples and view profile details</DialogDescription>
</DialogHeader>
<div className="space-y-4">
{profile.description && (
<div>
<h3 className="text-sm font-medium mb-1">Description</h3>
<p className="text-sm text-muted-foreground">{profile.description}</p>
</div>
)}
<div className="flex gap-2">
<Badge variant="outline">{profile.language}</Badge>
<span className="text-xs text-muted-foreground">
Created {formatDate(profile.created_at)}
</span>
</div>
<div className="border-t pt-4">
<SampleList profileId={profileId} />
</div>
</div>
</DialogContent>
</Dialog>
);
}
File diff suppressed because it is too large Load Diff
@@ -11,11 +11,7 @@ export function ProfileList() {
const setDialogOpen = useUIStore((state) => state.setProfileDialogOpen);
if (isLoading) {
return (
<div className="flex items-center justify-center p-8">
<div className="text-muted-foreground">Loading profiles...</div>
</div>
);
return null;
}
if (error) {
@@ -30,14 +26,6 @@ export function ProfileList() {
return (
<div className="flex flex-col">
<div className="flex items-center justify-between mb-4 shrink-0">
<h2 className="text-2xl font-bold">Voicebox</h2>
<Button onClick={() => setDialogOpen(true)}>
<Sparkles className="mr-2 h-4 w-4" />
New Profile
</Button>
</div>
<div className="shrink-0">
{allProfiles.length === 0 ? (
<Card>
@@ -48,12 +36,12 @@ export function ProfileList() {
</p>
<Button onClick={() => setDialogOpen(true)}>
<Sparkles className="mr-2 h-4 w-4" />
Create Profile
Create Voice
</Button>
</CardContent>
</Card>
) : (
<div className="grid gap-4 grid-cols-3 auto-rows-auto p-1">
<div className="grid gap-4 grid-cols-3 auto-rows-auto p-1 pb-[150px]">
{allProfiles.map((profile) => (
<ProfileCard key={profile.id} profile={profile} />
))}
+260 -53
View File
@@ -1,12 +1,133 @@
import { Plus, Trash2, Play } from 'lucide-react';
import { useState } from 'react';
import { Plus, Trash2, Play, Edit, Check, X, Volume2, Pause } from 'lucide-react';
import { useEffect, useRef, useState } from 'react';
import { Button } from '@/components/ui/button';
import { CircleButton } from '@/components/ui/circle-button';
import { Textarea } from '@/components/ui/textarea';
import { Slider } from '@/components/ui/slider';
import { useToast } from '@/components/ui/use-toast';
import { useDeleteSample, useProfileSamples } from '@/lib/hooks/useProfiles';
import { useServerStore } from '@/stores/serverStore';
import { usePlayerStore } from '@/stores/playerStore';
import { apiClient } from '@/lib/api/client';
import { useDeleteSample, useProfileSamples, useUpdateSample } from '@/lib/hooks/useProfiles';
import { formatAudioDuration } from '@/lib/utils/audio';
import { cn } from '@/lib/utils/cn';
import { SampleUpload } from './SampleUpload';
interface MiniSamplePlayerProps {
audioUrl: string;
}
function MiniSamplePlayer({ audioUrl }: MiniSamplePlayerProps) {
const audioRef = useRef<HTMLAudioElement | null>(null);
const [isPlaying, setIsPlaying] = useState(false);
const [currentTime, setCurrentTime] = useState(0);
const [duration, setDuration] = useState(0);
const [isLoading, setIsLoading] = useState(true);
useEffect(() => {
const audio = new Audio(audioUrl);
audioRef.current = audio;
const handleLoadedMetadata = () => {
setDuration(audio.duration);
setIsLoading(false);
};
const handleTimeUpdate = () => {
setCurrentTime(audio.currentTime);
};
const handleEnded = () => {
setIsPlaying(false);
setCurrentTime(0);
};
const handlePlay = () => setIsPlaying(true);
const handlePause = () => setIsPlaying(false);
audio.addEventListener('loadedmetadata', handleLoadedMetadata);
audio.addEventListener('timeupdate', handleTimeUpdate);
audio.addEventListener('ended', handleEnded);
audio.addEventListener('play', handlePlay);
audio.addEventListener('pause', handlePause);
return () => {
audio.pause();
audio.removeEventListener('loadedmetadata', handleLoadedMetadata);
audio.removeEventListener('timeupdate', handleTimeUpdate);
audio.removeEventListener('ended', handleEnded);
audio.removeEventListener('play', handlePlay);
audio.removeEventListener('pause', handlePause);
audio.src = '';
};
}, [audioUrl]);
const handlePlayPause = () => {
if (!audioRef.current) return;
if (isPlaying) {
audioRef.current.pause();
} else {
audioRef.current.play();
}
};
const handleSeek = (value: number[]) => {
if (!audioRef.current || duration === 0) return;
const progress = value[0] / 100;
audioRef.current.currentTime = progress * duration;
};
const handleStop = () => {
if (audioRef.current) {
audioRef.current.pause();
audioRef.current.currentTime = 0;
}
setIsPlaying(false);
setCurrentTime(0);
};
return (
<div className="border-t bg-muted/30 px-3 py-2 mt-2">
<div className="flex items-center gap-2">
<Button
type="button"
variant="ghost"
size="icon"
className="h-7 w-7 shrink-0"
onClick={handlePlayPause}
disabled={isLoading}
>
{isPlaying ? <Pause className="h-3.5 w-3.5" /> : <Play className="h-3.5 w-3.5 ml-0.5" />}
</Button>
<div className="flex-1 min-w-0 flex items-center gap-2">
<Slider
value={duration > 0 ? [(currentTime / duration) * 100] : [0]}
onValueChange={handleSeek}
max={100}
step={0.1}
className="flex-1"
/>
<div className="flex items-center gap-1 text-xs text-muted-foreground shrink-0 min-w-[70px]">
<span className="font-mono">{formatAudioDuration(currentTime)}</span>
<span>/</span>
<span className="font-mono">{formatAudioDuration(duration)}</span>
</div>
</div>
<Button
type="button"
variant="ghost"
size="icon"
className="h-7 w-7 shrink-0"
onClick={handleStop}
title="Stop"
>
<X className="h-3.5 w-3.5" />
</Button>
</div>
</div>
);
}
interface SampleListProps {
profileId: string;
}
@@ -14,12 +135,11 @@ interface SampleListProps {
export function SampleList({ profileId }: SampleListProps) {
const { data: samples, isLoading } = useProfileSamples(profileId);
const deleteSample = useDeleteSample();
const [uploadOpen, setUploadOpen] = useState(false);
const updateSample = useUpdateSample();
const { toast } = useToast();
const serverUrl = useServerStore((state) => state.serverUrl);
const setAudio = usePlayerStore((state) => state.setAudio);
const currentAudioId = usePlayerStore((state) => state.audioId);
const isPlaying = usePlayerStore((state) => state.isPlaying);
const [uploadOpen, setUploadOpen] = useState(false);
const [editingSampleId, setEditingSampleId] = useState<string | null>(null);
const [editedText, setEditedText] = useState<string>('');
const handleDelete = (sampleId: string) => {
if (confirm('Are you sure you want to delete this sample?')) {
@@ -27,9 +147,41 @@ export function SampleList({ profileId }: SampleListProps) {
}
};
const handlePlay = (audioPath: string, referenceText: string, sampleId: string) => {
const audioUrl = `${serverUrl}${audioPath}`;
setAudio(audioUrl, sampleId, referenceText.substring(0, 50));
const handleStartEdit = (sampleId: string, currentText: string) => {
setEditingSampleId(sampleId);
setEditedText(currentText);
};
const handleCancelEdit = () => {
setEditingSampleId(null);
setEditedText('');
};
const handleSaveEdit = async (sampleId: string) => {
if (!editedText.trim()) {
toast({
title: 'Invalid text',
description: 'Reference text cannot be empty.',
variant: 'destructive',
});
return;
}
try {
await updateSample.mutateAsync({ sampleId, referenceText: editedText.trim() });
toast({
title: 'Sample updated',
description: 'Reference text has been updated successfully.',
});
setEditingSampleId(null);
setEditedText('');
} catch (error) {
toast({
title: 'Update failed',
description: error instanceof Error ? error.message : 'Failed to update sample',
variant: 'destructive',
});
}
};
if (isLoading) {
@@ -37,54 +189,109 @@ export function SampleList({ profileId }: SampleListProps) {
}
return (
<div className="space-y-4">
<div className="flex items-center justify-between">
<h3 className="text-lg font-semibold">Audio Samples</h3>
<Button size="sm" onClick={() => setUploadOpen(true)}>
<Plus className="mr-2 h-4 w-4" />
Add Sample
</Button>
</div>
<div className="space-y-4 pt-4">
{samples && samples.length === 0 ? (
<div className="text-sm text-muted-foreground py-4">
No samples yet. Add your first audio sample.
<div className="flex flex-col items-center justify-center py-8 text-center border border-dashed rounded-lg">
<Volume2 className="h-8 w-8 text-muted-foreground/50 mb-2" />
<p className="text-sm text-muted-foreground">No samples yet</p>
<p className="text-xs text-muted-foreground/70 mt-1">Add your first audio sample to get started</p>
</div>
) : (
<div className="space-y-2">
{samples?.map((sample) => (
<div
key={sample.id}
className="flex items-center justify-between p-3 border rounded-lg"
>
<div className="flex-1">
<p className="text-sm font-medium">{sample.reference_text}</p>
<p className="text-xs text-muted-foreground mt-1">{sample.audio_path}</p>
{samples?.map((sample, index) => {
const isEditing = editingSampleId === sample.id;
return (
<div
key={sample.id}
className={cn(
'group relative rounded-lg border bg-card transition-all duration-200',
isEditing ? 'ring-2 ring-primary/20' : 'hover:border-primary/30'
)}
>
{isEditing ? (
/* Edit Mode */
<div className="p-4 space-y-3">
<div className="flex items-center gap-2 text-xs text-muted-foreground mb-2">
<Edit className="h-3 w-3" />
<span>Editing transcription</span>
</div>
<Textarea
value={editedText}
onChange={(e) => setEditedText(e.target.value)}
className="min-h-[100px] text-sm resize-none"
placeholder="Enter reference text..."
autoFocus
/>
<div className="flex items-center justify-end gap-2 pt-1">
<Button
type="button"
size="sm"
variant="ghost"
onClick={handleCancelEdit}
disabled={updateSample.isPending}
>
<X className="h-4 w-4 mr-1" />
Cancel
</Button>
<Button
type="button"
size="sm"
onClick={() => handleSaveEdit(sample.id)}
disabled={updateSample.isPending}
>
<Check className="h-4 w-4 mr-1" />
{updateSample.isPending ? 'Saving...' : 'Save'}
</Button>
</div>
</div>
) : (
<>
{/* View Mode */}
<div className="flex items-center gap-3 p-3 h-[72px]">
{/* Text Content */}
<div className="flex-1 min-w-0 py-0.5">
<p className="text-sm font-medium line-clamp-2 leading-snug">
{sample.reference_text}
</p>
</div>
{/* Action Buttons */}
<div className="shrink-0 flex items-center gap-0.5 opacity-0 group-hover:opacity-100 transition-opacity">
<CircleButton
icon={Edit}
title="Edit transcription"
onClick={() => handleStartEdit(sample.id, sample.reference_text)}
/>
<CircleButton
icon={Trash2}
title="Delete sample"
onClick={() => handleDelete(sample.id)}
disabled={deleteSample.isPending}
/>
</div>
{/* Sample Number Badge */}
<div className="absolute top-1 right-2 text-[10px] text-muted-foreground/50 font-medium">
#{index + 1}
</div>
</div>
{/* Mini Player - Always visible */}
<MiniSamplePlayer audioUrl={apiClient.getSampleUrl(sample.id)} />
</>
)}
</div>
<div className="flex gap-2">
<Button
variant="ghost"
size="sm"
onClick={() => handlePlay(sample.audio_path, sample.reference_text, sample.id)}
className={currentAudioId === sample.id && isPlaying ? 'text-primary' : ''}
>
<Play className="h-4 w-4 mr-1" />
Play
</Button>
<Button
variant="ghost"
size="sm"
onClick={() => handleDelete(sample.id)}
disabled={deleteSample.isPending}
>
<Trash2 className="h-4 w-4 text-destructive" />
</Button>
</div>
</div>
))}
);
})}
</div>
)}
<Button type="button" variant="outline" className="w-full" onClick={() => setUploadOpen(true)}>
<Plus className="mr-2 h-4 w-4" />
Add Sample
</Button>
<SampleUpload profileId={profileId} open={uploadOpen} onOpenChange={setUploadOpen} />
</div>
);
+130 -147
View File
@@ -1,6 +1,7 @@
import { zodResolver } from '@hookform/resolvers/zod';
import { Mic, Monitor, Upload } from 'lucide-react';
import { useEffect, useState } from 'react';
import { useForm } from 'react-hook-form';
import { useState, useEffect } from 'react';
import * as z from 'zod';
import { Button } from '@/components/ui/button';
import {
@@ -13,21 +14,23 @@ import {
import {
Form,
FormControl,
FormDescription,
FormField,
FormItem,
FormLabel,
FormMessage,
} from '@/components/ui/form';
import { Input } from '@/components/ui/input';
import { Textarea } from '@/components/ui/textarea';
import { Tabs, TabsContent, TabsList, TabsTrigger } from '@/components/ui/tabs';
import { Textarea } from '@/components/ui/textarea';
import { useToast } from '@/components/ui/use-toast';
import { useAddSample, useProfile } from '@/lib/hooks/useProfiles';
import { useTranscription } from '@/lib/hooks/useTranscription';
import { useAudioPlayer } from '@/lib/hooks/useAudioPlayer';
import { useAudioRecording } from '@/lib/hooks/useAudioRecording';
import { Mic, Square, Upload } from 'lucide-react';
import { formatAudioDuration } from '@/lib/utils/audio';
import { useAddSample, useProfile } from '@/lib/hooks/useProfiles';
import { useSystemAudioCapture } from '@/lib/hooks/useSystemAudioCapture';
import { useTranscription } from '@/lib/hooks/useTranscription';
import { isTauri } from '@/lib/tauri';
import { AudioSampleRecording } from './AudioSampleRecording';
import { AudioSampleSystem } from './AudioSampleSystem';
import { AudioSampleUpload } from './AudioSampleUpload';
const sampleSchema = z.object({
file: z.instanceof(File, { message: 'Please select an audio file' }),
@@ -50,7 +53,8 @@ export function SampleUpload({ profileId, open, onOpenChange }: SampleUploadProp
const transcribe = useTranscription();
const { data: profile } = useProfile(profileId);
const { toast } = useToast();
const [mode, setMode] = useState<'upload' | 'record'>('upload');
const [mode, setMode] = useState<'upload' | 'record' | 'system'>('upload');
const { isPlaying, playPause, cleanup: cleanupAudio } = useAudioPlayer();
const form = useForm<SampleFormValues>({
resolver: zodResolver(sampleSchema),
@@ -69,12 +73,16 @@ export function SampleUpload({ profileId, open, onOpenChange }: SampleUploadProp
stopRecording,
cancelRecording,
} = useAudioRecording({
maxDurationSeconds: 30,
onRecordingComplete: (blob) => {
maxDurationSeconds: 29,
onRecordingComplete: (blob, recordedDuration) => {
// Convert blob to File object
const file = new File([blob], `recording-${Date.now()}.webm`, {
type: blob.type || 'audio/webm',
});
}) as File & { recordedDuration?: number };
// Store the actual recorded duration to bypass metadata reading issues on Windows
if (recordedDuration !== undefined) {
file.recordedDuration = recordedDuration;
}
form.setValue('file', file, { shouldValidate: true });
toast({
title: 'Recording complete',
@@ -83,6 +91,33 @@ export function SampleUpload({ profileId, open, onOpenChange }: SampleUploadProp
},
});
const {
isRecording: isSystemRecording,
duration: systemDuration,
error: systemRecordingError,
isSupported: isSystemAudioSupported,
startRecording: startSystemRecording,
stopRecording: stopSystemRecording,
cancelRecording: cancelSystemRecording,
} = useSystemAudioCapture({
maxDurationSeconds: 29,
onRecordingComplete: (blob, recordedDuration) => {
// Convert blob to File object
const file = new File([blob], `system-audio-${Date.now()}.wav`, {
type: blob.type || 'audio/wav',
}) as File & { recordedDuration?: number };
// Store the actual recorded duration to bypass metadata reading issues on Windows
if (recordedDuration !== undefined) {
file.recordedDuration = recordedDuration;
}
form.setValue('file', file, { shouldValidate: true });
toast({
title: 'System audio captured',
description: 'Audio has been captured successfully.',
});
},
});
// Show recording errors
useEffect(() => {
if (recordingError) {
@@ -94,6 +129,17 @@ export function SampleUpload({ profileId, open, onOpenChange }: SampleUploadProp
}
}, [recordingError, toast]);
// Show system audio recording errors
useEffect(() => {
if (systemRecordingError) {
toast({
title: 'System audio capture error',
description: systemRecordingError,
variant: 'destructive',
});
}
}, [systemRecordingError, toast]);
async function handleTranscribe() {
const file = form.getValues('file');
if (!file) {
@@ -110,11 +156,6 @@ export function SampleUpload({ profileId, open, onOpenChange }: SampleUploadProp
const result = await transcribe.mutateAsync({ file, language });
form.setValue('referenceText', result.text, { shouldValidate: true });
toast({
title: 'Transcription complete',
description: 'Audio has been transcribed successfully.',
});
} catch (error) {
toast({
title: 'Transcription failed',
@@ -154,14 +195,27 @@ export function SampleUpload({ profileId, open, onOpenChange }: SampleUploadProp
if (isRecording) {
cancelRecording();
}
if (isSystemRecording) {
cancelSystemRecording();
}
cleanupAudio();
}
onOpenChange(newOpen);
}
function handleCancelRecording() {
cancelRecording();
// Reset file field by clearing the input
if (mode === 'record') {
cancelRecording();
} else if (mode === 'system') {
cancelSystemRecording();
}
form.resetField('file');
cleanupAudio();
}
function handlePlayPause() {
const file = form.getValues('file');
playPause(file);
}
return (
@@ -176,58 +230,40 @@ export function SampleUpload({ profileId, open, onOpenChange }: SampleUploadProp
<Form {...form}>
<form onSubmit={form.handleSubmit(onSubmit)} className="space-y-4">
<Tabs value={mode} onValueChange={(v) => setMode(v as 'upload' | 'record')}>
<TabsList className="grid w-full grid-cols-2">
<Tabs value={mode} onValueChange={(v) => setMode(v as 'upload' | 'record' | 'system')}>
<TabsList
className={`grid w-full ${isTauri() && isSystemAudioSupported ? 'grid-cols-3' : 'grid-cols-2'}`}
>
<TabsTrigger value="upload" className="flex items-center gap-2">
<Upload className="h-4 w-4" />
<Upload className="h-4 w-4 shrink-0" />
Upload
</TabsTrigger>
<TabsTrigger value="record" className="flex items-center gap-2">
<Mic className="h-4 w-4" />
<Mic className="h-4 w-4 shrink-0" />
Record
</TabsTrigger>
{isTauri() && isSystemAudioSupported && (
<TabsTrigger value="system" className="flex items-center gap-2">
<Monitor className="h-4 w-4 shrink-0" />
System Audio
</TabsTrigger>
)}
</TabsList>
<TabsContent value="upload" className="space-y-4">
<FormField
control={form.control}
name="file"
render={({ field: { onChange, value, ...field } }) => (
<FormItem>
<FormLabel>Audio File</FormLabel>
<FormControl>
<div className="flex items-center gap-2">
<Input
type="file"
accept="audio/*"
onChange={(e) => {
const file = e.target.files?.[0];
if (file) {
onChange(file);
}
}}
{...field}
/>
{selectedFile && (
<Button
type="button"
variant="outline"
onClick={handleTranscribe}
disabled={transcribe.isPending}
className="flex items-center gap-2"
>
<Mic className="h-4 w-4" />
{transcribe.isPending ? 'Transcribing...' : 'Transcribe'}
</Button>
)}
</div>
</FormControl>
<FormDescription>
Supported formats: WAV, MP3, M4A. Click "Transcribe" to automatically
extract text from the audio.
</FormDescription>
<FormMessage />
</FormItem>
render={({ field: { onChange, name } }) => (
<AudioSampleUpload
file={selectedFile}
onFileChange={onChange}
onTranscribe={handleTranscribe}
onPlayPause={handlePlayPause}
isPlaying={isPlaying}
isTranscribing={transcribe.isPending}
fieldName={name}
/>
)}
/>
</TabsContent>
@@ -237,94 +273,44 @@ export function SampleUpload({ profileId, open, onOpenChange }: SampleUploadProp
control={form.control}
name="file"
render={() => (
<FormItem>
<FormLabel>Record Audio</FormLabel>
<FormControl>
<div className="space-y-4">
{!isRecording && !selectedFile && (
<div className="flex flex-col items-center gap-4 p-6 border-2 border-dashed rounded-lg">
<Button
type="button"
onClick={startRecording}
size="lg"
className="flex items-center gap-2"
>
<Mic className="h-5 w-5" />
Start Recording
</Button>
<p className="text-sm text-muted-foreground text-center">
Click to start recording. Maximum duration: 30 seconds.
</p>
</div>
)}
{isRecording && (
<div className="flex flex-col items-center gap-4 p-6 border-2 border-destructive rounded-lg bg-destructive/5">
<div className="flex items-center gap-4">
<div className="flex items-center gap-2">
<div className="h-3 w-3 rounded-full bg-destructive animate-pulse" />
<span className="text-lg font-mono font-semibold">
{formatAudioDuration(duration)}
</span>
</div>
</div>
<Button
type="button"
onClick={stopRecording}
variant="destructive"
className="flex items-center gap-2"
>
<Square className="h-4 w-4" />
Stop Recording
</Button>
<p className="text-sm text-muted-foreground text-center">
Recording in progress... ({formatAudioDuration(30 - duration)}{' '}
remaining)
</p>
</div>
)}
{selectedFile && !isRecording && (
<div className="flex flex-col items-center gap-4 p-6 border-2 border-primary rounded-lg bg-primary/5">
<div className="flex items-center gap-2">
<Mic className="h-5 w-5 text-primary" />
<span className="font-medium">Recording complete</span>
</div>
<p className="text-sm text-muted-foreground">
File: {selectedFile.name}
</p>
<div className="flex gap-2">
<Button
type="button"
variant="outline"
onClick={handleTranscribe}
disabled={transcribe.isPending}
className="flex items-center gap-2"
>
<Mic className="h-4 w-4" />
{transcribe.isPending ? 'Transcribing...' : 'Transcribe'}
</Button>
<Button
type="button"
variant="outline"
onClick={handleCancelRecording}
className="flex items-center gap-2"
>
Record Again
</Button>
</div>
</div>
)}
</div>
</FormControl>
<FormDescription>
Record audio directly from your microphone. Maximum duration is 30 seconds.
</FormDescription>
<FormMessage />
</FormItem>
<AudioSampleRecording
file={selectedFile}
isRecording={isRecording}
duration={duration}
onStart={startRecording}
onStop={stopRecording}
onCancel={handleCancelRecording}
onTranscribe={handleTranscribe}
onPlayPause={handlePlayPause}
isPlaying={isPlaying}
isTranscribing={transcribe.isPending}
/>
)}
/>
</TabsContent>
{isTauri() && isSystemAudioSupported && (
<TabsContent value="system" className="space-y-4">
<FormField
control={form.control}
name="file"
render={() => (
<AudioSampleSystem
file={selectedFile}
isRecording={isSystemRecording}
duration={systemDuration}
onStart={startSystemRecording}
onStop={stopSystemRecording}
onCancel={handleCancelRecording}
onTranscribe={handleTranscribe}
onPlayPause={handlePlayPause}
isPlaying={isPlaying}
isTranscribing={transcribe.isPending}
/>
)}
/>
</TabsContent>
)}
</Tabs>
<FormField
@@ -340,9 +326,6 @@ export function SampleUpload({ profileId, open, onOpenChange }: SampleUploadProp
{...field}
/>
</FormControl>
<FormDescription>
This should match exactly what is spoken in the audio file.
</FormDescription>
<FormMessage />
</FormItem>
)}
+234
View File
@@ -0,0 +1,234 @@
import { useQuery, useQueryClient } from '@tanstack/react-query';
import { Edit, MoreHorizontal, Plus, Trash2, Mic } from 'lucide-react';
import { useMemo, useRef } from 'react';
import { Button } from '@/components/ui/button';
import {
DropdownMenu,
DropdownMenuContent,
DropdownMenuItem,
DropdownMenuTrigger,
} from '@/components/ui/dropdown-menu';
import { MultiSelect } from '@/components/ui/multi-select';
import {
Table,
TableBody,
TableCell,
TableHead,
TableHeader,
TableRow,
} from '@/components/ui/table';
import { ProfileForm } from '@/components/VoiceProfiles/ProfileForm';
import { apiClient } from '@/lib/api/client';
import type { VoiceProfileResponse } from '@/lib/api/types';
import { BOTTOM_SAFE_AREA_PADDING } from '@/lib/constants/ui';
import { useHistory } from '@/lib/hooks/useHistory';
import { useDeleteProfile, useProfileSamples, useProfiles } from '@/lib/hooks/useProfiles';
import { cn } from '@/lib/utils/cn';
import { usePlayerStore } from '@/stores/playerStore';
import { useUIStore } from '@/stores/uiStore';
export function VoicesTab() {
const { data: profiles, isLoading } = useProfiles();
const { data: historyData } = useHistory({ limit: 1000 });
const queryClient = useQueryClient();
const setDialogOpen = useUIStore((state) => state.setProfileDialogOpen);
const setEditingProfileId = useUIStore((state) => state.setEditingProfileId);
const deleteProfile = useDeleteProfile();
const scrollRef = useRef<HTMLDivElement>(null);
const audioUrl = usePlayerStore((state) => state.audioUrl);
const isPlayerVisible = !!audioUrl;
// Get generation counts per profile
const generationCounts = useMemo(() => {
const counts: Record<string, number> = {};
if (historyData?.items) {
historyData.items.forEach((item) => {
counts[item.profile_id] = (counts[item.profile_id] || 0) + 1;
});
}
return counts;
}, [historyData]);
// Get channel assignments for each profile
const { data: channelAssignments } = useQuery({
queryKey: ['profile-channels'],
queryFn: async () => {
if (!profiles) return {};
const assignments: Record<string, string[]> = {};
for (const profile of profiles) {
try {
const result = await apiClient.getProfileChannels(profile.id);
assignments[profile.id] = result.channel_ids;
} catch {
assignments[profile.id] = [];
}
}
return assignments;
},
enabled: !!profiles,
});
// Get all channels
const { data: channels } = useQuery({
queryKey: ['channels'],
queryFn: () => apiClient.listChannels(),
});
const handleEdit = (profileId: string) => {
setEditingProfileId(profileId);
setDialogOpen(true);
};
const handleDelete = (profileId: string) => {
if (confirm('Are you sure you want to delete this profile?')) {
deleteProfile.mutate(profileId);
}
};
const handleChannelChange = async (profileId: string, channelIds: string[]) => {
try {
await apiClient.setProfileChannels(profileId, channelIds);
queryClient.invalidateQueries({ queryKey: ['profile-channels'] });
} catch (error) {
console.error('Failed to update channels:', error);
}
};
if (isLoading) {
return (
<div className="flex items-center justify-center h-full">
<div className="text-muted-foreground">Loading voices...</div>
</div>
);
}
return (
<div className="h-full flex flex-col relative overflow-hidden">
{/* Scroll Mask - Always visible, behind content */}
<div className="absolute top-0 left-0 right-0 h-16 bg-gradient-to-b from-background to-transparent z-10 pointer-events-none" />
{/* Fixed Header */}
<div className="absolute top-0 left-0 right-0 z-20">
<div className="flex items-center justify-between mb-6">
<h1 className="text-2xl font-bold">Voices</h1>
<Button onClick={() => setDialogOpen(true)}>
<Plus className="h-4 w-4 mr-2" />
New Voice
</Button>
</div>
</div>
{/* Scrollable Content */}
<div
ref={scrollRef}
className={cn(
'flex-1 overflow-y-auto pt-16 relative z-0',
isPlayerVisible && BOTTOM_SAFE_AREA_PADDING,
)}
>
<Table>
<TableHeader>
<TableRow>
<TableHead>Name</TableHead>
<TableHead>Language</TableHead>
<TableHead>Generations</TableHead>
<TableHead>Samples</TableHead>
<TableHead>Channels</TableHead>
<TableHead className="w-[50px]"></TableHead>
</TableRow>
</TableHeader>
<TableBody>
{profiles?.map((profile) => (
<VoiceRow
key={profile.id}
profile={profile}
generationCount={generationCounts[profile.id] || 0}
channelIds={channelAssignments?.[profile.id] || []}
channels={channels || []}
onChannelChange={(channelIds) => handleChannelChange(profile.id, channelIds)}
onEdit={() => handleEdit(profile.id)}
onDelete={() => handleDelete(profile.id)}
/>
))}
</TableBody>
</Table>
</div>
<ProfileForm />
</div>
);
}
interface VoiceRowProps {
profile: VoiceProfileResponse;
generationCount: number;
channelIds: string[];
channels: Array<{ id: string; name: string; is_default: boolean }>;
onChannelChange: (channelIds: string[]) => void;
onEdit: () => void;
onDelete: () => void;
}
function VoiceRow({
profile,
generationCount,
channelIds,
channels,
onChannelChange,
onEdit,
onDelete,
}: VoiceRowProps) {
const { data: samples } = useProfileSamples(profile.id);
return (
<TableRow className="cursor-pointer" onClick={onEdit}>
<TableCell>
<div className="flex items-center gap-2">
<div className="h-8 w-8 rounded-lg bg-muted flex items-center justify-center shrink-0">
<Mic className="h-4 w-4 text-muted-foreground" />
</div>
<div>
<div className="font-medium">{profile.name}</div>
{profile.description && (
<div className="text-sm text-muted-foreground">{profile.description}</div>
)}
</div>
</div>
</TableCell>
<TableCell onClick={(e) => e.stopPropagation()}>{profile.language}</TableCell>
<TableCell onClick={(e) => e.stopPropagation()}>{generationCount}</TableCell>
<TableCell onClick={(e) => e.stopPropagation()}>{samples?.length || 0}</TableCell>
<TableCell onClick={(e) => e.stopPropagation()}>
<MultiSelect
options={channels.map((ch) => ({
value: ch.id,
label: `${ch.name}${ch.is_default ? ' (Default)' : ''}`,
}))}
value={channelIds}
onChange={onChannelChange}
placeholder="Select channels..."
className="min-w-[200px]"
/>
</TableCell>
<TableCell onClick={(e) => e.stopPropagation()}>
<DropdownMenu>
<DropdownMenuTrigger asChild>
<Button variant="ghost" size="icon">
<MoreHorizontal className="h-4 w-4" />
</Button>
</DropdownMenuTrigger>
<DropdownMenuContent>
<DropdownMenuItem onClick={onEdit}>
<Edit className="h-4 w-4 mr-2" />
Edit
</DropdownMenuItem>
<DropdownMenuItem onClick={onDelete} className="text-destructive">
<Trash2 className="h-4 w-4 mr-2" />
Delete
</DropdownMenuItem>
</DropdownMenuContent>
</DropdownMenu>
</TableCell>
</TableRow>
);
}
+114
View File
@@ -0,0 +1,114 @@
import * as AlertDialogPrimitive from '@radix-ui/react-alert-dialog';
import * as React from 'react';
import { cn } from '@/lib/utils/cn';
import { buttonVariants } from './button';
const AlertDialog = AlertDialogPrimitive.Root;
const AlertDialogTrigger = AlertDialogPrimitive.Trigger;
const AlertDialogPortal = AlertDialogPrimitive.Portal;
const AlertDialogOverlay = React.forwardRef<
React.ElementRef<typeof AlertDialogPrimitive.Overlay>,
React.ComponentPropsWithoutRef<typeof AlertDialogPrimitive.Overlay>
>(({ className, ...props }, ref) => (
<AlertDialogPrimitive.Overlay
className={cn(
'fixed inset-0 z-50 bg-black/80 data-[state=open]:animate-in data-[state=closed]:animate-out data-[state=closed]:fade-out-0 data-[state=open]:fade-in-0',
className,
)}
{...props}
ref={ref}
/>
));
AlertDialogOverlay.displayName = AlertDialogPrimitive.Overlay.displayName;
const AlertDialogContent = React.forwardRef<
React.ElementRef<typeof AlertDialogPrimitive.Content>,
React.ComponentPropsWithoutRef<typeof AlertDialogPrimitive.Content>
>(({ className, ...props }, ref) => (
<AlertDialogPortal>
<AlertDialogOverlay />
<AlertDialogPrimitive.Content
ref={ref}
className={cn(
'fixed left-[50%] top-[50%] z-50 grid w-full max-w-lg translate-x-[-50%] translate-y-[-50%] gap-4 border bg-background p-6 shadow-lg duration-200 data-[state=open]:animate-in data-[state=closed]:animate-out data-[state=closed]:fade-out-0 data-[state=open]:fade-in-0 data-[state=closed]:zoom-out-95 data-[state=open]:zoom-in-95 data-[state=closed]:slide-out-to-left-1/2 data-[state=closed]:slide-out-to-top-[48%] data-[state=open]:slide-in-from-left-1/2 data-[state=open]:slide-in-from-top-[48%] sm:rounded-lg',
className,
)}
{...props}
/>
</AlertDialogPortal>
));
AlertDialogContent.displayName = AlertDialogPrimitive.Content.displayName;
const AlertDialogHeader = ({ className, ...props }: React.HTMLAttributes<HTMLDivElement>) => (
<div className={cn('flex flex-col space-y-2 text-center sm:text-left', className)} {...props} />
);
AlertDialogHeader.displayName = 'AlertDialogHeader';
const AlertDialogFooter = ({ className, ...props }: React.HTMLAttributes<HTMLDivElement>) => (
<div
className={cn('flex flex-col-reverse sm:flex-row sm:justify-end sm:space-x-2', className)}
{...props}
/>
);
AlertDialogFooter.displayName = 'AlertDialogFooter';
const AlertDialogTitle = React.forwardRef<
React.ElementRef<typeof AlertDialogPrimitive.Title>,
React.ComponentPropsWithoutRef<typeof AlertDialogPrimitive.Title>
>(({ className, ...props }, ref) => (
<AlertDialogPrimitive.Title
ref={ref}
className={cn('text-lg font-semibold', className)}
{...props}
/>
));
AlertDialogTitle.displayName = AlertDialogPrimitive.Title.displayName;
const AlertDialogDescription = React.forwardRef<
React.ElementRef<typeof AlertDialogPrimitive.Description>,
React.ComponentPropsWithoutRef<typeof AlertDialogPrimitive.Description>
>(({ className, ...props }, ref) => (
<AlertDialogPrimitive.Description
ref={ref}
className={cn('text-sm text-muted-foreground', className)}
{...props}
/>
));
AlertDialogDescription.displayName = AlertDialogPrimitive.Description.displayName;
const AlertDialogAction = React.forwardRef<
React.ElementRef<typeof AlertDialogPrimitive.Action>,
React.ComponentPropsWithoutRef<typeof AlertDialogPrimitive.Action>
>(({ className, ...props }, ref) => (
<AlertDialogPrimitive.Action ref={ref} className={cn(buttonVariants(), className)} {...props} />
));
AlertDialogAction.displayName = AlertDialogPrimitive.Action.displayName;
const AlertDialogCancel = React.forwardRef<
React.ElementRef<typeof AlertDialogPrimitive.Cancel>,
React.ComponentPropsWithoutRef<typeof AlertDialogPrimitive.Cancel>
>(({ className, ...props }, ref) => (
<AlertDialogPrimitive.Cancel
ref={ref}
className={cn(buttonVariants({ variant: 'outline' }), 'mt-2 sm:mt-0', className)}
{...props}
/>
));
AlertDialogCancel.displayName = AlertDialogPrimitive.Cancel.displayName;
export {
AlertDialog,
AlertDialogPortal,
AlertDialogOverlay,
AlertDialogTrigger,
AlertDialogContent,
AlertDialogHeader,
AlertDialogFooter,
AlertDialogTitle,
AlertDialogDescription,
AlertDialogAction,
AlertDialogCancel,
};
+27 -19
View File
@@ -1,33 +1,41 @@
import * as React from 'react';
import { Check } from 'lucide-react';
import { cn } from '@/lib/utils/cn';
export interface CheckboxProps extends React.InputHTMLAttributes<HTMLInputElement> {
export interface CheckboxProps {
checked?: boolean;
onCheckedChange?: (checked: boolean) => void;
disabled?: boolean;
className?: string;
id?: string;
}
const Checkbox = React.forwardRef<HTMLInputElement, CheckboxProps>(
({ className, onCheckedChange, ...props }, ref) => {
const handleChange = (e: React.ChangeEvent<HTMLInputElement>) => {
if (onCheckedChange) {
onCheckedChange(e.target.checked);
}
// Call original onChange if provided
if (props.onChange) {
props.onChange(e);
}
};
const Checkbox = React.forwardRef<HTMLButtonElement, CheckboxProps>(
({ checked = false, onCheckedChange, disabled = false, className, id, ...props }, ref) => {
return (
<input
type="checkbox"
<button
type="button"
ref={ref}
id={id}
role="checkbox"
aria-checked={checked}
disabled={disabled}
onClick={() => {
if (!disabled && onCheckedChange) {
onCheckedChange(!checked);
}
}}
className={cn(
'h-4 w-4 rounded border-gray-300 text-primary focus:ring-2 focus:ring-primary focus:ring-offset-2 disabled:cursor-not-allowed disabled:opacity-50',
'h-4 w-4 rounded border-2 flex items-center justify-center shrink-0 transition-colors',
checked ? 'bg-accent border-accent' : 'border-muted-foreground/30',
disabled && 'opacity-50 cursor-not-allowed',
!disabled && 'cursor-pointer',
className,
)}
ref={ref}
onChange={handleChange}
{...props}
/>
>
{checked && <Check className="h-3 w-3 text-accent-foreground" />}
</button>
);
},
);
+102
View File
@@ -0,0 +1,102 @@
import * as React from 'react';
import { ChevronDown, Check } from 'lucide-react';
import { cn } from '@/lib/utils/cn';
import {
DropdownMenu,
DropdownMenuContent,
DropdownMenuTrigger,
} from '@/components/ui/dropdown-menu';
import * as DropdownMenuPrimitive from '@radix-ui/react-dropdown-menu';
export interface MultiSelectOption {
value: string;
label: string;
}
export interface MultiSelectProps {
options: MultiSelectOption[];
value: string[];
onChange: (value: string[]) => void;
placeholder?: string;
className?: string;
}
const MultiSelectCheckboxItem = React.forwardRef<
React.ElementRef<typeof DropdownMenuPrimitive.CheckboxItem>,
React.ComponentPropsWithoutRef<typeof DropdownMenuPrimitive.CheckboxItem>
>(({ className, children, checked, ...props }, ref) => (
<DropdownMenuPrimitive.CheckboxItem
ref={ref}
className={cn(
'relative flex cursor-default select-none items-center rounded-sm py-1.5 pl-8 pr-2 text-xs outline-none focus:bg-accent focus:text-accent-foreground data-disabled:pointer-events-none data-disabled:opacity-50',
className,
)}
checked={checked}
{...props}
>
<span className="absolute left-2 flex h-3.5 w-3.5 items-center justify-center">
<DropdownMenuPrimitive.ItemIndicator>
<Check className="h-4 w-4" />
</DropdownMenuPrimitive.ItemIndicator>
</span>
{children}
</DropdownMenuPrimitive.CheckboxItem>
));
MultiSelectCheckboxItem.displayName = DropdownMenuPrimitive.CheckboxItem.displayName;
export function MultiSelect({
options,
value,
onChange,
placeholder = 'Select...',
className,
}: MultiSelectProps) {
const [open, setOpen] = React.useState(false);
const handleSelect = (optionValue: string) => {
const newValue = value.includes(optionValue)
? value.filter((v) => v !== optionValue)
: [...value, optionValue];
onChange(newValue);
};
const displayText =
value.length === 0
? placeholder
: value.length === 1
? options.find((opt) => opt.value === value[0])?.label || placeholder
: `${value.length} selected`;
return (
<DropdownMenu open={open} onOpenChange={setOpen}>
<DropdownMenuTrigger asChild>
<button
type="button"
className={cn(
'flex h-8 w-full items-center justify-between rounded-full border border-border bg-card px-3 py-2 text-xs ring-offset-background placeholder:text-muted-foreground focus:outline-none focus:ring-2 focus:ring-ring focus:ring-offset-2 disabled:cursor-not-allowed disabled:opacity-50 hover:bg-background/50 transition-all',
className,
)}
>
<span className="line-clamp-1">{displayText}</span>
<ChevronDown className="h-4 w-4 opacity-50" />
</button>
</DropdownMenuTrigger>
<DropdownMenuContent
className="max-h-96 overflow-auto"
align="start"
onCloseAutoFocus={(e) => e.preventDefault()}
>
{options.map((option) => (
<MultiSelectCheckboxItem
key={option.value}
checked={value.includes(option.value)}
onSelect={() => handleSelect(option.value)}
onCheckedChange={() => handleSelect(option.value)}
>
{option.label}
</MultiSelectCheckboxItem>
))}
</DropdownMenuContent>
</DropdownMenu>
);
}
+28
View File
@@ -0,0 +1,28 @@
import * as PopoverPrimitive from '@radix-ui/react-popover';
import * as React from 'react';
import { cn } from '@/lib/utils/cn';
const Popover = PopoverPrimitive.Root;
const PopoverTrigger = PopoverPrimitive.Trigger;
const PopoverContent = React.forwardRef<
React.ElementRef<typeof PopoverPrimitive.Content>,
React.ComponentPropsWithoutRef<typeof PopoverPrimitive.Content>
>(({ className, align = 'center', sideOffset = 4, ...props }, ref) => (
<PopoverPrimitive.Portal>
<PopoverPrimitive.Content
ref={ref}
align={align}
sideOffset={sideOffset}
className={cn(
'z-50 w-72 rounded-md border bg-popover p-4 text-popover-foreground shadow-md outline-none data-[state=open]:animate-in data-[state=closed]:animate-out data-[state=closed]:fade-out-0 data-[state=open]:fade-in-0 data-[state=closed]:zoom-out-95 data-[state=open]:zoom-in-95 data-[side=bottom]:slide-in-from-top-2 data-[side=left]:slide-in-from-right-2 data-[side=right]:slide-in-from-left-2 data-[side=top]:slide-in-from-bottom-2',
className,
)}
{...props}
/>
</PopoverPrimitive.Portal>
));
PopoverContent.displayName = PopoverPrimitive.Content.displayName;
export { Popover, PopoverTrigger, PopoverContent };
+1 -1
View File
@@ -12,7 +12,7 @@ const Progress = React.forwardRef<
{...props}
>
<ProgressPrimitive.Indicator
className="h-full w-full flex-1 bg-primary transition-all"
className="h-full w-full flex-1 bg-accent transition-all"
style={{ transform: `translateX(-${100 - (value || 0)}%)` }}
/>
</ProgressPrimitive.Root>
+1 -1
View File
@@ -15,7 +15,7 @@ export function Toaster() {
<ToastProvider>
{toasts.map(({ id, title, description, action, ...props }) => (
<Toast key={id} {...props}>
<div className="grid gap-1">
<div className="grid gap-1 flex-1 min-w-0">
{title && <ToastTitle>{title}</ToastTitle>}
{description && <ToastDescription>{description}</ToastDescription>}
</div>
+3
View File
@@ -0,0 +1,3 @@
interface Window {
__voiceboxServerStartedByApp?: boolean;
}
+77 -12
View File
@@ -1,6 +1,6 @@
import { useEffect, useState } from 'react';
import { check, type Update } from '@tauri-apps/plugin-updater';
import { relaunch } from '@tauri-apps/plugin-process';
import { check, type Update } from '@tauri-apps/plugin-updater';
import { useCallback, useEffect, useState } from 'react';
export interface UpdateStatus {
checking: boolean;
@@ -8,9 +8,18 @@ export interface UpdateStatus {
version?: string;
downloading: boolean;
installing: boolean;
readyToInstall: boolean;
error?: string;
downloadProgress?: number; // 0-100 percentage
downloadedBytes?: number;
totalBytes?: number;
}
// Check if we're on Windows (NSIS installer handles restart automatically)
const isWindows = () => {
return navigator.userAgent.includes('Windows');
};
const isTauri = () => {
return '__TAURI_INTERNALS__' in window;
};
@@ -21,11 +30,12 @@ export function useAutoUpdater(checkOnMount = false) {
available: false,
downloading: false,
installing: false,
readyToInstall: false,
});
const [update, setUpdate] = useState<Update | null>(null);
const checkForUpdates = async () => {
const checkForUpdates = useCallback(async () => {
if (!isTauri()) {
return;
}
@@ -43,6 +53,7 @@ export function useAutoUpdater(checkOnMount = false) {
version: foundUpdate.version,
downloading: false,
installing: false,
readyToInstall: false,
});
} else {
setStatus({
@@ -50,6 +61,7 @@ export function useAutoUpdater(checkOnMount = false) {
available: false,
downloading: false,
installing: false,
readyToInstall: false,
});
}
} catch (error) {
@@ -58,41 +70,93 @@ export function useAutoUpdater(checkOnMount = false) {
available: false,
downloading: false,
installing: false,
readyToInstall: false,
error: error instanceof Error ? error.message : 'Failed to check for updates',
});
}
};
}, []);
// Download the update (but don't install yet)
const downloadAndInstall = async () => {
if (!update || !isTauri()) return;
try {
setStatus((prev) => ({ ...prev, downloading: true, error: undefined }));
await update.downloadAndInstall((event) => {
let downloadedBytes = 0;
let totalBytes = 0;
// Just download the update
await update.download((event) => {
switch (event.event) {
case 'Started':
setStatus((prev) => ({ ...prev, downloading: true }));
totalBytes = event.data.contentLength || 0;
downloadedBytes = 0;
setStatus((prev) => ({
...prev,
downloading: true,
totalBytes,
downloadedBytes: 0,
downloadProgress: 0,
}));
break;
case 'Progress':
console.log(`Downloaded ${event.data.chunkLength} bytes`);
case 'Progress': {
downloadedBytes += event.data.chunkLength;
const progress =
totalBytes > 0 ? Math.round((downloadedBytes / totalBytes) * 100) : undefined;
setStatus((prev) => ({
...prev,
downloadedBytes,
downloadProgress: progress,
}));
break;
}
case 'Finished':
setStatus((prev) => ({
...prev,
downloading: false,
installing: true,
readyToInstall: true,
downloadProgress: 100,
}));
break;
}
});
await relaunch();
} catch (error) {
setStatus((prev) => ({
...prev,
downloading: false,
installing: false,
readyToInstall: false,
downloadProgress: undefined,
downloadedBytes: undefined,
totalBytes: undefined,
error: error instanceof Error ? error.message : 'Failed to download update',
}));
}
};
// Install the downloaded update and restart the app
const restartAndInstall = async () => {
if (!update || !isTauri()) return;
try {
setStatus((prev) => ({ ...prev, installing: true, error: undefined }));
// Install the update
await update.install();
// On Windows with NSIS, the installer handles the restart automatically.
// The process will be killed by the NSIS installer, so we won't reach here.
// On macOS/Linux, we need to manually relaunch.
if (!isWindows()) {
await relaunch();
}
// If we're on Windows and somehow still running, the NSIS installer
// should have already handled everything. Just wait for the process to end.
} catch (error) {
setStatus((prev) => ({
...prev,
installing: false,
error: error instanceof Error ? error.message : 'Failed to install update',
}));
}
@@ -102,11 +166,12 @@ export function useAutoUpdater(checkOnMount = false) {
if (checkOnMount && isTauri()) {
checkForUpdates();
}
}, [checkOnMount]);
}, [checkOnMount, checkForUpdates]);
return {
status,
checkForUpdates,
downloadAndInstall,
restartAndInstall,
};
}
+29
View File
@@ -126,3 +126,32 @@
letter-spacing: 0.1em;
}
}
@keyframes fadeInScale {
from {
opacity: 0;
transform: scale(0.8);
}
to {
opacity: 1;
transform: scale(1);
}
}
@keyframes fadeIn {
from {
opacity: 0;
}
to {
opacity: 1;
}
}
.animate-fade-in-scale {
animation: fadeInScale 0.5s ease-out forwards;
}
.animate-fade-in-delayed {
animation: fadeIn 0.5s ease-out 0.15s forwards;
opacity: 0;
}
+325 -1
View File
@@ -1,4 +1,5 @@
import { useServerStore } from '@/stores/serverStore';
import type { LanguageCode } from '@/lib/constants/languages';
import type {
VoiceProfileCreate,
VoiceProfileResponse,
@@ -12,6 +13,17 @@ import type {
HealthResponse,
ModelStatusListResponse,
ModelDownloadRequest,
ActiveTasksResponse,
StoryCreate,
StoryResponse,
StoryDetailResponse,
StoryItemCreate,
StoryItemDetail,
StoryItemBatchUpdate,
StoryItemReorder,
StoryItemMove,
StoryItemTrim,
StoryItemSplit,
} from './types';
class ApiClient {
@@ -109,6 +121,76 @@ class ApiClient {
});
}
async updateProfileSample(
sampleId: string,
referenceText: string,
): Promise<ProfileSampleResponse> {
return this.request<ProfileSampleResponse>(`/profiles/samples/${sampleId}`, {
method: 'PUT',
body: JSON.stringify({ reference_text: referenceText }),
});
}
async exportProfile(profileId: string): Promise<Blob> {
const url = `${this.getBaseUrl()}/profiles/${profileId}/export`;
const response = await fetch(url);
if (!response.ok) {
const error = await response.json().catch(() => ({
detail: response.statusText,
}));
throw new Error(error.detail || `HTTP error! status: ${response.status}`);
}
return response.blob();
}
async importProfile(file: File): Promise<VoiceProfileResponse> {
const url = `${this.getBaseUrl()}/profiles/import`;
const formData = new FormData();
formData.append('file', file);
const response = await fetch(url, {
method: 'POST',
body: formData,
});
if (!response.ok) {
const error = await response.json().catch(() => ({
detail: response.statusText,
}));
throw new Error(error.detail || `HTTP error! status: ${response.status}`);
}
return response.json();
}
async uploadAvatar(profileId: string, file: File): Promise<VoiceProfileResponse> {
const url = `${this.getBaseUrl()}/profiles/${profileId}/avatar`;
const formData = new FormData();
formData.append('file', file);
const response = await fetch(url, {
method: 'POST',
body: formData,
});
if (!response.ok) {
const error = await response.json().catch(() => ({
detail: response.statusText,
}));
throw new Error(error.detail || `HTTP error! status: ${response.status}`);
}
return response.json();
}
async deleteAvatar(profileId: string): Promise<void> {
await this.request<void>(`/profiles/${profileId}/avatar`, {
method: 'DELETE',
});
}
// Generation
async generateSpeech(data: GenerationRequest): Promise<GenerationResponse> {
return this.request<GenerationResponse>('/generate', {
@@ -141,13 +223,65 @@ class ApiClient {
});
}
async exportGeneration(generationId: string): Promise<Blob> {
const url = `${this.getBaseUrl()}/history/${generationId}/export`;
const response = await fetch(url);
if (!response.ok) {
const error = await response.json().catch(() => ({
detail: response.statusText,
}));
throw new Error(error.detail || `HTTP error! status: ${response.status}`);
}
return response.blob();
}
async exportGenerationAudio(generationId: string): Promise<Blob> {
const url = `${this.getBaseUrl()}/history/${generationId}/export-audio`;
const response = await fetch(url);
if (!response.ok) {
const error = await response.json().catch(() => ({
detail: response.statusText,
}));
throw new Error(error.detail || `HTTP error! status: ${response.status}`);
}
return response.blob();
}
async importGeneration(file: File): Promise<{ id: string; profile_id: string; profile_name: string; text: string; message: string }> {
const url = `${this.getBaseUrl()}/history/import`;
const formData = new FormData();
formData.append('file', file);
const response = await fetch(url, {
method: 'POST',
body: formData,
});
if (!response.ok) {
const error = await response.json().catch(() => ({
detail: response.statusText,
}));
throw new Error(error.detail || `HTTP error! status: ${response.status}`);
}
return response.json();
}
// Audio
getAudioUrl(audioId: string): string {
return `${this.getBaseUrl()}/audio/${audioId}`;
}
getSampleUrl(sampleId: string): string {
return `${this.getBaseUrl()}/samples/${sampleId}`;
}
// Transcription
async transcribeAudio(file: File, language?: 'en' | 'zh'): Promise<TranscriptionResponse> {
async transcribeAudio(file: File, language?: LanguageCode): Promise<TranscriptionResponse> {
const formData = new FormData();
formData.append('file', file);
if (language) {
@@ -181,6 +315,196 @@ class ApiClient {
body: JSON.stringify({ model_name: modelName } as ModelDownloadRequest),
});
}
async deleteModel(modelName: string): Promise<{ message: string }> {
return this.request<{ message: string }>(`/models/${modelName}`, {
method: 'DELETE',
});
}
// Task Management
async getActiveTasks(): Promise<ActiveTasksResponse> {
return this.request<ActiveTasksResponse>('/tasks/active');
}
// Audio Channels
async listChannels(): Promise<
Array<{
id: string;
name: string;
is_default: boolean;
device_ids: string[];
created_at: string;
}>
> {
return this.request('/channels');
}
async createChannel(data: {
name: string;
device_ids: string[];
}): Promise<{
id: string;
name: string;
is_default: boolean;
device_ids: string[];
created_at: string;
}> {
return this.request('/channels', {
method: 'POST',
body: JSON.stringify(data),
});
}
async updateChannel(
channelId: string,
data: {
name?: string;
device_ids?: string[];
},
): Promise<{
id: string;
name: string;
is_default: boolean;
device_ids: string[];
created_at: string;
}> {
return this.request(`/channels/${channelId}`, {
method: 'PUT',
body: JSON.stringify(data),
});
}
async deleteChannel(channelId: string): Promise<{ message: string }> {
return this.request(`/channels/${channelId}`, {
method: 'DELETE',
});
}
async getChannelVoices(channelId: string): Promise<{ profile_ids: string[] }> {
return this.request(`/channels/${channelId}/voices`);
}
async setChannelVoices(
channelId: string,
profileIds: string[],
): Promise<{ message: string }> {
return this.request(`/channels/${channelId}/voices`, {
method: 'PUT',
body: JSON.stringify({ profile_ids: profileIds }),
});
}
async getProfileChannels(profileId: string): Promise<{ channel_ids: string[] }> {
return this.request(`/profiles/${profileId}/channels`);
}
async setProfileChannels(
profileId: string,
channelIds: string[],
): Promise<{ message: string }> {
return this.request(`/profiles/${profileId}/channels`, {
method: 'PUT',
body: JSON.stringify({ channel_ids: channelIds }),
});
}
// Stories
async listStories(): Promise<StoryResponse[]> {
return this.request<StoryResponse[]>('/stories');
}
async createStory(data: StoryCreate): Promise<StoryResponse> {
return this.request<StoryResponse>('/stories', {
method: 'POST',
body: JSON.stringify(data),
});
}
async getStory(storyId: string): Promise<StoryDetailResponse> {
return this.request<StoryDetailResponse>(`/stories/${storyId}`);
}
async updateStory(storyId: string, data: StoryCreate): Promise<StoryResponse> {
return this.request<StoryResponse>(`/stories/${storyId}`, {
method: 'PUT',
body: JSON.stringify(data),
});
}
async deleteStory(storyId: string): Promise<void> {
await this.request<void>(`/stories/${storyId}`, {
method: 'DELETE',
});
}
async addStoryItem(storyId: string, data: StoryItemCreate): Promise<StoryItemDetail> {
return this.request<StoryItemDetail>(`/stories/${storyId}/items`, {
method: 'POST',
body: JSON.stringify(data),
});
}
async removeStoryItem(storyId: string, itemId: string): Promise<void> {
await this.request<void>(`/stories/${storyId}/items/${itemId}`, {
method: 'DELETE',
});
}
async updateStoryItemTimes(storyId: string, data: StoryItemBatchUpdate): Promise<void> {
await this.request<void>(`/stories/${storyId}/items/times`, {
method: 'PUT',
body: JSON.stringify(data),
});
}
async reorderStoryItems(storyId: string, data: StoryItemReorder): Promise<StoryItemDetail[]> {
return this.request<StoryItemDetail[]>(`/stories/${storyId}/items/reorder`, {
method: 'PUT',
body: JSON.stringify(data),
});
}
async moveStoryItem(storyId: string, itemId: string, data: StoryItemMove): Promise<StoryItemDetail> {
return this.request<StoryItemDetail>(`/stories/${storyId}/items/${itemId}/move`, {
method: 'PUT',
body: JSON.stringify(data),
});
}
async trimStoryItem(storyId: string, itemId: string, data: StoryItemTrim): Promise<StoryItemDetail> {
return this.request<StoryItemDetail>(`/stories/${storyId}/items/${itemId}/trim`, {
method: 'PUT',
body: JSON.stringify(data),
});
}
async splitStoryItem(storyId: string, itemId: string, data: StoryItemSplit): Promise<StoryItemDetail[]> {
return this.request<StoryItemDetail[]>(`/stories/${storyId}/items/${itemId}/split`, {
method: 'POST',
body: JSON.stringify(data),
});
}
async duplicateStoryItem(storyId: string, itemId: string): Promise<StoryItemDetail> {
return this.request<StoryItemDetail>(`/stories/${storyId}/items/${itemId}/duplicate`, {
method: 'POST',
});
}
async exportStoryAudio(storyId: string): Promise<Blob> {
const url = `${this.getBaseUrl()}/stories/${storyId}/export-audio`;
const response = await fetch(url);
if (!response.ok) {
const error = await response.json().catch(() => ({
detail: response.statusText,
}));
throw new Error(error.detail || `HTTP error! status: ${response.status}`);
}
return response.blob();
}
}
export const apiClient = new ApiClient();
+99 -3
View File
@@ -1,9 +1,10 @@
// API Types matching backend Pydantic models
import type { LanguageCode } from '@/lib/constants/languages';
export interface VoiceProfileCreate {
name: string;
description?: string;
language: 'en' | 'zh';
language: LanguageCode;
}
export interface VoiceProfileResponse {
@@ -11,6 +12,7 @@ export interface VoiceProfileResponse {
name: string;
description?: string;
language: string;
avatar_path?: string;
created_at: string;
updated_at: string;
}
@@ -29,7 +31,7 @@ export interface ProfileSampleResponse {
export interface GenerationRequest {
profile_id: string;
text: string;
language: 'en' | 'zh';
language: LanguageCode;
seed?: number;
model_size?: '1.7B' | '0.6B';
}
@@ -62,7 +64,7 @@ export interface HistoryListResponse {
}
export interface TranscriptionRequest {
language?: 'en' | 'zh';
language?: LanguageCode;
}
export interface TranscriptionResponse {
@@ -105,3 +107,97 @@ export interface ModelStatusListResponse {
export interface ModelDownloadRequest {
model_name: string;
}
export interface ActiveDownloadTask {
model_name: string;
status: string;
started_at: string;
}
export interface ActiveGenerationTask {
task_id: string;
profile_id: string;
text_preview: string;
started_at: string;
}
export interface ActiveTasksResponse {
downloads: ActiveDownloadTask[];
generations: ActiveGenerationTask[];
}
export interface StoryCreate {
name: string;
description?: string;
}
export interface StoryResponse {
id: string;
name: string;
description?: string;
created_at: string;
updated_at: string;
item_count: number;
}
export interface StoryItemDetail {
id: string;
story_id: string;
generation_id: string;
start_time_ms: number;
track: number;
trim_start_ms: number;
trim_end_ms: number;
created_at: string;
profile_id: string;
profile_name: string;
text: string;
language: string;
audio_path: string;
duration: number;
seed?: number;
instruct?: string;
generation_created_at: string;
}
export interface StoryDetailResponse {
id: string;
name: string;
description?: string;
created_at: string;
updated_at: string;
items: StoryItemDetail[];
}
export interface StoryItemCreate {
generation_id: string;
start_time_ms?: number;
track?: number;
}
export interface StoryItemUpdateTime {
generation_id: string;
start_time_ms: number;
}
export interface StoryItemBatchUpdate {
updates: StoryItemUpdateTime[];
}
export interface StoryItemReorder {
generation_ids: string[];
}
export interface StoryItemMove {
start_time_ms: number;
track: number;
}
export interface StoryItemTrim {
trim_start_ms: number;
trim_end_ms: number;
}
export interface StoryItemSplit {
split_time_ms: number;
}
+26
View File
@@ -0,0 +1,26 @@
/**
* Supported languages for Qwen3-TTS
* Based on: https://github.com/QwenLM/Qwen3-TTS
*/
export const SUPPORTED_LANGUAGES = {
zh: 'Chinese',
en: 'English',
ja: 'Japanese',
ko: 'Korean',
de: 'German',
fr: 'French',
ru: 'Russian',
pt: 'Portuguese',
es: 'Spanish',
it: 'Italian',
} as const;
export type LanguageCode = keyof typeof SUPPORTED_LANGUAGES;
export const LANGUAGE_CODES = Object.keys(SUPPORTED_LANGUAGES) as LanguageCode[];
export const LANGUAGE_OPTIONS = LANGUAGE_CODES.map((code) => ({
value: code,
label: SUPPORTED_LANGUAGES[code],
}));
+15
View File
@@ -0,0 +1,15 @@
/**
* UI layout constants for safe area padding
*/
/**
* Top safe area padding - height of the drag region bar
* Corresponds to Tailwind's pt-12 (3rem / 48px)
*/
export const TOP_SAFE_AREA_PADDING = 'pt-12';
/**
* Bottom safe area padding - height of the audio player
* Corresponds to Tailwind's pb-32 (8rem / 128px)
*/
export const BOTTOM_SAFE_AREA_PADDING = 'pb-32';
-1
View File
@@ -1 +0,0 @@
# React Query hooks will be placed here
+66
View File
@@ -0,0 +1,66 @@
import { useRef, useState } from 'react';
import { useToast } from '@/components/ui/use-toast';
export function useAudioPlayer() {
const [isPlaying, setIsPlaying] = useState(false);
const audioRef = useRef<HTMLAudioElement | null>(null);
const { toast } = useToast();
const playPause = (file: File | null | undefined) => {
if (!file) return;
if (audioRef.current) {
if (isPlaying) {
audioRef.current.pause();
setIsPlaying(false);
} else {
audioRef.current.play();
setIsPlaying(true);
}
} else {
const audio = new Audio(URL.createObjectURL(file));
audioRef.current = audio;
audio.addEventListener('ended', () => {
setIsPlaying(false);
if (audioRef.current) {
URL.revokeObjectURL(audioRef.current.src);
}
audioRef.current = null;
});
audio.addEventListener('error', () => {
setIsPlaying(false);
toast({
title: 'Playback error',
description: 'Failed to play audio file',
variant: 'destructive',
});
if (audioRef.current) {
URL.revokeObjectURL(audioRef.current.src);
}
audioRef.current = null;
});
audio.play();
setIsPlaying(true);
}
};
const cleanup = () => {
if (audioRef.current) {
audioRef.current.pause();
if (audioRef.current.src.startsWith('blob:')) {
URL.revokeObjectURL(audioRef.current.src);
}
audioRef.current = null;
}
setIsPlaying(false);
};
return {
isPlaying,
playPause,
cleanup,
};
}
+24 -6
View File
@@ -1,13 +1,14 @@
import { useState, useRef, useCallback, useEffect } from 'react';
import { useCallback, useEffect, useRef, useState } from 'react';
import { isTauri } from '@/lib/tauri';
import { convertToWav } from '@/lib/utils/audio';
interface UseAudioRecordingOptions {
maxDurationSeconds?: number;
onRecordingComplete?: (blob: Blob) => void;
onRecordingComplete?: (blob: Blob, duration?: number) => void;
}
export function useAudioRecording({
maxDurationSeconds = 30,
maxDurationSeconds = 29,
onRecordingComplete,
}: UseAudioRecordingOptions = {}) {
const [isRecording, setIsRecording] = useState(false);
@@ -85,9 +86,26 @@ export function useAudioRecording({
}
};
mediaRecorder.onstop = () => {
const blob = new Blob(chunksRef.current, { type: 'audio/webm' });
onRecordingComplete?.(blob);
mediaRecorder.onstop = async () => {
const webmBlob = new Blob(chunksRef.current, { type: 'audio/webm' });
// Convert to WAV format to avoid needing ffmpeg on backend
try {
const wavBlob = await convertToWav(webmBlob);
// Pass the actual recorded duration
const recordedDuration = startTimeRef.current
? (Date.now() - startTimeRef.current) / 1000
: undefined;
onRecordingComplete?.(wavBlob, recordedDuration);
} catch (err) {
console.error('Error converting audio to WAV:', err);
// Fallback to original blob if conversion fails
const recordedDuration = startTimeRef.current
? (Date.now() - startTimeRef.current) / 1000
: undefined;
onRecordingComplete?.(webmBlob, recordedDuration);
}
// Stop all tracks
streamRef.current?.getTracks().forEach((track) => {
+122
View File
@@ -0,0 +1,122 @@
import { zodResolver } from '@hookform/resolvers/zod';
import { useState } from 'react';
import { useForm } from 'react-hook-form';
import * as z from 'zod';
import { useToast } from '@/components/ui/use-toast';
import { apiClient } from '@/lib/api/client';
import { LANGUAGE_CODES, type LanguageCode } from '@/lib/constants/languages';
import { useGeneration } from '@/lib/hooks/useGeneration';
import { useModelDownloadToast } from '@/lib/hooks/useModelDownloadToast';
import { useGenerationStore } from '@/stores/generationStore';
import { usePlayerStore } from '@/stores/playerStore';
const generationSchema = z.object({
text: z.string().min(1, 'Text is required').max(5000),
language: z.enum(LANGUAGE_CODES as [LanguageCode, ...LanguageCode[]]),
seed: z.number().int().optional(),
modelSize: z.enum(['1.7B', '0.6B']).optional(),
instruct: z.string().max(500).optional(),
});
export type GenerationFormValues = z.infer<typeof generationSchema>;
interface UseGenerationFormOptions {
onSuccess?: (generationId: string) => void;
defaultValues?: Partial<GenerationFormValues>;
}
export function useGenerationForm(options: UseGenerationFormOptions = {}) {
const { toast } = useToast();
const generation = useGeneration();
const setAudioWithAutoPlay = usePlayerStore((state) => state.setAudioWithAutoPlay);
const setIsGenerating = useGenerationStore((state) => state.setIsGenerating);
const [downloadingModelName, setDownloadingModelName] = useState<string | null>(null);
const [downloadingDisplayName, setDownloadingDisplayName] = useState<string | null>(null);
useModelDownloadToast({
modelName: downloadingModelName || '',
displayName: downloadingDisplayName || '',
enabled: !!downloadingModelName,
});
const form = useForm<GenerationFormValues>({
resolver: zodResolver(generationSchema),
defaultValues: {
text: '',
language: 'en',
seed: undefined,
modelSize: '1.7B',
instruct: '',
...options.defaultValues,
},
});
async function handleSubmit(
data: GenerationFormValues,
selectedProfileId: string | null,
): Promise<void> {
if (!selectedProfileId) {
toast({
title: 'No profile selected',
description: 'Please select a voice profile from the cards above.',
variant: 'destructive',
});
return;
}
try {
setIsGenerating(true);
const modelName = `qwen-tts-${data.modelSize}`;
const displayName = data.modelSize === '1.7B' ? 'Qwen TTS 1.7B' : 'Qwen TTS 0.6B';
try {
const modelStatus = await apiClient.getModelStatus();
const model = modelStatus.models.find((m) => m.model_name === modelName);
if (model && !model.downloaded) {
setDownloadingModelName(modelName);
setDownloadingDisplayName(displayName);
}
} catch (error) {
console.error('Failed to check model status:', error);
}
const result = await generation.mutateAsync({
profile_id: selectedProfileId,
text: data.text,
language: data.language,
seed: data.seed,
model_size: data.modelSize,
instruct: data.instruct || undefined,
});
toast({
title: 'Generation complete!',
description: `Audio generated (${result.duration.toFixed(2)}s)`,
});
const audioUrl = apiClient.getAudioUrl(result.id);
setAudioWithAutoPlay(audioUrl, result.id, selectedProfileId, data.text.substring(0, 50));
form.reset();
options.onSuccess?.(result.id);
} catch (error) {
toast({
title: 'Generation failed',
description: error instanceof Error ? error.message : 'Failed to generate audio',
variant: 'destructive',
});
} finally {
setIsGenerating(false);
setDownloadingModelName(null);
setDownloadingDisplayName(null);
}
}
return {
form,
handleSubmit,
isPending: generation.isPending,
};
}
+128
View File
@@ -1,6 +1,7 @@
import { useMutation, useQuery, useQueryClient } from '@tanstack/react-query';
import { apiClient } from '@/lib/api/client';
import type { HistoryQuery } from '@/lib/api/types';
import { isTauri } from '@/lib/tauri';
export function useHistory(query?: HistoryQuery) {
return useQuery({
@@ -27,3 +28,130 @@ export function useDeleteGeneration() {
},
});
}
export function useExportGeneration() {
return useMutation({
mutationFn: async ({ generationId, text }: { generationId: string; text: string }) => {
const blob = await apiClient.exportGeneration(generationId);
// Create safe filename from text
const safeText = text.substring(0, 30).replace(/[^a-z0-9]/gi, '-').toLowerCase();
const filename = `generation-${safeText}.voicebox.zip`;
if (isTauri()) {
// Use Tauri's native save dialog
try {
const { save } = await import('@tauri-apps/plugin-dialog');
const filePath = await save({
defaultPath: filename,
filters: [
{
name: 'Voicebox Generation',
extensions: ['voicebox.zip', 'zip'],
},
],
});
if (filePath) {
// Write file using Tauri's filesystem API
const { writeBinaryFile } = await import('@tauri-apps/plugin-fs');
const arrayBuffer = await blob.arrayBuffer();
await writeBinaryFile(filePath, new Uint8Array(arrayBuffer));
}
} catch (error) {
console.error('Failed to use Tauri dialog, falling back to browser download:', error);
// Fall back to browser download if Tauri dialog fails
const url = window.URL.createObjectURL(blob);
const a = document.createElement('a');
a.href = url;
a.download = filename;
document.body.appendChild(a);
a.click();
window.URL.revokeObjectURL(url);
document.body.removeChild(a);
}
} else {
// Browser: trigger download
const url = window.URL.createObjectURL(blob);
const a = document.createElement('a');
a.href = url;
a.download = filename;
document.body.appendChild(a);
a.click();
window.URL.revokeObjectURL(url);
document.body.removeChild(a);
}
return blob;
},
});
}
export function useExportGenerationAudio() {
return useMutation({
mutationFn: async ({ generationId, text }: { generationId: string; text: string }) => {
const blob = await apiClient.exportGenerationAudio(generationId);
// Create safe filename from text
const safeText = text.substring(0, 30).replace(/[^a-z0-9]/gi, '-').toLowerCase();
const filename = `${safeText}.wav`;
if (isTauri()) {
// Use Tauri's native save dialog
try {
const { save } = await import('@tauri-apps/plugin-dialog');
const filePath = await save({
defaultPath: filename,
filters: [
{
name: 'Audio File',
extensions: ['wav'],
},
],
});
if (filePath) {
// Write file using Tauri's filesystem API
const { writeBinaryFile } = await import('@tauri-apps/plugin-fs');
const arrayBuffer = await blob.arrayBuffer();
await writeBinaryFile(filePath, new Uint8Array(arrayBuffer));
}
} catch (error) {
console.error('Failed to use Tauri dialog, falling back to browser download:', error);
// Fall back to browser download if Tauri dialog fails
const url = window.URL.createObjectURL(blob);
const a = document.createElement('a');
a.href = url;
a.download = filename;
document.body.appendChild(a);
a.click();
window.URL.revokeObjectURL(url);
document.body.removeChild(a);
}
} else {
// Browser: trigger download
const url = window.URL.createObjectURL(blob);
const a = document.createElement('a');
a.href = url;
a.download = filename;
document.body.appendChild(a);
a.click();
window.URL.revokeObjectURL(url);
document.body.removeChild(a);
}
return blob;
},
});
}
export function useImportGeneration() {
const queryClient = useQueryClient();
return useMutation({
mutationFn: (file: File) => apiClient.importGeneration(file),
onSuccess: () => {
queryClient.invalidateQueries({ queryKey: ['history'] });
},
});
}
+176
View File
@@ -0,0 +1,176 @@
import { useEffect, useRef } from 'react';
import { useToast } from '@/components/ui/use-toast';
import { useServerStore } from '@/stores/serverStore';
import { Progress } from '@/components/ui/progress';
import { Loader2, CheckCircle2, XCircle } from 'lucide-react';
import type { ModelProgress } from '@/lib/api/types';
interface UseModelDownloadToastOptions {
modelName: string;
displayName: string;
enabled?: boolean;
}
/**
* Hook to show and update a toast notification with model download progress.
* Subscribes to Server-Sent Events for real-time progress updates.
*/
export function useModelDownloadToast({
modelName,
displayName,
enabled = false,
}: UseModelDownloadToastOptions) {
const { toast } = useToast();
const serverUrl = useServerStore((state) => state.serverUrl);
const toastIdRef = useRef<string | null>(null);
const toastUpdateRef = useRef<
((props: {
title?: React.ReactNode;
description?: React.ReactNode;
duration?: number;
variant?: 'default' | 'destructive';
open?: boolean;
}) => void) | null
>(null);
const eventSourceRef = useRef<EventSource | null>(null);
const formatBytes = (bytes: number): string => {
if (bytes === 0) return '0 B';
const k = 1024;
const sizes = ['B', 'KB', 'MB', 'GB'];
const i = Math.floor(Math.log(bytes) / Math.log(k));
return `${(bytes / Math.pow(k, i)).toFixed(1)} ${sizes[i]}`;
};
useEffect(() => {
if (!enabled || !serverUrl || !modelName) {
return;
}
// Create initial toast
const toastResult = toast({
title: displayName,
description: 'Starting download...',
duration: Infinity, // Don't auto-dismiss, we'll handle it manually
});
toastIdRef.current = toastResult.id;
toastUpdateRef.current = toastResult.update;
// Subscribe to progress updates via Server-Sent Events
const eventSource = new EventSource(`${serverUrl}/models/progress/${modelName}`);
eventSource.onmessage = (event) => {
try {
const progress = JSON.parse(event.data) as ModelProgress;
// Update toast with progress
if (toastIdRef.current && toastUpdateRef.current) {
const progressPercent = progress.total > 0 ? progress.progress : 0;
const progressText =
progress.total > 0
? `${formatBytes(progress.current)} / ${formatBytes(progress.total)} (${progress.progress.toFixed(1)}%)`
: '';
// Determine status icon and text
let statusIcon: React.ReactNode = null;
let statusText = 'Processing...';
switch (progress.status) {
case 'complete':
statusIcon = <CheckCircle2 className="h-4 w-4 text-green-500" />;
statusText = 'Download complete';
break;
case 'error':
statusIcon = <XCircle className="h-4 w-4 text-destructive" />;
statusText = `Error: ${progress.error || 'Unknown error'}`;
break;
case 'downloading':
statusIcon = <Loader2 className="h-4 w-4 animate-spin" />;
statusText = progress.filename ? `Downloading ${progress.filename}...` : 'Downloading...';
break;
case 'extracting':
statusIcon = <Loader2 className="h-4 w-4 animate-spin" />;
statusText = 'Extracting...';
break;
}
toastUpdateRef.current({
title: (
<div className="flex items-center gap-2">
{statusIcon}
<span>{displayName}</span>
</div>
),
description: (
<div className="space-y-2">
<div className="text-sm">{statusText}</div>
{progress.total > 0 && (
<>
<Progress value={progressPercent} className="h-2" />
<div className="text-xs text-muted-foreground">{progressText}</div>
</>
)}
</div>
),
duration: progress.status === 'complete' ? 5000 : Infinity,
variant: progress.status === 'error' ? 'destructive' : 'default',
});
// Close connection and dismiss toast on completion or error
if (progress.status === 'complete' || progress.status === 'error') {
eventSource.close();
eventSourceRef.current = null;
// Auto-dismiss on completion after delay
if (progress.status === 'complete') {
setTimeout(() => {
if (toastIdRef.current && toastUpdateRef.current) {
toastUpdateRef.current({
open: false,
});
toastIdRef.current = null;
toastUpdateRef.current = null;
}
}, 5000);
}
}
}
} catch (error) {
console.error('Error parsing progress event:', error);
}
};
eventSource.onerror = (error) => {
console.error('SSE error:', error);
eventSource.close();
eventSourceRef.current = null;
// Show error toast
if (toastIdRef.current && toastUpdateRef.current) {
toastUpdateRef.current({
title: displayName,
description: 'Failed to track download progress',
variant: 'destructive',
duration: 5000,
});
toastIdRef.current = null;
toastUpdateRef.current = null;
}
};
eventSourceRef.current = eventSource;
// Cleanup on unmount or when disabled
return () => {
if (eventSourceRef.current) {
eventSourceRef.current.close();
eventSourceRef.current = null;
}
// Note: We don't dismiss the toast here as it might still be showing completion state
};
}, [enabled, serverUrl, modelName, displayName, toast]);
return {
isTracking: enabled && eventSourceRef.current !== null,
};
}
+118
View File
@@ -1,6 +1,7 @@
import { useMutation, useQuery, useQueryClient } from '@tanstack/react-query';
import { apiClient } from '@/lib/api/client';
import type { VoiceProfileCreate } from '@/lib/api/types';
import { isTauri } from '@/lib/tauri';
export function useProfiles() {
return useQuery({
@@ -96,3 +97,120 @@ export function useDeleteSample() {
},
});
}
export function useUpdateSample() {
const queryClient = useQueryClient();
return useMutation({
mutationFn: ({ sampleId, referenceText }: { sampleId: string; referenceText: string }) =>
apiClient.updateProfileSample(sampleId, referenceText),
onSuccess: (data) => {
queryClient.invalidateQueries({
queryKey: ['profiles', data.profile_id, 'samples'],
});
queryClient.invalidateQueries({
queryKey: ['profiles', data.profile_id],
});
queryClient.invalidateQueries({ queryKey: ['profiles'] });
},
});
}
export function useExportProfile() {
return useMutation({
mutationFn: async (profileId: string) => {
const blob = await apiClient.exportProfile(profileId);
// Get profile name for filename
const profile = await apiClient.getProfile(profileId);
const safeName = profile.name.replace(/[^a-z0-9]/gi, '-').toLowerCase();
const filename = `profile-${safeName}.voicebox.zip`;
if (isTauri()) {
// Use Tauri's native save dialog
try {
const { save } = await import('@tauri-apps/plugin-dialog');
const filePath = await save({
defaultPath: filename,
filters: [
{
name: 'Voicebox Profile',
extensions: ['voicebox.zip', 'zip'],
},
],
});
if (filePath) {
// Write file using Tauri's filesystem API
const { writeBinaryFile } = await import('@tauri-apps/plugin-fs');
const arrayBuffer = await blob.arrayBuffer();
await writeBinaryFile(filePath, new Uint8Array(arrayBuffer));
}
} catch (error) {
console.error('Failed to use Tauri dialog, falling back to browser download:', error);
// Fall back to browser download if Tauri dialog fails
const url = window.URL.createObjectURL(blob);
const a = document.createElement('a');
a.href = url;
a.download = filename;
document.body.appendChild(a);
a.click();
window.URL.revokeObjectURL(url);
document.body.removeChild(a);
}
} else {
// Browser: trigger download
const url = window.URL.createObjectURL(blob);
const a = document.createElement('a');
a.href = url;
a.download = filename;
document.body.appendChild(a);
a.click();
window.URL.revokeObjectURL(url);
document.body.removeChild(a);
}
return blob;
},
});
}
export function useImportProfile() {
const queryClient = useQueryClient();
return useMutation({
mutationFn: (file: File) => apiClient.importProfile(file),
onSuccess: () => {
queryClient.invalidateQueries({ queryKey: ['profiles'] });
},
});
}
export function useUploadAvatar() {
const queryClient = useQueryClient();
return useMutation({
mutationFn: ({ profileId, file }: { profileId: string; file: File }) =>
apiClient.uploadAvatar(profileId, file),
onSuccess: (_, variables) => {
queryClient.invalidateQueries({ queryKey: ['profiles'] });
queryClient.invalidateQueries({
queryKey: ['profiles', variables.profileId],
});
},
});
}
export function useDeleteAvatar() {
const queryClient = useQueryClient();
return useMutation({
mutationFn: (profileId: string) => apiClient.deleteAvatar(profileId),
onSuccess: (_, profileId) => {
queryClient.invalidateQueries({ queryKey: ['profiles'] });
queryClient.invalidateQueries({
queryKey: ['profiles', profileId],
});
},
});
}
@@ -0,0 +1,87 @@
import { useCallback, useEffect, useRef, useState } from 'react';
import { apiClient } from '@/lib/api/client';
import { useGenerationStore } from '@/stores/generationStore';
import type { ActiveDownloadTask } from '@/lib/api/types';
// Polling interval in milliseconds
const POLL_INTERVAL = 2000;
/**
* Hook to monitor active tasks (downloads and generations).
* Polls the server periodically to catch downloads triggered from anywhere
* (transcription, generation, explicit download, etc.).
*
* Returns the active downloads so components can render download toasts.
*/
export function useRestoreActiveTasks() {
const [activeDownloads, setActiveDownloads] = useState<ActiveDownloadTask[]>([]);
const setIsGenerating = useGenerationStore((state) => state.setIsGenerating);
const setActiveGenerationId = useGenerationStore((state) => state.setActiveGenerationId);
// Track which downloads we've seen to detect new ones
const seenDownloadsRef = useRef<Set<string>>(new Set());
const fetchActiveTasks = useCallback(async () => {
try {
const tasks = await apiClient.getActiveTasks();
// Update generation state
if (tasks.generations.length > 0) {
setIsGenerating(true);
setActiveGenerationId(tasks.generations[0].task_id);
} else {
// Only clear if we were tracking a generation
const currentId = useGenerationStore.getState().activeGenerationId;
if (currentId) {
setIsGenerating(false);
setActiveGenerationId(null);
}
}
// Update active downloads
// Keep track of all active downloads (including new ones)
const currentDownloadNames = new Set(tasks.downloads.map((d) => d.model_name));
// Remove completed downloads from our seen set
for (const name of seenDownloadsRef.current) {
if (!currentDownloadNames.has(name)) {
seenDownloadsRef.current.delete(name);
}
}
// Add new downloads to seen set
for (const download of tasks.downloads) {
seenDownloadsRef.current.add(download.model_name);
}
setActiveDownloads(tasks.downloads);
} catch (error) {
// Silently fail - server might be temporarily unavailable
console.debug('Failed to fetch active tasks:', error);
}
}, [setIsGenerating, setActiveGenerationId]);
useEffect(() => {
// Fetch immediately on mount
fetchActiveTasks();
// Poll for active tasks
const interval = setInterval(fetchActiveTasks, POLL_INTERVAL);
return () => clearInterval(interval);
}, [fetchActiveTasks]);
return activeDownloads;
}
/**
* Map model names to display names for download toasts.
*/
export const MODEL_DISPLAY_NAMES: Record<string, string> = {
'qwen-tts-1.7B': 'Qwen TTS 1.7B',
'qwen-tts-0.6B': 'Qwen TTS 0.6B',
'whisper-base': 'Whisper Base',
'whisper-small': 'Whisper Small',
'whisper-medium': 'Whisper Medium',
'whisper-large': 'Whisper Large',
};
+216
View File
@@ -0,0 +1,216 @@
import { useMutation, useQuery, useQueryClient } from '@tanstack/react-query';
import { apiClient } from '@/lib/api/client';
import type { StoryCreate, StoryItemCreate, StoryItemBatchUpdate, StoryItemReorder, StoryItemMove, StoryItemTrim, StoryItemSplit } from '@/lib/api/types';
import { isTauri } from '@/lib/tauri';
export function useStories() {
return useQuery({
queryKey: ['stories'],
queryFn: () => apiClient.listStories(),
});
}
export function useStory(storyId: string | null) {
return useQuery({
queryKey: ['stories', storyId],
queryFn: () => apiClient.getStory(storyId!),
enabled: !!storyId,
});
}
export function useCreateStory() {
const queryClient = useQueryClient();
return useMutation({
mutationFn: (data: StoryCreate) => apiClient.createStory(data),
onSuccess: () => {
queryClient.invalidateQueries({ queryKey: ['stories'] });
},
});
}
export function useUpdateStory() {
const queryClient = useQueryClient();
return useMutation({
mutationFn: ({ storyId, data }: { storyId: string; data: StoryCreate }) =>
apiClient.updateStory(storyId, data),
onSuccess: (_, variables) => {
queryClient.invalidateQueries({ queryKey: ['stories'] });
queryClient.invalidateQueries({ queryKey: ['stories', variables.storyId] });
},
});
}
export function useDeleteStory() {
const queryClient = useQueryClient();
return useMutation({
mutationFn: (storyId: string) => apiClient.deleteStory(storyId),
onSuccess: () => {
queryClient.invalidateQueries({ queryKey: ['stories'] });
},
});
}
export function useAddStoryItem() {
const queryClient = useQueryClient();
return useMutation({
mutationFn: ({ storyId, data }: { storyId: string; data: StoryItemCreate }) =>
apiClient.addStoryItem(storyId, data),
onSuccess: (_, variables) => {
queryClient.invalidateQueries({ queryKey: ['stories'] });
queryClient.invalidateQueries({ queryKey: ['stories', variables.storyId] });
},
});
}
export function useRemoveStoryItem() {
const queryClient = useQueryClient();
return useMutation({
mutationFn: ({ storyId, itemId }: { storyId: string; itemId: string }) =>
apiClient.removeStoryItem(storyId, itemId),
onSuccess: (_, variables) => {
queryClient.invalidateQueries({ queryKey: ['stories'] });
queryClient.invalidateQueries({ queryKey: ['stories', variables.storyId] });
},
});
}
export function useUpdateStoryItemTimes() {
const queryClient = useQueryClient();
return useMutation({
mutationFn: ({ storyId, data }: { storyId: string; data: StoryItemBatchUpdate }) =>
apiClient.updateStoryItemTimes(storyId, data),
onSuccess: (_, variables) => {
queryClient.invalidateQueries({ queryKey: ['stories'] });
queryClient.invalidateQueries({ queryKey: ['stories', variables.storyId] });
},
});
}
export function useReorderStoryItems() {
const queryClient = useQueryClient();
return useMutation({
mutationFn: ({ storyId, data }: { storyId: string; data: StoryItemReorder }) =>
apiClient.reorderStoryItems(storyId, data),
onSuccess: (_, variables) => {
queryClient.invalidateQueries({ queryKey: ['stories'] });
queryClient.invalidateQueries({ queryKey: ['stories', variables.storyId] });
},
});
}
export function useMoveStoryItem() {
const queryClient = useQueryClient();
return useMutation({
mutationFn: ({ storyId, itemId, data }: { storyId: string; itemId: string; data: StoryItemMove }) =>
apiClient.moveStoryItem(storyId, itemId, data),
onSuccess: (_, variables) => {
queryClient.invalidateQueries({ queryKey: ['stories'] });
queryClient.invalidateQueries({ queryKey: ['stories', variables.storyId] });
},
});
}
export function useTrimStoryItem() {
const queryClient = useQueryClient();
return useMutation({
mutationFn: ({ storyId, itemId, data }: { storyId: string; itemId: string; data: StoryItemTrim }) =>
apiClient.trimStoryItem(storyId, itemId, data),
onSuccess: (_, variables) => {
queryClient.invalidateQueries({ queryKey: ['stories'] });
queryClient.invalidateQueries({ queryKey: ['stories', variables.storyId] });
},
});
}
export function useSplitStoryItem() {
const queryClient = useQueryClient();
return useMutation({
mutationFn: ({ storyId, itemId, data }: { storyId: string; itemId: string; data: StoryItemSplit }) =>
apiClient.splitStoryItem(storyId, itemId, data),
onSuccess: (_, variables) => {
queryClient.invalidateQueries({ queryKey: ['stories'] });
queryClient.invalidateQueries({ queryKey: ['stories', variables.storyId] });
},
});
}
export function useDuplicateStoryItem() {
const queryClient = useQueryClient();
return useMutation({
mutationFn: ({ storyId, itemId }: { storyId: string; itemId: string }) =>
apiClient.duplicateStoryItem(storyId, itemId),
onSuccess: (_, variables) => {
queryClient.invalidateQueries({ queryKey: ['stories'] });
queryClient.invalidateQueries({ queryKey: ['stories', variables.storyId] });
},
});
}
export function useExportStoryAudio() {
return useMutation({
mutationFn: async ({ storyId, storyName }: { storyId: string; storyName: string }) => {
const blob = await apiClient.exportStoryAudio(storyId);
// Create safe filename
const safeName = storyName.substring(0, 50).replace(/[^a-z0-9]/gi, '-').toLowerCase();
const filename = `${safeName || 'story'}.wav`;
if (isTauri()) {
// Use Tauri's native save dialog
try {
const { save } = await import('@tauri-apps/plugin-dialog');
const filePath = await save({
defaultPath: filename,
filters: [
{
name: 'Audio File',
extensions: ['wav'],
},
],
});
if (filePath) {
// Write file using Tauri's filesystem API
const { writeBinaryFile } = await import('@tauri-apps/plugin-fs');
const arrayBuffer = await blob.arrayBuffer();
await writeBinaryFile(filePath, new Uint8Array(arrayBuffer));
}
} catch (error) {
console.error('Failed to use Tauri dialog, falling back to browser download:', error);
// Fall back to browser download if Tauri dialog fails
const url = window.URL.createObjectURL(blob);
const a = document.createElement('a');
a.href = url;
a.download = filename;
document.body.appendChild(a);
a.click();
window.URL.revokeObjectURL(url);
document.body.removeChild(a);
}
} else {
// Browser: trigger download
const url = window.URL.createObjectURL(blob);
const a = document.createElement('a');
a.href = url;
a.download = filename;
document.body.appendChild(a);
a.click();
window.URL.revokeObjectURL(url);
document.body.removeChild(a);
}
return blob;
},
});
}
+389
View File
@@ -0,0 +1,389 @@
import { useCallback, useEffect, useRef } from 'react';
import { apiClient } from '@/lib/api/client';
import type { StoryItemDetail } from '@/lib/api/types';
import { useStoryStore } from '@/stores/storyStore';
interface ActiveSource {
source: AudioBufferSourceNode;
itemId: string;
generationId: string;
startTimeMs: number;
endTimeMs: number;
}
/**
* Hook for managing timecode-based story playback using Web Audio API.
* Supports multiple simultaneous audio sources for overlapping clips on different tracks.
* Uses AudioContext for sample-accurate timing synchronization.
*/
export function useStoryPlayback(items: StoryItemDetail[] | undefined) {
const isPlaying = useStoryStore((state) => state.isPlaying);
const playbackItems = useStoryStore((state) => state.playbackItems);
const playbackStartContextTime = useStoryStore((state) => state.playbackStartContextTime);
const playbackStartStoryTime = useStoryStore((state) => state.playbackStartStoryTime);
const setPlaybackTiming = useStoryStore((state) => state.setPlaybackTiming);
// AudioContext instance (created once)
const audioContextRef = useRef<AudioContext | null>(null);
// Master gain for volume control
const masterGainRef = useRef<GainNode | null>(null);
// Preloaded AudioBuffers by generation_id (audio file is shared between split clips)
const audioBuffersRef = useRef<Map<string, AudioBuffer>>(new Map());
// Currently playing AudioBufferSourceNodes by item.id (unique per clip)
const activeSourcesRef = useRef<Map<string, ActiveSource>>(new Map());
// Animation frame for syncing visual playhead
const animationFrameRef = useRef<number | null>(null);
// Get or create AudioContext and audio graph
const getAudioContext = useCallback(() => {
if (!audioContextRef.current) {
audioContextRef.current = new AudioContext();
console.log(
'[StoryPlayback] Created AudioContext, sample rate:',
audioContextRef.current.sampleRate,
);
// Create master gain node for volume control
masterGainRef.current = audioContextRef.current.createGain();
masterGainRef.current.gain.value = 1;
masterGainRef.current.connect(audioContextRef.current.destination);
}
// Resume context if suspended (browser autoplay policy)
if (audioContextRef.current.state === 'suspended') {
audioContextRef.current.resume().catch(() => {
// Ignore resume errors
});
}
return audioContextRef.current;
}, []);
// Stop a source by item id
const stopSource = useCallback((itemId: string) => {
const activeSource = activeSourcesRef.current.get(itemId);
if (activeSource) {
try {
activeSource.source.stop();
} catch {
// Source may have already stopped
}
activeSourcesRef.current.delete(itemId);
}
}, []);
// Preload audio files as AudioBuffers
useEffect(() => {
if (!items || items.length === 0) {
// Clear preloaded buffers when no items
audioBuffersRef.current.clear();
return;
}
const currentIds = new Set(items.map((item) => item.generation_id));
const audioContext = getAudioContext();
// Remove buffers for items that no longer exist
for (const [id] of audioBuffersRef.current) {
if (!currentIds.has(id)) {
audioBuffersRef.current.delete(id);
}
}
// Preload audio for new items
const preloadPromises: Promise<void>[] = [];
for (const item of items) {
if (!audioBuffersRef.current.has(item.generation_id)) {
const audioUrl = apiClient.getAudioUrl(item.generation_id);
console.log('[StoryPlayback] Preloading audio buffer:', item.generation_id);
const preloadPromise = fetch(audioUrl)
.then((response) => response.arrayBuffer())
.then((arrayBuffer) => audioContext.decodeAudioData(arrayBuffer))
.then((audioBuffer) => {
audioBuffersRef.current.set(item.generation_id, audioBuffer);
console.log(
'[StoryPlayback] Preloaded buffer:',
item.generation_id,
'duration:',
audioBuffer.duration,
);
})
.catch((err) => {
console.error('[StoryPlayback] Failed to preload audio:', item.generation_id, err);
});
preloadPromises.push(preloadPromise);
}
}
Promise.all(preloadPromises).then(() => {
console.log('[StoryPlayback] Preloaded', audioBuffersRef.current.size, 'audio buffers');
});
}, [items, getAudioContext]);
// Cleanup AudioContext on unmount
useEffect(() => {
return () => {
// Stop all sources
for (const [itemId] of activeSourcesRef.current) {
stopSource(itemId);
}
activeSourcesRef.current.clear();
// Clean up audio graph
if (masterGainRef.current) {
masterGainRef.current.disconnect();
masterGainRef.current = null;
}
if (audioContextRef.current && audioContextRef.current.state !== 'closed') {
audioContextRef.current.close().catch(() => {
// Ignore errors when closing
});
audioContextRef.current = null;
}
if (animationFrameRef.current !== null) {
cancelAnimationFrame(animationFrameRef.current);
}
};
}, [stopSource]);
// Find ALL items that should be playing at a given story time
const findActiveItems = useCallback(
(storyTimeMs: number, itemList: StoryItemDetail[]): StoryItemDetail[] => {
return itemList.filter((item) => {
const itemStart = item.start_time_ms;
// Use effective duration (accounting for trims)
const trimStartMs = item.trim_start_ms || 0;
const trimEndMs = item.trim_end_ms || 0;
const effectiveDurationMs = item.duration * 1000 - trimStartMs - trimEndMs;
const itemEnd = item.start_time_ms + effectiveDurationMs;
return storyTimeMs >= itemStart && storyTimeMs < itemEnd;
});
},
[],
);
// Convert AudioContext time to story time (ms)
const contextTimeToStoryTime = useCallback(
(contextTime: number): number => {
if (playbackStartContextTime === null || playbackStartStoryTime === null) {
return 0;
}
const elapsedContextTime = contextTime - playbackStartContextTime;
return playbackStartStoryTime + elapsedContextTime * 1000;
},
[playbackStartContextTime, playbackStartStoryTime],
);
// Convert story time (ms) to AudioContext time
const storyTimeToContextTime = useCallback(
(storyTimeMs: number): number => {
if (playbackStartContextTime === null || playbackStartStoryTime === null) {
return 0;
}
const elapsedStoryTime = (storyTimeMs - playbackStartStoryTime) / 1000;
return playbackStartContextTime + elapsedStoryTime;
},
[playbackStartContextTime, playbackStartStoryTime],
);
// Stop all sources
const stopAllSources = useCallback(() => {
console.log('[StoryPlayback] Stopping all sources');
for (const [itemId] of activeSourcesRef.current) {
stopSource(itemId);
}
activeSourcesRef.current.clear();
}, [stopSource]);
// Schedule playback for all items that should be playing
const schedulePlayback = useCallback(
(storyTimeMs: number, itemList: StoryItemDetail[]) => {
const audioContext = getAudioContext();
const currentContextTime = audioContext.currentTime;
// Find all items that should be playing
const shouldBePlaying = findActiveItems(storyTimeMs, itemList);
const shouldBePlayingIds = new Set(shouldBePlaying.map((item) => item.id));
// Stop sources that shouldn't be playing anymore
for (const [itemId] of activeSourcesRef.current) {
if (!shouldBePlayingIds.has(itemId)) {
stopSource(itemId);
}
}
// Schedule new sources for items that should be playing
for (const item of shouldBePlaying) {
if (!activeSourcesRef.current.has(item.id)) {
const buffer = audioBuffersRef.current.get(item.generation_id);
if (!buffer) {
console.warn('[StoryPlayback] Buffer not loaded for:', item.generation_id);
continue;
}
// Calculate when this item should start in AudioContext time
const itemStartContextTime = storyTimeToContextTime(item.start_time_ms);
// Calculate effective duration and trim offsets
const trimStartSec = (item.trim_start_ms || 0) / 1000;
const trimEndSec = (item.trim_end_ms || 0) / 1000;
const effectiveDuration = item.duration - trimStartSec - trimEndSec;
const itemEndStoryTime = item.start_time_ms + effectiveDuration * 1000;
// Calculate offset into the buffer (if seeking mid-way)
// Offset is relative to the trimmed start of the clip
const offsetIntoEffectiveClip = Math.max(0, (storyTimeMs - item.start_time_ms) / 1000);
const offsetIntoBuffer = trimStartSec + offsetIntoEffectiveClip;
const duration = effectiveDuration - offsetIntoEffectiveClip;
// If the item should have already started, schedule it to start immediately
const startAtContextTime = Math.max(currentContextTime, itemStartContextTime);
console.log('[StoryPlayback] Scheduling source:', {
itemId: item.id,
generationId: item.generation_id,
storyTimeMs,
itemStart: item.start_time_ms,
offsetIntoBuffer,
startAtContextTime,
duration,
});
const source = audioContext.createBufferSource();
source.buffer = buffer;
source.connect(masterGainRef.current || audioContext.destination);
const activeSource: ActiveSource = {
source,
itemId: item.id,
generationId: item.generation_id,
startTimeMs: item.start_time_ms,
endTimeMs: itemEndStoryTime,
};
activeSourcesRef.current.set(item.id, activeSource);
// Schedule playback
source.start(startAtContextTime, offsetIntoBuffer, duration);
// Clean up when source ends
source.onended = () => {
console.log('[StoryPlayback] Source ended:', item.id);
activeSourcesRef.current.delete(item.id);
};
}
}
},
[getAudioContext, findActiveItems, storyTimeToContextTime, stopSource],
);
// Sync visual playhead from AudioContext time
useEffect(() => {
if (!isPlaying || playbackStartContextTime === null || playbackStartStoryTime === null) {
if (animationFrameRef.current !== null) {
cancelAnimationFrame(animationFrameRef.current);
animationFrameRef.current = null;
}
return;
}
const audioContext = getAudioContext();
const itemList = playbackItems || [];
const syncPlayhead = () => {
if (!useStoryStore.getState().isPlaying) {
return;
}
const currentContextTime = audioContext.currentTime;
const currentStoryTime = contextTimeToStoryTime(currentContextTime);
const totalDuration = useStoryStore.getState().totalDurationMs;
// Update store with current story time
useStoryStore.setState({ currentTimeMs: Math.min(currentStoryTime, totalDuration) });
// Schedule any items that should be playing
schedulePlayback(currentStoryTime, itemList);
// Check if we've reached the end
if (currentStoryTime >= totalDuration) {
// Check if all sources have ended
if (activeSourcesRef.current.size === 0) {
console.log('[StoryPlayback] Reached end');
useStoryStore.getState().stop();
return;
}
}
// Continue sync loop
animationFrameRef.current = requestAnimationFrame(syncPlayhead);
};
// Initial sync
const currentContextTime = audioContext.currentTime;
const currentStoryTime = contextTimeToStoryTime(currentContextTime);
schedulePlayback(currentStoryTime, itemList);
// Start sync loop
animationFrameRef.current = requestAnimationFrame(syncPlayhead);
return () => {
if (animationFrameRef.current !== null) {
cancelAnimationFrame(animationFrameRef.current);
animationFrameRef.current = null;
}
};
}, [
isPlaying,
playbackItems,
playbackStartContextTime,
playbackStartStoryTime,
getAudioContext,
contextTimeToStoryTime,
schedulePlayback,
]);
// Handle play/pause changes - stop sources when paused
useEffect(() => {
if (!isPlaying) {
console.log('[StoryPlayback] Stopping playback');
stopAllSources();
}
}, [isPlaying, stopAllSources]);
// Handle seek - reset timing anchors when they become null (triggered by seek)
useEffect(() => {
if (!isPlaying || !playbackItems || playbackItems.length === 0) {
return;
}
// Only run when timing anchors are null (after a seek)
if (playbackStartContextTime !== null && playbackStartStoryTime !== null) {
return;
}
const audioContext = getAudioContext();
const currentContextTime = audioContext.currentTime;
const currentStoryTime = useStoryStore.getState().currentTimeMs;
console.log('[StoryPlayback] Setting timing anchors after seek:', {
contextTime: currentContextTime,
storyTime: currentStoryTime,
});
setPlaybackTiming(currentContextTime, currentStoryTime);
// Stop all existing sources and reschedule from new position
stopAllSources();
schedulePlayback(currentStoryTime, playbackItems);
}, [
isPlaying,
playbackItems,
playbackStartContextTime,
playbackStartStoryTime,
getAudioContext,
stopAllSources,
schedulePlayback,
setPlaybackTiming,
]);
}
+177
View File
@@ -0,0 +1,177 @@
import { useState, useRef, useCallback, useEffect } from 'react';
import { invoke } from '@tauri-apps/api/core';
import { isTauri } from '@/lib/tauri';
interface UseSystemAudioCaptureOptions {
maxDurationSeconds?: number;
onRecordingComplete?: (blob: Blob, duration?: number) => void;
}
/**
* Hook for native system audio capture using Tauri commands.
* Uses ScreenCaptureKit on macOS and WASAPI loopback on Windows.
*/
export function useSystemAudioCapture({
maxDurationSeconds = 29,
onRecordingComplete,
}: UseSystemAudioCaptureOptions = {}) {
const [isRecording, setIsRecording] = useState(false);
const [duration, setDuration] = useState(0);
const [error, setError] = useState<string | null>(null);
const [isSupported, setIsSupported] = useState(false);
const timerRef = useRef<number | null>(null);
const startTimeRef = useRef<number | null>(null);
const stopRecordingRef = useRef<(() => Promise<void>) | null>(null);
const isRecordingRef = useRef(false);
// Check if system audio capture is supported
useEffect(() => {
if (!isTauri()) {
setIsSupported(false);
return;
}
invoke<boolean>('is_system_audio_supported')
.then((supported) => {
setIsSupported(supported);
})
.catch(() => {
setIsSupported(false);
});
}, []);
const startRecording = useCallback(async () => {
if (!isTauri()) {
const errorMsg = 'System audio capture is only available in the desktop app.';
setError(errorMsg);
return;
}
if (!isSupported) {
const errorMsg = 'System audio capture is not supported on this platform.';
setError(errorMsg);
return;
}
try {
setError(null);
setDuration(0);
// Start native capture
await invoke('start_system_audio_capture', {
maxDurationSecs: maxDurationSeconds,
});
setIsRecording(true);
isRecordingRef.current = true;
startTimeRef.current = Date.now();
// Start timer
timerRef.current = window.setInterval(() => {
if (startTimeRef.current) {
const elapsed = (Date.now() - startTimeRef.current) / 1000;
setDuration(elapsed);
// Auto-stop at max duration
if (elapsed >= maxDurationSeconds && stopRecordingRef.current) {
void stopRecordingRef.current();
}
}
}, 100);
} catch (err) {
const errorMessage =
err instanceof Error
? err.message
: 'Failed to start system audio capture. Please check permissions.';
setError(errorMessage);
setIsRecording(false);
}
}, [maxDurationSeconds, isSupported]);
const stopRecording = useCallback(async () => {
if (!isRecording || !isTauri()) {
return;
}
try {
setIsRecording(false);
isRecordingRef.current = false;
if (timerRef.current !== null) {
clearInterval(timerRef.current);
timerRef.current = null;
}
// Stop capture and get base64 WAV data
const base64Data = await invoke<string>('stop_system_audio_capture');
// Convert base64 to Blob
const binaryString = atob(base64Data);
const bytes = new Uint8Array(binaryString.length);
for (let i = 0; i < binaryString.length; i++) {
bytes[i] = binaryString.charCodeAt(i);
}
const blob = new Blob([bytes], { type: 'audio/wav' });
// Pass the actual recorded duration
const recordedDuration = startTimeRef.current
? (Date.now() - startTimeRef.current) / 1000
: undefined;
onRecordingComplete?.(blob, recordedDuration);
} catch (err) {
const errorMessage =
err instanceof Error
? err.message
: 'Failed to stop system audio capture.';
setError(errorMessage);
}
}, [isRecording, onRecordingComplete]);
// Store stopRecording in ref for use in timer
useEffect(() => {
stopRecordingRef.current = stopRecording;
}, [stopRecording]);
const cancelRecording = useCallback(async () => {
if (isRecordingRef.current) {
await stopRecording();
}
setIsRecording(false);
isRecordingRef.current = false;
setDuration(0);
if (timerRef.current !== null) {
clearInterval(timerRef.current);
timerRef.current = null;
}
}, [stopRecording]);
// Cleanup on unmount only
useEffect(() => {
return () => {
if (timerRef.current !== null) {
clearInterval(timerRef.current);
timerRef.current = null;
}
// Cancel recording on unmount if still recording
if (isRecordingRef.current && isTauri()) {
// Call stop directly without the callback to avoid stale closure
invoke('stop_system_audio_capture').catch((err) => {
console.error('Error stopping audio capture on unmount:', err);
});
}
};
// biome-ignore lint/correctness/useExhaustiveDependencies: Only run on unmount
}, []);
return {
isRecording,
duration,
error,
isSupported,
startRecording,
stopRecording,
cancelRecording,
};
}
+2 -1
View File
@@ -1,9 +1,10 @@
import { useMutation } from '@tanstack/react-query';
import { apiClient } from '@/lib/api/client';
import type { LanguageCode } from '@/lib/constants/languages';
export function useTranscription() {
return useMutation({
mutationFn: ({ file, language }: { file: File; language?: 'en' | 'zh' }) =>
mutationFn: ({ file, language }: { file: File; language?: LanguageCode }) =>
apiClient.transcribeAudio(file, language),
});
}
+22
View File
@@ -12,6 +12,13 @@ export function isTauri(): boolean {
return '__TAURI_INTERNALS__' in window;
}
/**
* Check if running on macOS
*/
export function isMacOS(): boolean {
return navigator.platform.toLowerCase().includes('mac');
}
/**
* Start the bundled Python server (Tauri only)
*/
@@ -47,6 +54,21 @@ export async function stopServer(): Promise<void> {
}
}
/**
* Set whether the server should keep running when the app closes (Tauri only)
*/
export async function setKeepServerRunning(keepRunning: boolean): Promise<void> {
if (!isTauri()) {
return;
}
try {
await invoke('set_keep_server_running', { keepRunning });
} catch (error) {
console.error('Failed to set keep server running setting:', error);
}
}
/**
* Setup window close handler to check setting and stop server if needed
*/
+136
View File
@@ -16,3 +16,139 @@ export function formatAudioDuration(seconds: number): string {
const secs = Math.floor(seconds % 60);
return `${mins}:${secs.toString().padStart(2, '0')}`;
}
/**
* Get audio duration from a File.
* If the file has a recordedDuration property (from recording hooks),
* use that instead of trying to read metadata. This fixes issues on Windows
* where WebM files from MediaRecorder don't have proper duration metadata.
*/
export async function getAudioDuration(
file: File & { recordedDuration?: number },
): Promise<number> {
if (file.recordedDuration !== undefined && Number.isFinite(file.recordedDuration)) {
return file.recordedDuration;
}
return new Promise((resolve, reject) => {
const audio = new Audio();
const url = URL.createObjectURL(file);
audio.addEventListener('loadedmetadata', () => {
URL.revokeObjectURL(url);
if (Number.isFinite(audio.duration) && audio.duration > 0) {
resolve(audio.duration);
} else {
reject(new Error('Audio file has invalid duration metadata'));
}
});
audio.addEventListener('error', () => {
URL.revokeObjectURL(url);
reject(new Error('Failed to load audio file'));
});
audio.src = url;
});
}
/**
* Convert any audio blob to WAV format using Web Audio API.
* This ensures compatibility without requiring ffmpeg on the backend.
*/
export async function convertToWav(audioBlob: Blob): Promise<Blob> {
// Create audio context
const audioContext = new AudioContext();
// Read blob as array buffer
const arrayBuffer = await audioBlob.arrayBuffer();
// Decode audio data
const audioBuffer = await audioContext.decodeAudioData(arrayBuffer);
// Convert to WAV
const wavBlob = audioBufferToWav(audioBuffer);
// Close audio context to free resources
await audioContext.close();
return wavBlob;
}
/**
* Convert AudioBuffer to WAV blob.
*/
function audioBufferToWav(buffer: AudioBuffer): Blob {
const numberOfChannels = buffer.numberOfChannels;
const sampleRate = buffer.sampleRate;
const format = 1; // PCM
const bitDepth = 16;
const bytesPerSample = bitDepth / 8;
const blockAlign = numberOfChannels * bytesPerSample;
// Interleave channels
const interleaved = interleaveChannels(buffer);
// Create WAV file
const dataLength = interleaved.length * bytesPerSample;
const buffer2 = new ArrayBuffer(44 + dataLength);
const view = new DataView(buffer2);
// Write WAV header
writeString(view, 0, 'RIFF');
view.setUint32(4, 36 + dataLength, true);
writeString(view, 8, 'WAVE');
writeString(view, 12, 'fmt ');
view.setUint32(16, 16, true); // fmt chunk size
view.setUint16(20, format, true); // audio format (PCM)
view.setUint16(22, numberOfChannels, true);
view.setUint32(24, sampleRate, true);
view.setUint32(28, sampleRate * blockAlign, true); // byte rate
view.setUint16(32, blockAlign, true);
view.setUint16(34, bitDepth, true);
writeString(view, 36, 'data');
view.setUint32(40, dataLength, true);
// Write audio data
floatTo16BitPCM(view, 44, interleaved);
return new Blob([buffer2], { type: 'audio/wav' });
}
/**
* Interleave multiple channels into a single array.
*/
function interleaveChannels(buffer: AudioBuffer): Float32Array {
const numberOfChannels = buffer.numberOfChannels;
const length = buffer.length;
const interleaved = new Float32Array(length * numberOfChannels);
for (let channel = 0; channel < numberOfChannels; channel++) {
const channelData = buffer.getChannelData(channel);
for (let i = 0; i < length; i++) {
interleaved[i * numberOfChannels + channel] = channelData[i];
}
}
return interleaved;
}
/**
* Write string to DataView.
*/
function writeString(view: DataView, offset: number, string: string): void {
for (let i = 0; i < string.length; i++) {
view.setUint8(offset + i, string.charCodeAt(i));
}
}
/**
* Convert float32 audio data to 16-bit PCM.
*/
function floatTo16BitPCM(view: DataView, offset: number, input: Float32Array): void {
for (let i = 0; i < input.length; i++, offset += 2) {
const s = Math.max(-1, Math.min(1, input[i]));
view.setInt16(offset, s < 0 ? s * 0x8000 : s * 0x7fff, true);
}
}
+19
View File
@@ -0,0 +1,19 @@
const DEBUG = import.meta.env.DEV;
export const debug = {
log: (...args: unknown[]) => {
if (DEBUG) {
console.log(...args);
}
},
error: (...args: unknown[]) => {
if (DEBUG) {
console.error(...args);
}
},
warn: (...args: unknown[]) => {
if (DEBUG) {
console.warn(...args);
}
},
};
+16 -1
View File
@@ -7,7 +7,22 @@ export function formatDuration(seconds: number): string {
}
export function formatDate(date: string | Date): string {
return formatDistance(new Date(date), new Date(), { addSuffix: true });
// Parse the date string - if it doesn't have timezone info, treat it as UTC
let dateObj: Date;
if (typeof date === 'string') {
// If the string doesn't end with Z or have timezone offset, assume it's UTC
const dateStr = date.trim();
if (!dateStr.includes('Z') && !dateStr.match(/[+-]\d{2}:\d{2}$/)) {
// No timezone info, treat as UTC
dateObj = new Date(dateStr + 'Z');
} else {
dateObj = new Date(dateStr);
}
} else {
dateObj = date;
}
return formatDistance(dateObj, new Date(), { addSuffix: true }).replace(/^about /i, '');
}
export function formatFileSize(bytes: number): string {
+2 -2
View File
@@ -1,5 +1,5 @@
import { QueryClient, QueryClientProvider } from '@tanstack/react-query';
import { ReactQueryDevtools } from '@tanstack/react-query-devtools';
// import { ReactQueryDevtools } from '@tanstack/react-query-devtools';
import React from 'react';
import ReactDOM from 'react-dom/client';
import App from './App';
@@ -20,7 +20,7 @@ ReactDOM.createRoot(document.getElementById('root')!).render(
<React.StrictMode>
<QueryClientProvider client={queryClient}>
<App />
<ReactQueryDevtools initialIsOpen={false} />
{/* <ReactQueryDevtools initialIsOpen={false} /> */}
</QueryClientProvider>
</React.StrictMode>,
);
+134
View File
@@ -0,0 +1,134 @@
import { createRootRoute, createRoute, createRouter, Outlet } from '@tanstack/react-router';
import { AppFrame } from '@/components/AppFrame/AppFrame';
import { AudioTab } from '@/components/AudioTab/AudioTab';
import { MainEditor } from '@/components/MainEditor/MainEditor';
import { ModelsTab } from '@/components/ModelsTab/ModelsTab';
import { ServerTab } from '@/components/ServerTab/ServerTab';
import { Sidebar } from '@/components/Sidebar';
import { StoriesTab } from '@/components/StoriesTab/StoriesTab';
import { Toaster } from '@/components/ui/toaster';
import { VoicesTab } from '@/components/VoicesTab/VoicesTab';
import { useModelDownloadToast } from '@/lib/hooks/useModelDownloadToast';
import { MODEL_DISPLAY_NAMES, useRestoreActiveTasks } from '@/lib/hooks/useRestoreActiveTasks';
import { isMacOS } from '@/lib/tauri';
// Root layout component
function RootLayout() {
// Monitor active downloads/generations and show toasts for them
const activeDownloads = useRestoreActiveTasks();
return (
<AppFrame>
<div className="flex flex-1 min-h-0 overflow-hidden">
<Sidebar isMacOS={isMacOS()} />
<main className="flex-1 ml-20 overflow-hidden flex flex-col">
<div className="container mx-auto px-8 max-w-[1800px] h-full overflow-hidden flex flex-col">
<Outlet />
</div>
</main>
</div>
{/* Show download toasts for any active downloads (from anywhere) */}
{activeDownloads.map((download) => {
const displayName = MODEL_DISPLAY_NAMES[download.model_name] || download.model_name;
return (
<DownloadToastRestorer
key={download.model_name}
modelName={download.model_name}
displayName={displayName}
/>
);
})}
<Toaster />
</AppFrame>
);
}
/**
* Component that restores a download toast for a specific model.
*/
function DownloadToastRestorer({
modelName,
displayName,
}: {
modelName: string;
displayName: string;
}) {
// Use the download toast hook to restore the toast
useModelDownloadToast({
modelName,
displayName,
enabled: true,
});
return null;
}
// Root route with layout
const rootRoute = createRootRoute({
component: RootLayout,
});
// Index route (main/generate)
const indexRoute = createRoute({
getParentRoute: () => rootRoute,
path: '/',
component: MainEditor,
});
// Stories route
const storiesRoute = createRoute({
getParentRoute: () => rootRoute,
path: '/stories',
component: StoriesTab,
});
// Voices route
const voicesRoute = createRoute({
getParentRoute: () => rootRoute,
path: '/voices',
component: VoicesTab,
});
// Audio route
const audioRoute = createRoute({
getParentRoute: () => rootRoute,
path: '/audio',
component: AudioTab,
});
// Models route
const modelsRoute = createRoute({
getParentRoute: () => rootRoute,
path: '/models',
component: ModelsTab,
});
// Server route
const serverRoute = createRoute({
getParentRoute: () => rootRoute,
path: '/server',
component: ServerTab,
});
// Route tree
const routeTree = rootRoute.addChildren([
indexRoute,
storiesRoute,
voicesRoute,
audioRoute,
modelsRoute,
serverRoute,
]);
// Create router
export const router = createRouter({ routeTree });
// Register router for type safety
declare module '@tanstack/react-router' {
interface Register {
router: typeof router;
}
}
+42
View File
@@ -0,0 +1,42 @@
import { create } from 'zustand';
import { persist } from 'zustand/middleware';
export interface AudioChannel {
id: string;
name: string;
is_default: boolean;
device_ids: string[];
created_at: string;
}
interface AudioChannelStore {
channels: AudioChannel[];
setChannels: (channels: AudioChannel[]) => void;
addChannel: (channel: AudioChannel) => void;
updateChannel: (id: string, channel: Partial<AudioChannel>) => void;
removeChannel: (id: string) => void;
}
export const useAudioChannelStore = create<AudioChannelStore>()(
persist(
(set) => ({
channels: [],
setChannels: (channels) => set({ channels }),
addChannel: (channel) =>
set((state) => ({
channels: [...state.channels, channel],
})),
updateChannel: (id, updates) =>
set((state) => ({
channels: state.channels.map((ch) => (ch.id === id ? { ...ch, ...updates } : ch)),
})),
removeChannel: (id) =>
set((state) => ({
channels: state.channels.filter((ch) => ch.id !== id),
})),
}),
{
name: 'voicebox-audio-channels',
},
),
);
+15
View File
@@ -0,0 +1,15 @@
import { create } from 'zustand';
interface GenerationState {
isGenerating: boolean;
activeGenerationId: string | null;
setIsGenerating: (generating: boolean) => void;
setActiveGenerationId: (id: string | null) => void;
}
export const useGenerationStore = create<GenerationState>((set) => ({
isGenerating: false,
activeGenerationId: null,
setIsGenerating: (generating) => set({ isGenerating: generating }),
setActiveGenerationId: (id) => set({ activeGenerationId: id }),
}));
+37 -2
View File
@@ -3,53 +3,88 @@ import { create } from 'zustand';
interface PlayerState {
audioUrl: string | null;
audioId: string | null;
profileId: string | null;
title: string | null;
isPlaying: boolean;
currentTime: number;
duration: number;
volume: number;
isLooping: boolean;
shouldRestart: boolean;
shouldAutoPlay: boolean;
onFinish: (() => void) | null;
setAudio: (url: string, id: string, title?: string) => void;
setAudio: (url: string, id: string, profileId: string | null, title?: string) => void;
setAudioWithAutoPlay: (url: string, id: string, profileId: string | null, title?: string) => void;
setIsPlaying: (playing: boolean) => void;
setCurrentTime: (time: number) => void;
setDuration: (duration: number) => void;
setVolume: (volume: number) => void;
toggleLoop: () => void;
restartCurrentAudio: () => void;
clearRestartFlag: () => void;
clearAutoPlayFlag: () => void;
setOnFinish: (callback: (() => void) | null) => void;
reset: () => void;
}
export const usePlayerStore = create<PlayerState>((set) => ({
audioUrl: null,
audioId: null,
profileId: null,
title: null,
isPlaying: false,
currentTime: 0,
duration: 0,
volume: 1,
isLooping: false,
shouldRestart: false,
shouldAutoPlay: false,
onFinish: null,
setAudio: (url, id, title) =>
setAudio: (url, id, profileId, title) =>
set({
audioUrl: url,
audioId: id,
profileId: profileId || null,
title: title || null,
currentTime: 0,
isPlaying: false,
shouldRestart: false,
shouldAutoPlay: false,
}),
setAudioWithAutoPlay: (url, id, profileId, title) =>
set({
audioUrl: url,
audioId: id,
profileId: profileId || null,
title: title || null,
currentTime: 0,
isPlaying: false,
shouldRestart: false,
shouldAutoPlay: true,
}),
setIsPlaying: (playing) => set({ isPlaying: playing }),
setCurrentTime: (time) => set({ currentTime: time }),
setDuration: (duration) => set({ duration }),
setVolume: (volume) => set({ volume }),
toggleLoop: () => set((state) => ({ isLooping: !state.isLooping })),
restartCurrentAudio: () => set({ shouldRestart: true }),
clearRestartFlag: () => set({ shouldRestart: false }),
clearAutoPlayFlag: () => set({ shouldAutoPlay: false }),
setOnFinish: (callback) => set({ onFinish: callback }),
reset: () =>
set({
audioUrl: null,
audioId: null,
profileId: null,
title: null,
isPlaying: false,
currentTime: 0,
duration: 0,
isLooping: false,
shouldRestart: false,
shouldAutoPlay: false,
onFinish: null,
}),
}));
+1 -1
View File
@@ -18,7 +18,7 @@ interface ServerStore {
export const useServerStore = create<ServerStore>()(
persist(
(set) => ({
serverUrl: 'http://localhost:8000',
serverUrl: 'http://127.0.0.1:17493',
setServerUrl: (url) => set({ serverUrl: url }),
isConnected: false,
+148
View File
@@ -0,0 +1,148 @@
import { create } from 'zustand';
import type { StoryItemDetail } from '@/lib/api/types';
interface StoryPlaybackState {
// Selection
selectedStoryId: string | null;
setSelectedStoryId: (id: string | null) => void;
selectedClipId: string | null;
setSelectedClipId: (id: string | null) => void;
// Track editor UI state
trackEditorHeight: number;
setTrackEditorHeight: (height: number) => void;
// Playback state
isPlaying: boolean;
currentTimeMs: number;
totalDurationMs: number;
playbackStoryId: string | null;
playbackItems: StoryItemDetail[] | null;
// Web Audio API timing (null when not playing)
playbackStartContextTime: number | null; // AudioContext.currentTime when playback started
playbackStartStoryTime: number | null; // Story time (ms) when playback started
// Actions
play: (storyId: string, items: StoryItemDetail[]) => void;
pause: () => void;
stop: () => void;
seek: (timeMs: number) => void;
setPlaybackTiming: (contextTime: number, storyTime: number) => void; // Set timing anchors for Web Audio API
setActiveStory: (storyId: string, items: StoryItemDetail[], totalDurationMs: number) => void; // Activate story for seeking without playing
}
const DEFAULT_TRACK_EDITOR_HEIGHT = 250;
export const useStoryStore = create<StoryPlaybackState>((set, get) => ({
// Selection
selectedStoryId: null,
setSelectedStoryId: (id) => set({ selectedStoryId: id }),
selectedClipId: null,
setSelectedClipId: (id) => set({ selectedClipId: id }),
// Track editor UI state
trackEditorHeight: DEFAULT_TRACK_EDITOR_HEIGHT,
setTrackEditorHeight: (height) => set({ trackEditorHeight: height }),
// Playback state
isPlaying: false,
currentTimeMs: 0,
totalDurationMs: 0,
playbackStoryId: null,
playbackItems: null,
playbackStartContextTime: null,
playbackStartStoryTime: null,
// Actions
play: (storyId, items) => {
// Calculate total duration from items
const maxEndTimeMs = Math.max(
...items.map((item) => item.start_time_ms + item.duration * 1000),
0,
);
// Find the minimum start time (first item)
const minStartTimeMs = Math.min(...items.map((item) => item.start_time_ms), 0);
// If resuming the same story, keep position; otherwise start at first item
const currentState = get();
const shouldResume = currentState.playbackStoryId === storyId && currentState.currentTimeMs > 0;
const startTimeMs = shouldResume ? currentState.currentTimeMs : minStartTimeMs;
console.log('[StoryStore] Play called:', {
storyId,
itemCount: items.length,
items: items.map((i) => ({
id: i.generation_id,
start: i.start_time_ms,
duration: i.duration,
})),
maxEndTimeMs,
minStartTimeMs,
startTimeMs,
shouldResume,
});
set({
isPlaying: true,
playbackStoryId: storyId,
playbackItems: items,
totalDurationMs: maxEndTimeMs,
currentTimeMs: startTimeMs,
// Reset timing anchors - will be set fresh by the playback hook
playbackStartContextTime: null,
playbackStartStoryTime: null,
});
},
pause: () => {
set({
isPlaying: false,
// Keep timing anchors so we can resume from same position
});
},
stop: () => {
set({
isPlaying: false,
currentTimeMs: 0,
playbackStoryId: null,
playbackItems: null,
totalDurationMs: 0,
playbackStartContextTime: null,
playbackStartStoryTime: null,
});
},
seek: (timeMs) => {
const state = get();
const clampedTime = Math.max(0, Math.min(timeMs, state.totalDurationMs));
set({
currentTimeMs: clampedTime,
// Reset timing anchors - will be set by hook when playback resumes
playbackStartContextTime: null,
playbackStartStoryTime: null,
});
},
setPlaybackTiming: (contextTime, storyTime) => {
set({
playbackStartContextTime: contextTime,
playbackStartStoryTime: storyTime,
});
},
setActiveStory: (storyId, items, totalDurationMs) => {
const currentState = get();
// Only update if switching to a different story
if (currentState.playbackStoryId !== storyId) {
set({
playbackStoryId: storyId,
playbackItems: items,
totalDurationMs,
currentTimeMs: 0,
isPlaying: false,
});
}
},
}));
+20
View File
@@ -1,5 +1,18 @@
import { create } from 'zustand';
// Draft state for the create voice profile form
export interface ProfileFormDraft {
name: string;
description: string;
language: string;
referenceText: string;
sampleMode: 'upload' | 'record' | 'system';
// Note: File objects can't be persisted, so we store metadata
sampleFileName?: string;
sampleFileType?: string;
sampleFileData?: string; // Base64 encoded
}
interface UIStore {
// Sidebar
sidebarOpen: boolean;
@@ -18,6 +31,10 @@ interface UIStore {
selectedProfileId: string | null;
setSelectedProfileId: (id: string | null) => void;
// Profile form draft (for persisting create voice modal state)
profileFormDraft: ProfileFormDraft | null;
setProfileFormDraft: (draft: ProfileFormDraft | null) => void;
// Theme
theme: 'light' | 'dark';
setTheme: (theme: 'light' | 'dark') => void;
@@ -38,6 +55,9 @@ export const useUIStore = create<UIStore>((set) => ({
selectedProfileId: null,
setSelectedProfileId: (id) => set({ selectedProfileId: id }),
profileFormDraft: null,
setProfileFormDraft: (draft) => set({ profileFormDraft: draft }),
theme: 'light',
setTheme: (theme) => {
set({ theme });
+28 -7
View File
@@ -19,8 +19,13 @@ Production-quality FastAPI backend for Qwen3-TTS voice cloning.
backend/
├── main.py # FastAPI app with all routes
├── models.py # Pydantic request/response models
├── tts.py # Qwen3-TTS inference
├── transcribe.py # Whisper ASR
├── platform_detect.py # Platform detection for backend selection
├── tts.py # TTS backend abstraction (delegates to MLX or PyTorch)
├── transcribe.py # STT backend abstraction (delegates to MLX or PyTorch)
├── backends/ # Backend implementations
│ ├── __init__.py # Backend factory and protocols
│ ├── mlx_backend.py # MLX backend (Apple Silicon)
│ └── pytorch_backend.py # PyTorch backend (Windows/Linux/Intel)
├── profiles.py # Voice profile CRUD
├── history.py # Generation history
├── studio.py # Audio editing (TODO)
@@ -31,6 +36,15 @@ backend/
└── validation.py # Input validation
```
### Backend Selection
Voicebox automatically selects the best backend based on platform:
- **Apple Silicon (M1/M2/M3)**: Uses MLX backend with native Metal acceleration (4-5x faster)
- **Windows/Linux/Intel Mac**: Uses PyTorch backend (CUDA GPU if available, CPU fallback)
The backend is detected at runtime via `platform_detect.py`. Both backends implement the same interface, so the API remains consistent across platforms.
## API Endpoints
### Health & Info
@@ -47,12 +61,20 @@ Health check with model status.
"status": "healthy",
"model_loaded": true,
"gpu_available": true,
"vram_used_mb": 1024.5
"gpu_type": "Metal (Apple Silicon via MLX)",
"backend_type": "mlx",
"vram_used_mb": null
}
```
**Backend Types:**
- `"mlx"` - MLX backend (Apple Silicon with Metal acceleration)
- `"pytorch"` - PyTorch backend (Windows/Linux/Intel Mac)
### Voice Profiles
**Note:** The database is automatically initialized when the server starts. No manual setup required.
#### `POST /profiles`
Create a new voice profile.
@@ -266,13 +288,12 @@ data/
pip install -r requirements.txt
```
### 2. Initialize Database
**Note:** On Apple Silicon, also install MLX dependencies for faster inference:
```bash
python -c "from database import init_db; init_db()"
pip install -r requirements-mlx.txt
```
### 3. Download Models (Automatic)
### 2. Download Models (Automatic)
The Qwen3-TTS models are automatically downloaded from HuggingFace Hub on first use, similar to how Whisper models work.
+2
View File
@@ -1 +1,3 @@
# Backend package
__version__ = "0.1.10"
+166
View File
@@ -0,0 +1,166 @@
"""
Backend abstraction layer for TTS and STT.
Provides a unified interface for MLX and PyTorch backends.
"""
from typing import Protocol, Optional, Tuple, List
from typing_extensions import runtime_checkable
import numpy as np
from ..platform_detect import get_backend_type
@runtime_checkable
class TTSBackend(Protocol):
"""Protocol for TTS backend implementations."""
async def load_model(self, model_size: str) -> None:
"""Load TTS model."""
...
async def create_voice_prompt(
self,
audio_path: str,
reference_text: str,
use_cache: bool = True,
) -> Tuple[dict, bool]:
"""
Create voice prompt from reference audio.
Returns:
Tuple of (voice_prompt_dict, was_cached)
"""
...
async def combine_voice_prompts(
self,
audio_paths: List[str],
reference_texts: List[str],
) -> Tuple[np.ndarray, str]:
"""
Combine multiple voice prompts.
Returns:
Tuple of (combined_audio_array, combined_text)
"""
...
async def generate(
self,
text: str,
voice_prompt: dict,
language: str = "en",
seed: Optional[int] = None,
instruct: Optional[str] = None,
) -> Tuple[np.ndarray, int]:
"""
Generate audio from text.
Returns:
Tuple of (audio_array, sample_rate)
"""
...
def unload_model(self) -> None:
"""Unload model to free memory."""
...
def is_loaded(self) -> bool:
"""Check if model is loaded."""
...
def _get_model_path(self, model_size: str) -> str:
"""
Get model path for a given size.
Returns:
Model path or HuggingFace Hub ID
"""
...
@runtime_checkable
class STTBackend(Protocol):
"""Protocol for STT (Speech-to-Text) backend implementations."""
async def load_model(self, model_size: str) -> None:
"""Load STT model."""
...
async def transcribe(
self,
audio_path: str,
language: Optional[str] = None,
) -> str:
"""
Transcribe audio to text.
Returns:
Transcribed text
"""
...
def unload_model(self) -> None:
"""Unload model to free memory."""
...
def is_loaded(self) -> bool:
"""Check if model is loaded."""
...
# Global backend instances
_tts_backend: Optional[TTSBackend] = None
_stt_backend: Optional[STTBackend] = None
def get_tts_backend() -> TTSBackend:
"""
Get or create TTS backend instance based on platform.
Returns:
TTS backend instance (MLX or PyTorch)
"""
global _tts_backend
if _tts_backend is None:
backend_type = get_backend_type()
if backend_type == "mlx":
from .mlx_backend import MLXTTSBackend
_tts_backend = MLXTTSBackend()
else:
from .pytorch_backend import PyTorchTTSBackend
_tts_backend = PyTorchTTSBackend()
return _tts_backend
def get_stt_backend() -> STTBackend:
"""
Get or create STT backend instance based on platform.
Returns:
STT backend instance (MLX or PyTorch)
"""
global _stt_backend
if _stt_backend is None:
backend_type = get_backend_type()
if backend_type == "mlx":
from .mlx_backend import MLXSTTBackend
_stt_backend = MLXSTTBackend()
else:
from .pytorch_backend import PyTorchSTTBackend
_stt_backend = PyTorchSTTBackend()
return _stt_backend
def reset_backends():
"""Reset backend instances (useful for testing)."""
global _tts_backend, _stt_backend
_tts_backend = None
_stt_backend = None
+445
View File
@@ -0,0 +1,445 @@
"""
MLX backend implementation for TTS and STT using mlx-audio.
"""
from typing import Optional, List, Tuple
import asyncio
import numpy as np
from pathlib import Path
from . import TTSBackend, STTBackend
from ..utils.cache import get_cache_key, get_cached_voice_prompt, cache_voice_prompt
from ..utils.audio import normalize_audio, load_audio
from ..utils.progress import get_progress_manager
from ..utils.hf_progress import HFProgressTracker, create_hf_progress_callback
from ..utils.tasks import get_task_manager
class MLXTTSBackend:
"""MLX-based TTS backend using mlx-audio."""
def __init__(self, model_size: str = "1.7B"):
self.model = None
self.model_size = model_size
self._current_model_size = None
def is_loaded(self) -> bool:
"""Check if model is loaded."""
return self.model is not None
def _get_model_path(self, model_size: str) -> str:
"""
Get the MLX model path.
Args:
model_size: Model size (1.7B or 0.6B)
Returns:
HuggingFace Hub model ID for MLX
"""
# MLX model mapping
mlx_model_map = {
"1.7B": "mlx-community/Qwen3-TTS-12Hz-1.7B-Base-bf16",
# 0.6B not yet converted to MLX format
"0.6B": "mlx-community/Qwen3-TTS-12Hz-1.7B-Base-bf16", # Fallback to 1.7B
}
if model_size not in mlx_model_map:
raise ValueError(f"Unknown model size: {model_size}")
hf_model_id = mlx_model_map[model_size]
print(f"Will download MLX model from HuggingFace Hub: {hf_model_id}")
return hf_model_id
async def load_model_async(self, model_size: Optional[str] = None):
"""
Lazy load the MLX TTS model.
Args:
model_size: Model size to load (1.7B or 0.6B)
"""
if model_size is None:
model_size = self.model_size
# If already loaded with correct size, return
if self.model is not None and self._current_model_size == model_size:
return
# Unload existing model if different size requested
if self.model is not None and self._current_model_size != model_size:
self.unload_model()
# Run blocking load in thread pool
await asyncio.to_thread(self._load_model_sync, model_size)
# Alias for compatibility
load_model = load_model_async
def _load_model_sync(self, model_size: str):
"""Synchronous model loading."""
try:
from mlx_audio.tts import load
# Get model path
model_path = self._get_model_path(model_size)
# Set up progress tracking
progress_manager = get_progress_manager()
model_name = f"qwen-tts-{model_size}"
# Start tracking download task
task_manager = get_task_manager()
task_manager.start_download(model_name)
print(f"Loading MLX TTS model {model_size}...")
# Initialize progress state
progress_manager.update_progress(
model_name=model_name,
current=0,
total=1,
filename="",
status="downloading",
)
# Set up progress callback
progress_callback = create_hf_progress_callback(model_name, progress_manager)
tracker = HFProgressTracker(progress_callback)
# Use progress tracker during download
with tracker.patch_download():
# Load MLX model (downloads automatically)
self.model = load(model_path)
self._current_model_size = model_size
self.model_size = model_size
# Mark as complete
progress_manager.mark_complete(model_name)
task_manager.complete_download(model_name)
print(f"MLX TTS model {model_size} loaded successfully")
except ImportError as e:
print(f"Error: mlx_audio package not found. Install with: pip install mlx-audio")
progress_manager = get_progress_manager()
task_manager = get_task_manager()
model_name = f"qwen-tts-{model_size}"
progress_manager.mark_error(model_name, str(e))
task_manager.error_download(model_name, str(e))
raise
except Exception as e:
print(f"Error loading MLX TTS model: {e}")
progress_manager = get_progress_manager()
task_manager = get_task_manager()
model_name = f"qwen-tts-{model_size}"
progress_manager.mark_error(model_name, str(e))
task_manager.error_download(model_name, str(e))
raise
def unload_model(self):
"""Unload the model to free memory."""
if self.model is not None:
del self.model
self.model = None
self._current_model_size = None
print("MLX TTS model unloaded")
async def create_voice_prompt(
self,
audio_path: str,
reference_text: str,
use_cache: bool = True,
) -> Tuple[dict, bool]:
"""
Create voice prompt from reference audio.
MLX backend stores voice prompt as a dict with audio path and text.
The actual voice prompt processing happens during generation.
Args:
audio_path: Path to reference audio file
reference_text: Transcript of reference audio
use_cache: Whether to use cached prompt if available
Returns:
Tuple of (voice_prompt_dict, was_cached)
"""
await self.load_model_async(None)
# Check cache if enabled
if use_cache:
cache_key = get_cache_key(audio_path, reference_text)
cached_prompt = get_cached_voice_prompt(cache_key)
if cached_prompt is not None:
# Return cached prompt (should be dict format)
if isinstance(cached_prompt, dict):
return cached_prompt, True
# MLX voice prompt format - store audio path and text
# The model will process this during generation
voice_prompt_items = {
"ref_audio": str(audio_path),
"ref_text": reference_text,
}
# Cache if enabled
if use_cache:
cache_key = get_cache_key(audio_path, reference_text)
cache_voice_prompt(cache_key, voice_prompt_items)
return voice_prompt_items, False
async def combine_voice_prompts(
self,
audio_paths: List[str],
reference_texts: List[str],
) -> Tuple[np.ndarray, str]:
"""
Combine multiple reference samples for better quality.
Args:
audio_paths: List of audio file paths
reference_texts: List of reference texts
Returns:
Tuple of (combined_audio, combined_text)
"""
combined_audio = []
for audio_path in audio_paths:
audio, sr = load_audio(audio_path)
audio = normalize_audio(audio)
combined_audio.append(audio)
# Concatenate audio
mixed = np.concatenate(combined_audio)
mixed = normalize_audio(mixed)
# Combine texts
combined_text = " ".join(reference_texts)
return mixed, combined_text
async def generate(
self,
text: str,
voice_prompt: dict,
language: str = "en",
seed: Optional[int] = None,
instruct: Optional[str] = None,
) -> Tuple[np.ndarray, int]:
"""
Generate audio from text using voice prompt.
Args:
text: Text to synthesize
voice_prompt: Voice prompt dictionary with ref_audio and ref_text
language: Language code (en or zh) - may not be fully supported by MLX
seed: Random seed for reproducibility
instruct: Natural language instruction (may not be supported by MLX)
Returns:
Tuple of (audio_array, sample_rate)
"""
await self.load_model_async(None)
print(f"Generating audio for text: {text}")
def _generate_sync():
"""Run synchronous generation in thread pool."""
# MLX generate() returns a generator yielding GenerationResult objects
audio_chunks = []
sample_rate = 24000
# Set seed if provided (MLX uses numpy random)
if seed is not None:
import mlx.core as mx
np.random.seed(seed)
mx.random.seed(seed)
# Extract voice prompt info
ref_audio = voice_prompt.get("ref_audio") or voice_prompt.get("ref_audio_path")
ref_text = voice_prompt.get("ref_text", "")
# Check if model supports voice cloning via generate method
# MLX API may support ref_audio parameter directly
try:
# Try with voice cloning parameters if supported
if ref_audio:
# Check if generate accepts ref_audio parameter
import inspect
sig = inspect.signature(self.model.generate)
if "ref_audio" in sig.parameters:
# Generate with voice cloning
for result in self.model.generate(text, ref_audio=ref_audio, ref_text=ref_text):
audio_chunks.append(np.array(result.audio))
sample_rate = result.sample_rate
else:
# Fallback: generate without voice cloning
for result in self.model.generate(text):
audio_chunks.append(np.array(result.audio))
sample_rate = result.sample_rate
else:
# No voice prompt, generate normally
for result in self.model.generate(text):
audio_chunks.append(np.array(result.audio))
sample_rate = result.sample_rate
except Exception as e:
# If voice cloning fails, try without it
print(f"Warning: Voice cloning failed, generating without voice prompt: {e}")
for result in self.model.generate(text):
audio_chunks.append(np.array(result.audio))
sample_rate = result.sample_rate
# Concatenate all chunks
if audio_chunks:
audio = np.concatenate([np.asarray(chunk, dtype=np.float32) for chunk in audio_chunks])
else:
# Fallback: empty audio
audio = np.array([], dtype=np.float32)
return audio, sample_rate
# Run blocking inference in thread pool
audio, sample_rate = await asyncio.to_thread(_generate_sync)
return audio, sample_rate
class MLXSTTBackend:
"""MLX-based STT backend using mlx-audio Whisper."""
def __init__(self, model_size: str = "base"):
self.model = None
self.model_size = model_size
def is_loaded(self) -> bool:
"""Check if model is loaded."""
return self.model is not None
async def load_model_async(self, model_size: Optional[str] = None):
"""
Lazy load the MLX Whisper model.
Args:
model_size: Model size (tiny, base, small, medium, large)
"""
if model_size is None:
model_size = self.model_size
if self.model is not None and self.model_size == model_size:
return
# Run blocking load in thread pool
await asyncio.to_thread(self._load_model_sync, model_size)
# Alias for compatibility
load_model = load_model_async
def _load_model_sync(self, model_size: str):
"""Synchronous model loading."""
try:
from mlx_audio.asr import load
# MLX Whisper model naming
model_name = f"mlx-community/whisper-{model_size}"
# Set up progress tracking
progress_manager = get_progress_manager()
progress_model_name = f"whisper-{model_size}"
# Start tracking download task
task_manager = get_task_manager()
task_manager.start_download(progress_model_name)
print(f"Loading MLX Whisper model {model_size}...")
# Initialize progress state
progress_manager.update_progress(
model_name=progress_model_name,
current=0,
total=1,
filename="",
status="downloading",
)
# Set up progress callback
progress_callback = create_hf_progress_callback(progress_model_name, progress_manager)
tracker = HFProgressTracker(progress_callback)
# Use progress tracker during download
with tracker.patch_download():
self.model = load(model_name)
self.model_size = model_size
# Mark as complete
progress_manager.mark_complete(progress_model_name)
task_manager.complete_download(progress_model_name)
print(f"MLX Whisper model {model_size} loaded successfully")
except ImportError as e:
print(f"Error: mlx_audio package not found. Install with: pip install mlx-audio")
progress_manager = get_progress_manager()
task_manager = get_task_manager()
progress_model_name = f"whisper-{model_size}"
progress_manager.mark_error(progress_model_name, str(e))
task_manager.error_download(progress_model_name, str(e))
raise
except Exception as e:
print(f"Error loading MLX Whisper model: {e}")
progress_manager = get_progress_manager()
task_manager = get_task_manager()
progress_model_name = f"whisper-{model_size}"
progress_manager.mark_error(progress_model_name, str(e))
task_manager.error_download(progress_model_name, str(e))
raise
def unload_model(self):
"""Unload the model to free memory."""
if self.model is not None:
del self.model
self.model = None
print("MLX Whisper model unloaded")
async def transcribe(
self,
audio_path: str,
language: Optional[str] = None,
) -> str:
"""
Transcribe audio to text.
Args:
audio_path: Path to audio file
language: Optional language hint (en or zh)
Returns:
Transcribed text
"""
await self.load_model_async(None)
def _transcribe_sync():
"""Run synchronous transcription in thread pool."""
# Load audio
audio, sr = load_audio(audio_path, sample_rate=16000)
# MLX Whisper transcription
# The API may vary - check mlx-audio documentation
# For now, assuming similar API to PyTorch Whisper
result = self.model.transcribe(audio, language=language)
# Extract text from result (format may vary)
if isinstance(result, str):
return result.strip()
elif isinstance(result, dict):
return result.get("text", "").strip()
else:
# Try to get text attribute
return str(result).strip()
# Run blocking transcription in thread pool
return await asyncio.to_thread(_transcribe_sync)
+455
View File
@@ -0,0 +1,455 @@
"""
PyTorch backend implementation for TTS and STT.
"""
from typing import Optional, List, Tuple
import asyncio
import torch
import numpy as np
from pathlib import Path
from . import TTSBackend, STTBackend
from ..utils.cache import get_cache_key, get_cached_voice_prompt, cache_voice_prompt
from ..utils.audio import normalize_audio, load_audio
from ..utils.progress import get_progress_manager
from ..utils.hf_progress import HFProgressTracker, create_hf_progress_callback
from ..utils.tasks import get_task_manager
class PyTorchTTSBackend:
"""PyTorch-based TTS backend using Qwen3-TTS."""
def __init__(self, model_size: str = "1.7B"):
self.model = None
self.model_size = model_size
self.device = self._get_device()
self._current_model_size = None
def _get_device(self) -> str:
"""Get the best available device."""
if torch.cuda.is_available():
return "cuda"
elif hasattr(torch.backends, 'mps') and torch.backends.mps.is_available():
# MPS can have issues, use CPU for stability
return "cpu"
return "cpu"
def is_loaded(self) -> bool:
"""Check if model is loaded."""
return self.model is not None
def _get_model_path(self, model_size: str) -> str:
"""
Get the HuggingFace Hub model ID.
Args:
model_size: Model size (1.7B or 0.6B)
Returns:
HuggingFace Hub model ID
"""
hf_model_map = {
"1.7B": "Qwen/Qwen3-TTS-12Hz-1.7B-Base",
"0.6B": "Qwen/Qwen3-TTS-12Hz-0.6B-Base",
}
if model_size not in hf_model_map:
raise ValueError(f"Unknown model size: {model_size}")
return hf_model_map[model_size]
async def load_model_async(self, model_size: Optional[str] = None):
"""
Lazy load the TTS model with automatic downloading from HuggingFace Hub.
Args:
model_size: Model size to load (1.7B or 0.6B)
"""
if model_size is None:
model_size = self.model_size
# If already loaded with correct size, return
if self.model is not None and self._current_model_size == model_size:
return
# Unload existing model if different size requested
if self.model is not None and self._current_model_size != model_size:
self.unload_model()
# Run blocking load in thread pool
await asyncio.to_thread(self._load_model_sync, model_size)
# Alias for compatibility
load_model = load_model_async
def _load_model_sync(self, model_size: str):
"""Synchronous model loading."""
try:
from qwen_tts import Qwen3TTSModel
# Get model path (local or HuggingFace Hub ID)
model_path = self._get_model_path(model_size)
# Set up progress tracking
progress_manager = get_progress_manager()
model_name = f"qwen-tts-{model_size}"
print(f"Loading TTS model {model_size} on {self.device}...")
# Start tracking download task
task_manager = get_task_manager()
task_manager.start_download(model_name)
# Initialize progress state to show download has started
progress_manager.update_progress(
model_name=model_name,
current=0,
total=1, # Set to 1 initially, will be updated by callback
filename="",
status="downloading",
)
# Set up progress callback
progress_callback = create_hf_progress_callback(model_name, progress_manager)
tracker = HFProgressTracker(progress_callback)
# Use progress tracker during download
with tracker.patch_download():
# Load the model - downloads will happen automatically with progress tracking
self.model = Qwen3TTSModel.from_pretrained(
model_path,
device_map=self.device,
torch_dtype=torch.float32 if self.device == "cpu" else torch.bfloat16,
)
# Mark as complete
progress_manager.mark_complete(model_name)
task_manager.complete_download(model_name)
self._current_model_size = model_size
self.model_size = model_size
print(f"TTS model {model_size} loaded successfully")
except ImportError as e:
print(f"Error: qwen_tts package not found. Install with: pip install git+https://github.com/QwenLM/Qwen3-TTS.git")
progress_manager = get_progress_manager()
task_manager = get_task_manager()
model_name = f"qwen-tts-{model_size}"
progress_manager.mark_error(model_name, str(e))
task_manager.error_download(model_name, str(e))
raise
except Exception as e:
print(f"Error loading TTS model: {e}")
print(f"Tip: The model will be automatically downloaded from HuggingFace Hub on first use.")
progress_manager = get_progress_manager()
task_manager = get_task_manager()
model_name = f"qwen-tts-{model_size}"
progress_manager.mark_error(model_name, str(e))
task_manager.error_download(model_name, str(e))
raise
def unload_model(self):
"""Unload the model to free memory."""
if self.model is not None:
del self.model
self.model = None
self._current_model_size = None
if torch.cuda.is_available():
torch.cuda.empty_cache()
print("TTS model unloaded")
async def create_voice_prompt(
self,
audio_path: str,
reference_text: str,
use_cache: bool = True,
) -> Tuple[dict, bool]:
"""
Create voice prompt from reference audio.
Args:
audio_path: Path to reference audio file
reference_text: Transcript of reference audio
use_cache: Whether to use cached prompt if available
Returns:
Tuple of (voice_prompt_dict, was_cached)
"""
await self.load_model_async(None)
# Check cache if enabled
if use_cache:
cache_key = get_cache_key(audio_path, reference_text)
cached_prompt = get_cached_voice_prompt(cache_key)
if cached_prompt is not None:
# Cache stores as torch.Tensor but actual prompt is dict
# Convert if needed
if isinstance(cached_prompt, dict):
return cached_prompt, True
elif isinstance(cached_prompt, torch.Tensor):
# Legacy cache format - convert to dict
# This shouldn't happen in practice, but handle it
return {"prompt": cached_prompt}, True
def _create_prompt_sync():
"""Run synchronous voice prompt creation in thread pool."""
return self.model.create_voice_clone_prompt(
ref_audio=str(audio_path),
ref_text=reference_text,
x_vector_only_mode=False,
)
# Run blocking operation in thread pool
voice_prompt_items = await asyncio.to_thread(_create_prompt_sync)
# Cache if enabled
if use_cache:
cache_key = get_cache_key(audio_path, reference_text)
cache_voice_prompt(cache_key, voice_prompt_items)
return voice_prompt_items, False
async def combine_voice_prompts(
self,
audio_paths: List[str],
reference_texts: List[str],
) -> Tuple[np.ndarray, str]:
"""
Combine multiple reference samples for better quality.
Args:
audio_paths: List of audio file paths
reference_texts: List of reference texts
Returns:
Tuple of (combined_audio, combined_text)
"""
combined_audio = []
for audio_path in audio_paths:
audio, sr = load_audio(audio_path)
audio = normalize_audio(audio)
combined_audio.append(audio)
# Concatenate audio
mixed = np.concatenate(combined_audio)
mixed = normalize_audio(mixed)
# Combine texts
combined_text = " ".join(reference_texts)
return mixed, combined_text
async def generate(
self,
text: str,
voice_prompt: dict,
language: str = "en",
seed: Optional[int] = None,
instruct: Optional[str] = None,
) -> Tuple[np.ndarray, int]:
"""
Generate audio from text using voice prompt.
Args:
text: Text to synthesize
voice_prompt: Voice prompt dictionary from create_voice_prompt
language: Language code (en or zh)
seed: Random seed for reproducibility
instruct: Natural language instruction for speech delivery control
Returns:
Tuple of (audio_array, sample_rate)
"""
# Load model
await self.load_model_async(None)
def _generate_sync():
"""Run synchronous generation in thread pool."""
# Set seed if provided
if seed is not None:
torch.manual_seed(seed)
if torch.cuda.is_available():
torch.cuda.manual_seed(seed)
# Generate audio - this is the blocking operation
wavs, sample_rate = self.model.generate_voice_clone(
text=text,
voice_clone_prompt=voice_prompt,
instruct=instruct,
)
return wavs[0], sample_rate
# Run blocking inference in thread pool to avoid blocking event loop
audio, sample_rate = await asyncio.to_thread(_generate_sync)
return audio, sample_rate
class PyTorchSTTBackend:
"""PyTorch-based STT backend using Whisper."""
def __init__(self, model_size: str = "base"):
self.model = None
self.processor = None
self.model_size = model_size
self.device = self._get_device()
def _get_device(self) -> str:
"""Get the best available device."""
if torch.cuda.is_available():
return "cuda"
elif hasattr(torch.backends, 'mps') and torch.backends.mps.is_available():
# MPS support for Whisper
return "cpu" # Use CPU for stability
return "cpu"
def is_loaded(self) -> bool:
"""Check if model is loaded."""
return self.model is not None
async def load_model_async(self, model_size: Optional[str] = None):
"""
Lazy load the Whisper model.
Args:
model_size: Model size (tiny, base, small, medium, large)
"""
if model_size is None:
model_size = self.model_size
if self.model is not None and self.model_size == model_size:
return
# Run blocking load in thread pool
await asyncio.to_thread(self._load_model_sync, model_size)
# Alias for compatibility
load_model = load_model_async
def _load_model_sync(self, model_size: str):
"""Synchronous model loading."""
try:
from transformers import WhisperProcessor, WhisperForConditionalGeneration
model_name = f"openai/whisper-{model_size}"
# Set up progress tracking
progress_manager = get_progress_manager()
progress_model_name = f"whisper-{model_size}"
# Start tracking download task
task_manager = get_task_manager()
task_manager.start_download(progress_model_name)
print(f"Loading Whisper model {model_size} on {self.device}...")
# Initialize progress state to show download has started
progress_manager.update_progress(
model_name=progress_model_name,
current=0,
total=1, # Set to 1 initially, will be updated by callback
filename="",
status="downloading",
)
# Set up progress callback
progress_callback = create_hf_progress_callback(progress_model_name, progress_manager)
tracker = HFProgressTracker(progress_callback)
# Use progress tracker during download
with tracker.patch_download():
self.processor = WhisperProcessor.from_pretrained(model_name)
self.model = WhisperForConditionalGeneration.from_pretrained(model_name)
self.model.to(self.device)
self.model_size = model_size
# Mark as complete
progress_manager.mark_complete(progress_model_name)
task_manager.complete_download(progress_model_name)
print(f"Whisper model {model_size} loaded successfully")
except Exception as e:
print(f"Error loading Whisper model: {e}")
progress_manager = get_progress_manager()
task_manager = get_task_manager()
progress_model_name = f"whisper-{model_size}"
progress_manager.mark_error(progress_model_name, str(e))
task_manager.error_download(progress_model_name, str(e))
raise
def unload_model(self):
"""Unload the model to free memory."""
if self.model is not None:
del self.model
del self.processor
self.model = None
self.processor = None
if torch.cuda.is_available():
torch.cuda.empty_cache()
print("Whisper model unloaded")
async def transcribe(
self,
audio_path: str,
language: Optional[str] = None,
) -> str:
"""
Transcribe audio to text.
Args:
audio_path: Path to audio file
language: Optional language hint (en or zh)
Returns:
Transcribed text
"""
await self.load_model_async(None)
def _transcribe_sync():
"""Run synchronous transcription in thread pool."""
# Load audio
audio, sr = load_audio(audio_path, sample_rate=16000)
# Process audio
inputs = self.processor(
audio,
sampling_rate=16000,
return_tensors="pt",
)
inputs = inputs.to(self.device)
# Set language if provided
forced_decoder_ids = None
if language:
# Support all languages from frontend: en, zh, ja, ko, de, fr, ru, pt, es, it
# Whisper supports these and many more
forced_decoder_ids = self.processor.get_decoder_prompt_ids(
language=language,
task="transcribe",
)
# Generate transcription
with torch.no_grad():
predicted_ids = self.model.generate(
inputs["input_features"],
forced_decoder_ids=forced_decoder_ids,
)
# Decode
transcription = self.processor.batch_decode(
predicted_ids,
skip_special_tokens=True,
)[0]
return transcription.strip()
# Run blocking transcription in thread pool
return await asyncio.to_thread(_transcribe_sync)
+41 -20
View File
@@ -4,16 +4,19 @@ PyInstaller build script for creating standalone Python server binary.
import PyInstaller.__main__
import os
import platform
from pathlib import Path
def is_apple_silicon():
"""Check if running on Apple Silicon."""
return platform.system() == "Darwin" and platform.machine() == "arm64"
def build_server():
"""Build Python server as standalone binary."""
backend_dir = Path(__file__).parent
# Check for local editable qwen_tts install
local_qwen_path = Path.home() / 'Projects' / 'voice' / 'Qwen3-TTS'
# PyInstaller arguments
args = [
'server.py', # Use server.py as entry point instead of main.py
@@ -21,24 +24,13 @@ def build_server():
'--name', 'voicebox-server',
]
# Add local qwen_tts path if it exists (for editable installs)
if local_qwen_path.exists():
args.extend(['--paths', str(local_qwen_path)])
print(f"Using local qwen_tts source from: {local_qwen_path}")
# Add local qwen_tts path if specified (for editable installs)
qwen_tts_path = os.getenv('QWEN_TTS_PATH')
if qwen_tts_path and Path(qwen_tts_path).exists():
args.extend(['--paths', str(qwen_tts_path)])
print(f"Using local qwen_tts source from: {qwen_tts_path}")
# Exclude unnecessary modules to reduce size
args.extend([
'--exclude-module', 'matplotlib',
'--exclude-module', 'IPython',
'--exclude-module', 'notebook',
'--exclude-module', 'pytest',
'--exclude-module', 'setuptools',
'--exclude-module', 'torch.distributions',
'--exclude-module', 'torch.testing',
'--exclude-module', 'tensorboard',
])
# Add hidden imports
# Add common hidden imports
args.extend([
'--hidden-import', 'backend',
'--hidden-import', 'backend.main',
@@ -49,6 +41,9 @@ def build_server():
'--hidden-import', 'backend.history',
'--hidden-import', 'backend.tts',
'--hidden-import', 'backend.transcribe',
'--hidden-import', 'backend.platform_detect',
'--hidden-import', 'backend.backends',
'--hidden-import', 'backend.backends.pytorch_backend',
'--hidden-import', 'backend.utils.audio',
'--hidden-import', 'backend.utils.cache',
'--hidden-import', 'backend.utils.progress',
@@ -70,6 +65,32 @@ def build_server():
'--copy-metadata', 'qwen-tts',
'--collect-submodules', 'qwen_tts',
'--collect-data', 'qwen_tts',
# Fix for pkg_resources and jaraco namespace packages
'--hidden-import', 'pkg_resources.extern',
'--collect-submodules', 'jaraco',
])
# Add MLX-specific imports if building on Apple Silicon
if is_apple_silicon():
print("Building for Apple Silicon - including MLX dependencies")
args.extend([
'--hidden-import', 'backend.backends.mlx_backend',
'--hidden-import', 'mlx',
'--hidden-import', 'mlx.core',
'--hidden-import', 'mlx.nn',
'--hidden-import', 'mlx_audio',
'--hidden-import', 'mlx_audio.tts',
'--hidden-import', 'mlx_audio.asr',
'--collect-submodules', 'mlx',
'--collect-submodules', 'mlx_audio',
# Collect MLX data files including Metal shader libraries (.metallib)
'--collect-data', 'mlx',
'--collect-data', 'mlx_audio',
])
else:
print("Building for non-Apple Silicon platform - PyTorch only")
args.extend([
'--noconfirm',
'--clean',
])
+263
View File
@@ -0,0 +1,263 @@
"""
Audio channel management module.
"""
from typing import List, Optional
from datetime import datetime
import uuid
from sqlalchemy.orm import Session
from .models import (
AudioChannelCreate,
AudioChannelUpdate,
AudioChannelResponse,
ChannelVoiceAssignment,
ProfileChannelAssignment,
)
from .database import (
AudioChannel as DBAudioChannel,
ChannelDeviceMapping as DBChannelDeviceMapping,
ProfileChannelMapping as DBProfileChannelMapping,
VoiceProfile as DBVoiceProfile,
)
async def list_channels(db: Session) -> List[AudioChannelResponse]:
"""List all audio channels."""
channels = db.query(DBAudioChannel).all()
result = []
for channel in channels:
# Get device IDs for this channel
device_mappings = db.query(DBChannelDeviceMapping).filter_by(
channel_id=channel.id
).all()
device_ids = [m.device_id for m in device_mappings]
result.append(AudioChannelResponse(
id=channel.id,
name=channel.name,
is_default=channel.is_default,
device_ids=device_ids,
created_at=channel.created_at,
))
return result
async def get_channel(channel_id: str, db: Session) -> Optional[AudioChannelResponse]:
"""Get a channel by ID."""
channel = db.query(DBAudioChannel).filter_by(id=channel_id).first()
if not channel:
return None
# Get device IDs
device_mappings = db.query(DBChannelDeviceMapping).filter_by(
channel_id=channel.id
).all()
device_ids = [m.device_id for m in device_mappings]
return AudioChannelResponse(
id=channel.id,
name=channel.name,
is_default=channel.is_default,
device_ids=device_ids,
created_at=channel.created_at,
)
async def create_channel(
data: AudioChannelCreate,
db: Session,
) -> AudioChannelResponse:
"""Create a new audio channel."""
# Check if name already exists
existing = db.query(DBAudioChannel).filter_by(name=data.name).first()
if existing:
raise ValueError(f"Channel with name '{data.name}' already exists")
# Create channel
channel = DBAudioChannel(
id=str(uuid.uuid4()),
name=data.name,
is_default=False,
created_at=datetime.utcnow(),
)
db.add(channel)
db.flush()
# Add device mappings
for device_id in data.device_ids:
mapping = DBChannelDeviceMapping(
id=str(uuid.uuid4()),
channel_id=channel.id,
device_id=device_id,
)
db.add(mapping)
db.commit()
db.refresh(channel)
return AudioChannelResponse(
id=channel.id,
name=channel.name,
is_default=channel.is_default,
device_ids=data.device_ids,
created_at=channel.created_at,
)
async def update_channel(
channel_id: str,
data: AudioChannelUpdate,
db: Session,
) -> Optional[AudioChannelResponse]:
"""Update an audio channel."""
channel = db.query(DBAudioChannel).filter_by(id=channel_id).first()
if not channel:
return None
if channel.is_default:
raise ValueError("Cannot modify the default channel")
# Update name if provided
if data.name is not None:
# Check if name already exists (excluding current channel)
existing = db.query(DBAudioChannel).filter(
DBAudioChannel.name == data.name,
DBAudioChannel.id != channel_id
).first()
if existing:
raise ValueError(f"Channel with name '{data.name}' already exists")
channel.name = data.name
# Update device mappings if provided
if data.device_ids is not None:
# Delete existing mappings
db.query(DBChannelDeviceMapping).filter_by(channel_id=channel_id).delete()
# Add new mappings
for device_id in data.device_ids:
mapping = DBChannelDeviceMapping(
id=str(uuid.uuid4()),
channel_id=channel.id,
device_id=device_id,
)
db.add(mapping)
db.commit()
db.refresh(channel)
# Get updated device IDs
device_mappings = db.query(DBChannelDeviceMapping).filter_by(
channel_id=channel.id
).all()
device_ids = [m.device_id for m in device_mappings]
return AudioChannelResponse(
id=channel.id,
name=channel.name,
is_default=channel.is_default,
device_ids=device_ids,
created_at=channel.created_at,
)
async def delete_channel(channel_id: str, db: Session) -> bool:
"""Delete an audio channel."""
channel = db.query(DBAudioChannel).filter_by(id=channel_id).first()
if not channel:
return False
if channel.is_default:
raise ValueError("Cannot delete the default channel")
# Delete device mappings
db.query(DBChannelDeviceMapping).filter_by(channel_id=channel_id).delete()
# Delete profile-channel mappings
db.query(DBProfileChannelMapping).filter_by(channel_id=channel_id).delete()
# Delete channel
db.delete(channel)
db.commit()
return True
async def get_channel_voices(channel_id: str, db: Session) -> List[str]:
"""Get list of profile IDs assigned to a channel."""
mappings = db.query(DBProfileChannelMapping).filter_by(
channel_id=channel_id
).all()
return [m.profile_id for m in mappings]
async def set_channel_voices(
channel_id: str,
data: ChannelVoiceAssignment,
db: Session,
) -> None:
"""Set which voices are assigned to a channel."""
# Verify channel exists
channel = db.query(DBAudioChannel).filter_by(id=channel_id).first()
if not channel:
raise ValueError(f"Channel {channel_id} not found")
# Verify all profiles exist
for profile_id in data.profile_ids:
profile = db.query(DBVoiceProfile).filter_by(id=profile_id).first()
if not profile:
raise ValueError(f"Profile {profile_id} not found")
# Delete existing mappings for this channel
db.query(DBProfileChannelMapping).filter_by(channel_id=channel_id).delete()
# Add new mappings
for profile_id in data.profile_ids:
mapping = DBProfileChannelMapping(
profile_id=profile_id,
channel_id=channel_id,
)
db.add(mapping)
db.commit()
async def get_profile_channels(profile_id: str, db: Session) -> List[str]:
"""Get list of channel IDs assigned to a profile."""
mappings = db.query(DBProfileChannelMapping).filter_by(
profile_id=profile_id
).all()
return [m.channel_id for m in mappings]
async def set_profile_channels(
profile_id: str,
data: ProfileChannelAssignment,
db: Session,
) -> None:
"""Set which channels a profile is assigned to."""
# Verify profile exists
profile = db.query(DBVoiceProfile).filter_by(id=profile_id).first()
if not profile:
raise ValueError(f"Profile {profile_id} not found")
# Verify all channels exist
for channel_id in data.channel_ids:
channel = db.query(DBAudioChannel).filter_by(id=channel_id).first()
if not channel:
raise ValueError(f"Channel {channel_id} not found")
# Delete existing mappings for this profile
db.query(DBProfileChannelMapping).filter_by(profile_id=profile_id).delete()
# Add new mappings
for channel_id in data.channel_ids:
mapping = DBProfileChannelMapping(
profile_id=profile_id,
channel_id=channel_id,
)
db.add(mapping)
db.commit()
+207 -2
View File
@@ -2,7 +2,7 @@
SQLite database ORM using SQLAlchemy.
"""
from sqlalchemy import create_engine, Column, String, Integer, Float, DateTime, Text, ForeignKey
from sqlalchemy import create_engine, Column, String, Integer, Float, DateTime, Text, ForeignKey, Boolean
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import sessionmaker, Session
from datetime import datetime
@@ -17,11 +17,12 @@ Base = declarative_base()
class VoiceProfile(Base):
"""Voice profile database model."""
__tablename__ = "profiles"
id = Column(String, primary_key=True, default=lambda: str(uuid.uuid4()))
name = Column(String, unique=True, nullable=False)
description = Column(Text)
language = Column(String, default="en")
avatar_path = Column(String, nullable=True)
created_at = Column(DateTime, default=datetime.utcnow)
updated_at = Column(DateTime, default=datetime.utcnow, onupdate=datetime.utcnow)
@@ -51,6 +52,31 @@ class Generation(Base):
created_at = Column(DateTime, default=datetime.utcnow)
class Story(Base):
"""Story database model."""
__tablename__ = "stories"
id = Column(String, primary_key=True, default=lambda: str(uuid.uuid4()))
name = Column(String, nullable=False)
description = Column(Text)
created_at = Column(DateTime, default=datetime.utcnow)
updated_at = Column(DateTime, default=datetime.utcnow, onupdate=datetime.utcnow)
class StoryItem(Base):
"""Story item database model (links generations to stories)."""
__tablename__ = "story_items"
id = Column(String, primary_key=True, default=lambda: str(uuid.uuid4()))
story_id = Column(String, ForeignKey("stories.id"), nullable=False)
generation_id = Column(String, ForeignKey("generations.id"), nullable=False)
start_time_ms = Column(Integer, nullable=False, default=0) # Milliseconds from story start
track = Column(Integer, nullable=False, default=0) # Track number (0 = main track)
trim_start_ms = Column(Integer, nullable=False, default=0) # Milliseconds trimmed from start
trim_end_ms = Column(Integer, nullable=False, default=0) # Milliseconds trimmed from end
created_at = Column(DateTime, default=datetime.utcnow)
class Project(Base):
"""Audio studio project database model."""
__tablename__ = "projects"
@@ -62,6 +88,33 @@ class Project(Base):
updated_at = Column(DateTime, default=datetime.utcnow, onupdate=datetime.utcnow)
class AudioChannel(Base):
"""Audio channel (bus) database model."""
__tablename__ = "audio_channels"
id = Column(String, primary_key=True, default=lambda: str(uuid.uuid4()))
name = Column(String, nullable=False)
is_default = Column(Boolean, default=False)
created_at = Column(DateTime, default=datetime.utcnow)
class ChannelDeviceMapping(Base):
"""Mapping between channels and OS audio devices."""
__tablename__ = "channel_device_mappings"
id = Column(String, primary_key=True, default=lambda: str(uuid.uuid4()))
channel_id = Column(String, ForeignKey("audio_channels.id"), nullable=False)
device_id = Column(String, nullable=False) # OS device identifier
class ProfileChannelMapping(Base):
"""Mapping between voice profiles and audio channels (many-to-many)."""
__tablename__ = "profile_channel_mappings"
profile_id = Column(String, ForeignKey("profiles.id"), primary_key=True)
channel_id = Column(String, ForeignKey("audio_channels.id"), primary_key=True)
# Database setup will be initialized in init_db()
engine = None
SessionLocal = None
@@ -81,7 +134,159 @@ def init_db():
)
SessionLocal = sessionmaker(autocommit=False, autoflush=False, bind=engine)
# Run migrations before creating tables
_run_migrations(engine)
Base.metadata.create_all(bind=engine)
# Create default channel if it doesn't exist
db = SessionLocal()
try:
default_channel = db.query(AudioChannel).filter(AudioChannel.is_default == True).first()
if not default_channel:
default_channel = AudioChannel(
id=str(uuid.uuid4()),
name="Default",
is_default=True
)
db.add(default_channel)
# Assign all existing profiles to default channel
profiles = db.query(VoiceProfile).all()
for profile in profiles:
mapping = ProfileChannelMapping(
profile_id=profile.id,
channel_id=default_channel.id
)
db.add(mapping)
db.commit()
finally:
db.close()
def _run_migrations(engine):
"""Run database migrations."""
from sqlalchemy import inspect, text
inspector = inspect(engine)
# Check if story_items table exists
if 'story_items' not in inspector.get_table_names():
return # Table doesn't exist yet, will be created fresh
# Get columns in story_items table
columns = {col['name'] for col in inspector.get_columns('story_items')}
# Migration: Remove position column and ensure start_time_ms exists
# SQLite doesn't support DROP COLUMN easily, so we recreate the table
if 'position' in columns:
print("Migrating story_items: removing position column, using start_time_ms")
with engine.connect() as conn:
# Check if start_time_ms already exists
has_start_time = 'start_time_ms' in columns
if not has_start_time:
# First, add the new column temporarily
conn.execute(text("ALTER TABLE story_items ADD COLUMN start_time_ms INTEGER DEFAULT 0"))
# Calculate timecodes from position ordering
result = conn.execute(text("""
SELECT si.id, si.story_id, si.position, g.duration
FROM story_items si
JOIN generations g ON si.generation_id = g.id
ORDER BY si.story_id, si.position
"""))
rows = result.fetchall()
current_story_id = None
current_time_ms = 0
for row in rows:
item_id, story_id, position, duration = row
if story_id != current_story_id:
current_story_id = story_id
current_time_ms = 0
conn.execute(
text("UPDATE story_items SET start_time_ms = :time WHERE id = :id"),
{"time": current_time_ms, "id": item_id}
)
current_time_ms += int(duration * 1000) + 200
conn.commit()
# Now recreate the table without the position column
# 1. Create new table
conn.execute(text("""
CREATE TABLE story_items_new (
id VARCHAR PRIMARY KEY,
story_id VARCHAR NOT NULL,
generation_id VARCHAR NOT NULL,
start_time_ms INTEGER NOT NULL DEFAULT 0,
created_at DATETIME,
FOREIGN KEY (story_id) REFERENCES stories(id),
FOREIGN KEY (generation_id) REFERENCES generations(id)
)
"""))
# 2. Copy data
conn.execute(text("""
INSERT INTO story_items_new (id, story_id, generation_id, start_time_ms, created_at)
SELECT id, story_id, generation_id, start_time_ms, created_at FROM story_items
"""))
# 3. Drop old table
conn.execute(text("DROP TABLE story_items"))
# 4. Rename new table
conn.execute(text("ALTER TABLE story_items_new RENAME TO story_items"))
conn.commit()
print("Migrated story_items table to use start_time_ms (removed position column)")
# Migration: Add track column if it doesn't exist
# Re-check columns after potential position migration
columns = {col['name'] for col in inspector.get_columns('story_items')}
if 'track' not in columns:
print("Migrating story_items: adding track column")
with engine.connect() as conn:
conn.execute(text("ALTER TABLE story_items ADD COLUMN track INTEGER NOT NULL DEFAULT 0"))
conn.commit()
print("Added track column to story_items")
# Migration: Add trim columns if they don't exist
# Re-check columns after potential track migration
columns = {col['name'] for col in inspector.get_columns('story_items')}
if 'trim_start_ms' not in columns:
print("Migrating story_items: adding trim_start_ms column")
with engine.connect() as conn:
conn.execute(text("ALTER TABLE story_items ADD COLUMN trim_start_ms INTEGER NOT NULL DEFAULT 0"))
conn.commit()
print("Added trim_start_ms column to story_items")
columns = {col['name'] for col in inspector.get_columns('story_items')}
if 'trim_end_ms' not in columns:
print("Migrating story_items: adding trim_end_ms column")
with engine.connect() as conn:
conn.execute(text("ALTER TABLE story_items ADD COLUMN trim_end_ms INTEGER NOT NULL DEFAULT 0"))
conn.commit()
print("Added trim_end_ms column to story_items")
# Migration: Add avatar_path to profiles table
if 'profiles' in inspector.get_table_names():
columns = {col['name'] for col in inspector.get_columns('profiles')}
if 'avatar_path' not in columns:
print("Migrating profiles: adding avatar_path column")
with engine.connect() as conn:
conn.execute(text("ALTER TABLE profiles ADD COLUMN avatar_path VARCHAR"))
conn.commit()
print("Added avatar_path column to profiles")
def get_db():
+438
View File
@@ -0,0 +1,438 @@
"""
Voice profile export/import module.
Handles exporting profiles to ZIP archives and importing them back.
Also handles exporting individual generations.
"""
import json
import zipfile
import io
from pathlib import Path
from typing import Optional
from sqlalchemy.orm import Session
from .models import VoiceProfileResponse
from .database import VoiceProfile as DBVoiceProfile, ProfileSample as DBProfileSample, Generation as DBGeneration
from .profiles import create_profile, add_profile_sample
from .models import VoiceProfileCreate
from . import config
def _get_profiles_dir() -> Path:
"""Get profiles directory from config."""
return config.get_profiles_dir()
def _get_unique_profile_name(name: str, db: Session) -> str:
"""
Get a unique profile name by appending a number if needed.
Args:
name: Original profile name
db: Database session
Returns:
Unique profile name
"""
base_name = name
counter = 1
while True:
existing = db.query(DBVoiceProfile).filter_by(name=name).first()
if not existing:
return name
name = f"{base_name} ({counter})"
counter += 1
def export_profile_to_zip(profile_id: str, db: Session) -> bytes:
"""
Export a voice profile to a ZIP archive.
Args:
profile_id: Profile ID to export
db: Database session
Returns:
ZIP file contents as bytes
Raises:
ValueError: If profile not found or has no samples
"""
# Get profile
profile = db.query(DBVoiceProfile).filter_by(id=profile_id).first()
if not profile:
raise ValueError(f"Profile {profile_id} not found")
# Get all samples
samples = db.query(DBProfileSample).filter_by(profile_id=profile_id).all()
if not samples:
raise ValueError(f"Profile {profile_id} has no samples")
# Create ZIP in memory
zip_buffer = io.BytesIO()
with zipfile.ZipFile(zip_buffer, 'w', zipfile.ZIP_DEFLATED) as zip_file:
# Check if profile has avatar
has_avatar = False
if profile.avatar_path:
avatar_path = Path(profile.avatar_path)
if avatar_path.exists():
has_avatar = True
# Add avatar to ZIP root with original extension
avatar_ext = avatar_path.suffix
zip_file.write(avatar_path, f"avatar{avatar_ext}")
# Create manifest.json
manifest = {
"version": "1.0",
"profile": {
"name": profile.name,
"description": profile.description,
"language": profile.language,
},
"has_avatar": has_avatar,
}
zip_file.writestr("manifest.json", json.dumps(manifest, indent=2))
# Create samples.json mapping
samples_data = {}
profile_dir = _get_profiles_dir() / profile_id
for sample in samples:
# Get filename from audio_path (should be {sample_id}.wav)
audio_path = Path(sample.audio_path)
filename = audio_path.name
# Read audio file
if not audio_path.exists():
raise ValueError(f"Audio file not found: {audio_path}")
# Add to samples directory in ZIP
zip_path = f"samples/{filename}"
zip_file.write(audio_path, zip_path)
# Map filename to reference text
samples_data[filename] = sample.reference_text
zip_file.writestr("samples.json", json.dumps(samples_data, indent=2))
zip_buffer.seek(0)
return zip_buffer.read()
async def import_profile_from_zip(file_bytes: bytes, db: Session) -> VoiceProfileResponse:
"""
Import a voice profile from a ZIP archive.
Args:
file_bytes: ZIP file contents
db: Database session
Returns:
Created profile
Raises:
ValueError: If ZIP is invalid or missing required files
"""
zip_buffer = io.BytesIO(file_bytes)
try:
with zipfile.ZipFile(zip_buffer, 'r') as zip_file:
# Validate ZIP structure
namelist = zip_file.namelist()
if "manifest.json" not in namelist:
raise ValueError("ZIP archive missing manifest.json")
if "samples.json" not in namelist:
raise ValueError("ZIP archive missing samples.json")
# Read manifest
manifest_data = json.loads(zip_file.read("manifest.json"))
if "version" not in manifest_data:
raise ValueError("Invalid manifest.json: missing version")
if "profile" not in manifest_data:
raise ValueError("Invalid manifest.json: missing profile")
profile_data = manifest_data["profile"]
# Read samples mapping
samples_data = json.loads(zip_file.read("samples.json"))
if not isinstance(samples_data, dict):
raise ValueError("Invalid samples.json: must be a dictionary")
# Get unique profile name
original_name = profile_data.get("name", "Imported Profile")
unique_name = _get_unique_profile_name(original_name, db)
# Create profile
profile_create = VoiceProfileCreate(
name=unique_name,
description=profile_data.get("description"),
language=profile_data.get("language", "en"),
)
profile = await create_profile(profile_create, db)
# Extract and add samples
profile_dir = _get_profiles_dir() / profile.id
profile_dir.mkdir(parents=True, exist_ok=True)
# Handle avatar if present
avatar_files = [f for f in namelist if f.startswith("avatar.")]
if avatar_files:
try:
avatar_file = avatar_files[0]
# Extract to temporary file
import tempfile
with tempfile.NamedTemporaryFile(suffix=Path(avatar_file).suffix, delete=False) as tmp:
tmp.write(zip_file.read(avatar_file))
tmp_path = tmp.name
try:
from .profiles import upload_avatar
await upload_avatar(profile.id, tmp_path, db)
finally:
Path(tmp_path).unlink(missing_ok=True)
except Exception as e:
# Avatar import is optional - continue even if it fails
pass
for filename, reference_text in samples_data.items():
# Validate filename
if not filename.endswith('.wav'):
raise ValueError(f"Invalid sample filename: {filename} (must be .wav)")
# Extract audio file to temp location
zip_path = f"samples/{filename}"
if zip_path not in namelist:
raise ValueError(f"Sample file not found in ZIP: {zip_path}")
# Extract to temporary file
import tempfile
with tempfile.NamedTemporaryFile(suffix=".wav", delete=False) as tmp:
tmp.write(zip_file.read(zip_path))
tmp_path = tmp.name
try:
# Add sample to profile
await add_profile_sample(
profile.id,
tmp_path,
reference_text,
db,
)
finally:
# Clean up temp file
Path(tmp_path).unlink(missing_ok=True)
return profile
except zipfile.BadZipFile:
raise ValueError("Invalid ZIP file")
except json.JSONDecodeError as e:
raise ValueError(f"Invalid JSON in archive: {e}")
except Exception as e:
if isinstance(e, ValueError):
raise
raise ValueError(f"Error importing profile: {str(e)}")
def export_generation_to_zip(generation_id: str, db: Session) -> bytes:
"""
Export a generation to a ZIP archive.
Args:
generation_id: Generation ID to export
db: Database session
Returns:
ZIP file contents as bytes
Raises:
ValueError: If generation not found
"""
# Get generation
generation = db.query(DBGeneration).filter_by(id=generation_id).first()
if not generation:
raise ValueError(f"Generation {generation_id} not found")
# Get profile info
profile = db.query(DBVoiceProfile).filter_by(id=generation.profile_id).first()
if not profile:
raise ValueError(f"Profile {generation.profile_id} not found")
# Get audio file
audio_path = Path(generation.audio_path)
if not audio_path.exists():
raise ValueError(f"Audio file not found: {audio_path}")
# Create ZIP in memory
zip_buffer = io.BytesIO()
with zipfile.ZipFile(zip_buffer, 'w', zipfile.ZIP_DEFLATED) as zip_file:
# Create manifest.json
manifest = {
"version": "1.0",
"generation": {
"id": generation.id,
"text": generation.text,
"language": generation.language,
"duration": generation.duration,
"seed": generation.seed,
"instruct": generation.instruct,
"created_at": generation.created_at.isoformat(),
},
"profile": {
"id": profile.id,
"name": profile.name,
"description": profile.description,
"language": profile.language,
}
}
zip_file.writestr("manifest.json", json.dumps(manifest, indent=2))
# Add audio file
filename = audio_path.name
zip_file.write(audio_path, f"audio/{filename}")
zip_buffer.seek(0)
return zip_buffer.read()
async def import_generation_from_zip(file_bytes: bytes, db: Session) -> dict:
"""
Import a generation from a ZIP archive.
Args:
file_bytes: ZIP file contents
db: Database session
Returns:
Dictionary with generation ID and profile info
Raises:
ValueError: If ZIP is invalid or missing required files
"""
from pathlib import Path
import tempfile
import shutil
from datetime import datetime
from . import config
zip_buffer = io.BytesIO(file_bytes)
try:
with zipfile.ZipFile(zip_buffer, 'r') as zip_file:
# Validate ZIP structure
namelist = zip_file.namelist()
if "manifest.json" not in namelist:
raise ValueError("ZIP archive missing manifest.json")
# Read manifest
manifest_data = json.loads(zip_file.read("manifest.json"))
if "version" not in manifest_data:
raise ValueError("Invalid manifest.json: missing version")
if "generation" not in manifest_data:
raise ValueError("Invalid manifest.json: missing generation data")
generation_data = manifest_data["generation"]
profile_data = manifest_data.get("profile", {})
# Validate required fields
required_fields = ["text", "language", "duration"]
for field in required_fields:
if field not in generation_data:
raise ValueError(f"Invalid manifest.json: missing generation.{field}")
# Find audio file in archive
audio_files = [f for f in namelist if f.startswith("audio/") and f.endswith(".wav")]
if not audio_files:
raise ValueError("No audio file found in ZIP archive")
audio_file_path = audio_files[0]
# Check if we should match an existing profile or create metadata
profile_id = None
profile_name = profile_data.get("name", "Unknown Profile")
# Try to find matching profile by name
if profile_name and profile_name != "Unknown Profile":
existing_profile = db.query(DBVoiceProfile).filter_by(name=profile_name).first()
if existing_profile:
profile_id = existing_profile.id
# If no matching profile, use a placeholder or the first available profile
if not profile_id:
# Get any profile, or None if no profiles exist
any_profile = db.query(DBVoiceProfile).first()
if any_profile:
profile_id = any_profile.id
profile_name = any_profile.name
else:
raise ValueError("No voice profiles found. Please create a profile before importing generations.")
# Extract audio file to temporary location
with tempfile.NamedTemporaryFile(suffix=".wav", delete=False) as tmp:
tmp.write(zip_file.read(audio_file_path))
tmp_path = tmp.name
try:
# Create generations directory
generations_dir = config.get_generations_dir()
generations_dir.mkdir(parents=True, exist_ok=True)
# Generate new ID for this generation
new_generation_id = str(__import__('uuid').uuid4())
# Copy audio to generations directory
audio_dest = generations_dir / f"{new_generation_id}.wav"
shutil.copy(tmp_path, audio_dest)
# Create generation record
db_generation = DBGeneration(
id=new_generation_id,
profile_id=profile_id,
text=generation_data["text"],
language=generation_data["language"],
audio_path=str(audio_dest),
duration=generation_data["duration"],
seed=generation_data.get("seed"),
instruct=generation_data.get("instruct"),
created_at=datetime.utcnow(),
)
db.add(db_generation)
db.commit()
db.refresh(db_generation)
return {
"id": db_generation.id,
"profile_id": profile_id,
"profile_name": profile_name,
"text": db_generation.text,
"message": f"Generation imported successfully (assigned to profile: {profile_name})"
}
finally:
# Clean up temp file
Path(tmp_path).unlink(missing_ok=True)
except zipfile.BadZipFile:
raise ValueError("Invalid ZIP file")
except json.JSONDecodeError as e:
raise ValueError(f"Invalid JSON in archive: {e}")
except Exception as e:
if isinstance(e, ValueError):
raise
raise ValueError(f"Error importing generation: {str(e)}")
+928 -40
View File
File diff suppressed because it is too large Load Diff
+162 -2
View File
@@ -11,7 +11,7 @@ class VoiceProfileCreate(BaseModel):
"""Request model for creating a voice profile."""
name: str = Field(..., min_length=1, max_length=100)
description: Optional[str] = Field(None, max_length=500)
language: str = Field(default="en", pattern="^(en|zh)$")
language: str = Field(default="en", pattern="^(zh|en|ja|ko|de|fr|ru|pt|es|it)$")
class VoiceProfileResponse(BaseModel):
@@ -20,6 +20,7 @@ class VoiceProfileResponse(BaseModel):
name: str
description: Optional[str]
language: str
avatar_path: Optional[str] = None
created_at: datetime
updated_at: datetime
@@ -32,6 +33,11 @@ class ProfileSampleCreate(BaseModel):
reference_text: str = Field(..., min_length=1, max_length=1000)
class ProfileSampleUpdate(BaseModel):
"""Request model for updating a profile sample."""
reference_text: str = Field(..., min_length=1, max_length=1000)
class ProfileSampleResponse(BaseModel):
"""Response model for profile sample."""
id: str
@@ -47,7 +53,7 @@ class GenerationRequest(BaseModel):
"""Request model for voice generation."""
profile_id: str
text: str = Field(..., min_length=1, max_length=5000)
language: str = Field(default="en", pattern="^(en|zh)$")
language: str = Field(default="en", pattern="^(zh|en|ja|ko|de|fr|ru|pt|es|it)$")
seed: Optional[int] = Field(None, ge=0)
model_size: Optional[str] = Field(default="1.7B", pattern="^(1\\.7B|0\\.6B)$")
instruct: Optional[str] = Field(None, max_length=500)
@@ -118,7 +124,9 @@ class HealthResponse(BaseModel):
model_downloaded: Optional[bool] = None # Whether model is cached/downloaded
model_size: Optional[str] = None # Current model size if loaded
gpu_available: bool
gpu_type: Optional[str] = None # GPU type (CUDA, MPS, or None)
vram_used_mb: Optional[float] = None
backend_type: Optional[str] = None # Backend type (mlx or pytorch)
class ModelStatus(BaseModel):
@@ -138,3 +146,155 @@ class ModelStatusListResponse(BaseModel):
class ModelDownloadRequest(BaseModel):
"""Request model for triggering model download."""
model_name: str
class ActiveDownloadTask(BaseModel):
"""Response model for active download task."""
model_name: str
status: str
started_at: datetime
class ActiveGenerationTask(BaseModel):
"""Response model for active generation task."""
task_id: str
profile_id: str
text_preview: str
started_at: datetime
class ActiveTasksResponse(BaseModel):
"""Response model for active tasks."""
downloads: List[ActiveDownloadTask]
generations: List[ActiveGenerationTask]
class AudioChannelCreate(BaseModel):
"""Request model for creating an audio channel."""
name: str = Field(..., min_length=1, max_length=100)
device_ids: List[str] = Field(default_factory=list)
class AudioChannelUpdate(BaseModel):
"""Request model for updating an audio channel."""
name: Optional[str] = Field(None, min_length=1, max_length=100)
device_ids: Optional[List[str]] = None
class AudioChannelResponse(BaseModel):
"""Response model for audio channel."""
id: str
name: str
is_default: bool
device_ids: List[str]
created_at: datetime
class Config:
from_attributes = True
class ChannelVoiceAssignment(BaseModel):
"""Request model for assigning voices to a channel."""
profile_ids: List[str]
class ProfileChannelAssignment(BaseModel):
"""Request model for assigning channels to a profile."""
channel_ids: List[str]
class StoryCreate(BaseModel):
"""Request model for creating a story."""
name: str = Field(..., min_length=1, max_length=100)
description: Optional[str] = Field(None, max_length=500)
class StoryResponse(BaseModel):
"""Response model for story (list view)."""
id: str
name: str
description: Optional[str]
created_at: datetime
updated_at: datetime
item_count: int = 0
class Config:
from_attributes = True
class StoryItemDetail(BaseModel):
"""Detail model for story item with generation info."""
id: str
story_id: str
generation_id: str
start_time_ms: int
track: int = 0
trim_start_ms: int = 0
trim_end_ms: int = 0
created_at: datetime
# Generation details
profile_id: str
profile_name: str
text: str
language: str
audio_path: str
duration: float
seed: Optional[int]
instruct: Optional[str]
generation_created_at: datetime
class Config:
from_attributes = True
class StoryDetailResponse(BaseModel):
"""Response model for story with items."""
id: str
name: str
description: Optional[str]
created_at: datetime
updated_at: datetime
items: List[StoryItemDetail] = []
class Config:
from_attributes = True
class StoryItemCreate(BaseModel):
"""Request model for adding a generation to a story."""
generation_id: str
start_time_ms: Optional[int] = None # If not provided, will be calculated automatically
track: Optional[int] = 0 # Track number (0 = main track)
class StoryItemUpdateTime(BaseModel):
"""Request model for updating a story item's timecode."""
generation_id: str
start_time_ms: int = Field(..., ge=0)
class StoryItemBatchUpdate(BaseModel):
"""Request model for batch updating story item timecodes."""
updates: List[StoryItemUpdateTime]
class StoryItemReorder(BaseModel):
"""Request model for reordering story items."""
generation_ids: List[str] = Field(..., min_length=1)
class StoryItemMove(BaseModel):
"""Request model for moving a story item (position and/or track)."""
start_time_ms: int = Field(..., ge=0)
track: int = 0
class StoryItemTrim(BaseModel):
"""Request model for trimming a story item."""
trim_start_ms: int = Field(..., ge=0)
trim_end_ms: int = Field(..., ge=0)
class StoryItemSplit(BaseModel):
"""Request model for splitting a story item."""
split_time_ms: int = Field(..., ge=0) # Time within the clip to split at (relative to clip start)

Some files were not shown because too many files have changed in this diff Show More