Compare commits

...
Author SHA1 Message Date
Jamie Pine 229841e05e Add GPU type information to health check response
- Updated the health check endpoint to include the type of GPU available (CUDA or MPS).
- Modified the HealthResponse model to accommodate the new gpu_type field, enhancing the response with detailed GPU information.
- This change improves the clarity of system capabilities for users and developers.
2026-01-29 03:12:11 -08:00
Jamie Pine 123e8215e4 Merge branch 'main' into windows-server-shutdown 2026-01-29 03:00:08 -08:00
Jamie Pine 2a3afec2ca Implement graceful shutdown for the server and enhance process management on Windows
- Added a new `/shutdown` endpoint to allow graceful server shutdown via HTTP.
- Implemented process tree management functions to handle child processes during shutdown on Windows.
- Updated the `stop_server` function to attempt graceful shutdown before forcefully terminating processes.
- Enhanced error handling and logging for shutdown operations.
2026-01-29 02:58:21 -08:00
Jamie Pine 99ddd5a0b4 Add asynchronous model download handling for TTS and Whisper models
- Implemented background tasks for downloading TTS and Whisper models to prevent blocking HTTP responses.
- Enhanced error handling during model downloads, providing users with real-time feedback on download status.
- Updated HTTP responses to indicate when models are being downloaded, improving user experience during model initialization.
2026-01-29 02:55:17 -08:00
Jamie Pine 8d730621bc Refactor model download handling to use background tasks
- Moved model download logic into a separate asynchronous function to allow non-blocking HTTP responses.
- Improved error handling by tracking download status and reporting errors without interrupting the main request flow.
- The frontend is now expected to poll the progress endpoint for download status updates.
2026-01-29 02:42:02 -08:00
Jamie Pine e23118f610 Bump version: 0.1.7 → 0.1.8 2026-01-29 02:21:01 -08:00
Jamie Pine d4bfdc0d68 Update version handling in backend and improve HuggingFace cache management
- Added __version__ variable in backend/__init__.py to centralize versioning.
- Updated main.py to use __version__ for API versioning in the FastAPI app.
- Enhanced cache directory handling by utilizing HuggingFace's constants for improved compatibility across platforms.
2026-01-29 02:20:33 -08:00
Jamie Pine 116c108906 Update screenshot asset in landing page for consistency with current design 2026-01-29 00:06:06 -08:00
Jamie Pine 2d23c8e06a Swap screenshot assets in landing page for improved visual representation
- Replaced app screenshot paths to ensure correct images are displayed.
- Adjusted alt text for screenshots to accurately reflect their content.
2026-01-29 00:06:00 -08:00
Jamie Pine 3973a59ba3 Revise README to clarify Voicebox features and benefits
- Changed section title from "Why Voicebox?" to "What is Voicebox?" for better clarity.
- Expanded description to emphasize local-first voice cloning capabilities and professional tools.
- Highlighted privacy, model flexibility, and native performance as key advantages over cloud services.
2026-01-28 23:59:47 -08:00
Jamie Pine d9aa75253a Enhance README with new features and multi-track editor details
- Added multi-sample support for higher quality cloning.
- Introduced a new Stories Editor section with features for multi-track composition, inline audio editing, auto-playback, and voice mixing.
- Updated recording section to include system audio capture for macOS and Windows.
2026-01-28 23:55:44 -08:00
Jamie Pine b22bf36565 Update README and landing page with new screenshots; bump version to 0.1.7
- Replaced existing screenshot paths in README and landing page with new assets.
- Added additional screenshots to the landing page for enhanced visual representation.
- Updated version in Cargo.lock from 0.1.6 to 0.1.7.
2026-01-28 23:51:43 -08:00
Jamie Pine 33f4ed9b44 Bump version: 0.1.6 → 0.1.7 2026-01-28 22:28:18 -08:00
Jamie Pine cc37e04221 Refactor HistoryTable and SampleList components for improved code consistency
- Cleaned up formatting in HistoryTable for better readability.
- Adjusted import statements in SampleList to maintain consistent structure.
2026-01-28 22:28:01 -08:00
Jamie Pine 2b4fbe5173 Refactor AudioPlayer and related components to support conditional auto-play functionality
- Updated AudioPlayer to auto-play only if the shouldAutoPlay flag is set, enhancing user control over playback.
- Refactored HistoryTable, SampleList, and useGenerationForm to utilize setAudioWithAutoPlay for consistent audio loading and playback behavior.
- Improved user experience by ensuring audio is only played when explicitly intended, reducing unexpected playback.
2026-01-28 22:27:37 -08:00
Jamie Pine ea943876dc formatting 2026-01-28 22:23:27 -08:00
Jamie Pine b55d8cc567 Implement auto-activation of stories in StoryTrackEditor and improve playback state management
- Added useEffect to automatically activate the story when the editor is shown, ensuring the playhead is visible.
- Introduced setActiveStory function in storyStore to manage story activation without playback.
- Updated playback state checks to reflect the current playing status accurately.
- Enhanced UI to always display the playhead for better user experience during playback.
2026-01-28 22:22:47 -08:00
Jamie Pine 036d90dc8e Enhance story item management with trimming, splitting, and duplication features
- Updated StoryTrackEditor and StoryContent components to support trimming and splitting of story items.
- Introduced new API endpoints for trimming, splitting, and duplicating story items, enhancing item management capabilities.
- Refactored related hooks and state management to accommodate new functionalities.
- Improved data models to include trim start and end times for better audio playback control.
- Enhanced UI interactions for selecting and managing story items within the track editor.
2026-01-28 22:16:53 -08:00
Jamie PineandGitHub 27ae6dfbab Merge pull request #3 from jamiepine/stories
Stories
2026-01-28 21:18:19 -08:00
Jamie Pine 51b9e2fd3d Bump version: 0.1.5 → 0.1.6 2026-01-28 20:48:48 -08:00
Jamie Pine be25ddbe0e Refactor FloatingGenerateBox for improved code organization and readability
- Cleaned up import statements for better structure and consistency.
- Adjusted formatting and spacing in the FloatingGenerateBox component for enhanced readability.
- Streamlined the use of hooks and state management within the component.
- Ensured consistent styling and layout adjustments for better user experience.
2026-01-28 20:48:45 -08:00
Jamie Pine 9cd4921291 Enhance FloatingGenerateBox with auto-resizing textarea and default voice selection
- Added auto-resizing functionality to the textarea in FloatingGenerateBox, improving user experience when inputting text.
- Implemented logic to set the first voice profile as default if none is selected, ensuring a smoother workflow.
- Updated StoryContent to remove hardcoded height for the generate box, simplifying layout calculations.
- Refactored StoryTrackEditor to improve background styling for better visual consistency.
2026-01-28 20:48:13 -08:00
Jamie Pine 2349bd24ba Enhance FloatingGenerateBox and StoryContent with new features and improved UI
- Refactored FloatingGenerateBox to improve layout and ensure consistent styling for the voice selector.
- Added a popover component to StoryContent for adding generations, including search functionality for better user experience.
- Implemented story item editing and deletion capabilities in StoryList, enhancing story management features.
- Updated import statements and added new hooks for better organization and functionality across components.
2026-01-28 20:39:09 -08:00
Jamie Pine cd82ed0664 Refactor FloatingGenerateBox and StoriesTab for improved layout and interaction
- Adjusted FloatingGenerateBox positioning to align with the story list, ensuring consistent UI across different routes.
- Modified StoriesTab layout to enhance responsiveness, including setting a maximum width for the story list and adjusting the right column for better content display.
- Streamlined StoryChatItem interaction by simplifying the play functionality, allowing double-click to trigger playback directly from the text area.
- Enhanced StoryContent component by cleaning up unused playback controls and improving overall structure for better readability.
2026-01-28 20:31:17 -08:00
Jamie Pine c4884a0443 Enhance StoryContent and StoryTrackEditor for improved playback and UI dynamics
- Added auto-scrolling functionality to StoryContent for the currently playing item, enhancing user experience during playback.
- Refactored StoryTrackEditor to dynamically calculate container width, ensuring proper layout for varying story lengths.
- Updated audio playback management to improve timing anchor handling and playback scheduling.
- Cleaned up import statements for better organization and readability across components.
2026-01-28 20:15:59 -08:00
Jamie Pine 232d231788 Refactor story management components and enhance track editor integration
- Updated AppFrame to conditionally render StoryTrackEditor based on the selected story and route.
- Modified FloatingGenerateBox to adjust its position based on the visibility of the track editor.
- Improved StoriesTab by removing direct track editor rendering and relying on the new store state for height management.
- Enhanced StoryContent to dynamically calculate bottom padding based on the track editor's height.
- Introduced trackEditorHeight state in storyStore for better UI management of the track editor's visibility and size.
2026-01-28 19:50:46 -08:00
Jamie Pine 1cf90c81dd Enhance story item management with track editing functionality
- Introduced StoryTrackEditor component for managing story item positions and tracks.
- Updated StoriesTab to conditionally render the track editor based on selected story.
- Implemented moveStoryItem API endpoint to handle item repositioning and track changes.
- Enhanced story item data model to include track information.
- Improved audio playback management to support multiple tracks using Web Audio API.
- Added hooks for moving story items and managing playback timing.
2026-01-28 19:35:53 -08:00
Jamie Pine 3204e193fa Implement story management features and update dependencies
- Introduced story management functionality, including creating, listing, and managing story items.
- Added new components for story display and interaction, including StoriesTab, StoryList, and StoryContent.
- Integrated drag-and-drop functionality for reordering story items using @dnd-kit.
- Updated dependencies for @dnd-kit packages to enhance drag-and-drop capabilities.
- Bumped version for @voicebox/app, @voicebox/landing, @voicebox/tauri, and @voicebox/web to 0.1.5.
- Enhanced audio playback features to support story mode with auto-play functionality.
- Improved error handling and user feedback through toast notifications in story-related actions.
2026-01-28 19:10:58 -08:00
Jamie Pine 9d5d6cb56a Update UpdateStatus component logic and bump voicebox version to 0.1.5
- Modified conditional rendering in UpdateStatus to display content when the status is ready to install, improving user feedback.
- Bumped voicebox package version from 0.1.4 to 0.1.5 for dependency updates.
2026-01-28 15:14:23 -08:00
Jamie Pine 153eaba5f3 Refactor UpdateStatus component for improved UI and code organization
- Adjusted the order of import statements for better readability.
- Updated the conditional rendering logic to display a different message when an update is not ready to install, enhancing user feedback.
2026-01-28 15:14:05 -08:00
Jamie Pine 3370e3b419 Bump version: 0.1.4 → 0.1.5 2026-01-28 14:38:43 -08:00
Jamie Pine 615bd188a0 Refactor import statements in SampleUpload component for improved readability
- Adjusted the order of imports in SampleUpload.tsx to follow a more conventional structure, enhancing code organization.
2026-01-28 14:38:31 -08:00
Jamie Pine 7208f51eee Refactor audio generation components and improve debugging capabilities
- Introduced useGenerationForm hook to streamline audio generation form handling, including validation and model download management.
- Updated FloatingGenerateBox and GenerationForm components to utilize the new hook, enhancing code organization and reducing duplication.
- Replaced console logging with a debug utility for better logging control during audio playback and generation processes.
- Improved error handling in HistoryTable and MainEditor components by integrating toast notifications for user feedback.
- Adjusted audio recording duration limits across various components for consistency.
2026-01-28 14:30:33 -08:00
Jamie Pine 07a91a2381 Update app version to 0.1.4 and integrate @tanstack/react-router for improved routing functionality
- Bumped version for @voicebox/app, @voicebox/landing, @voicebox/tauri, and @voicebox/web to 0.1.4.
- Added @tanstack/react-router dependency for enhanced routing capabilities.
- Refactored App component to utilize RouterProvider for routing management.
- Created a new router configuration in router.tsx to define application routes and layout.
- Updated Sidebar component to use Link from @tanstack/react-router for navigation.
2026-01-28 00:00:18 -08:00
Jamie Pine 70cc36857d Bump version: 0.1.3 → 0.1.4 2026-01-27 17:25:15 -08:00
Jamie PineandGitHub 7f66d02591 Merge pull request #2 from jamiepine/channels
Channels
2026-01-27 17:23:58 -08:00
Jamie Pine 9f7a5a492e Refactor ConnectionForm and Checkbox component for improved functionality and UI
- Updated ConnectionForm to utilize a Checkbox component for managing the "keep server running" setting, enhancing user interaction.
- Refactored Checkbox component to use a button element for better accessibility and visual feedback.
- Streamlined import statements and improved code organization across multiple components for better readability.
2026-01-27 17:23:13 -08:00
Jamie Pine cb44377b09 Remove CheckCircle2 icon from various components for a cleaner UI
- Eliminated CheckCircle2 icon from ModelManagement, ModelProgress, ServerStatus, and UpdateStatus components to streamline the visual presentation.
- Updated import statements accordingly to reflect the removal of unused icons.
2026-01-27 17:14:17 -08:00
Jamie Pine a42a946586 Refactor UI components for improved layout consistency and responsiveness
- Updated App and AudioTab components to utilize TOP_SAFE_AREA_PADDING and BOTTOM_SAFE_AREA_PADDING constants for better layout adjustments.
- Enhanced HistoryTable and ModelManagement components for improved visual consistency.
- Streamlined import statements and component structure across various files for better organization and readability.
2026-01-27 17:12:16 -08:00
Jamie Pine a3cbe7f2b6 Refactor UI layout and introduce safe area constants for improved responsiveness
- Updated App, AppFrame, and AudioTab components to utilize new TOP_SAFE_AREA_PADDING and BOTTOM_SAFE_AREA_PADDING constants for consistent layout adjustments.
- Enhanced Sidebar and MainEditor components for better organization and user experience.
- Improved VoicesTab and ProfileList components by integrating new layout features and removing redundant import functionality.
- Streamlined HistoryTable and ModelManagement components for better visual consistency and interaction.
2026-01-27 17:11:38 -08:00
Jamie Pine d8d9eeaa6a Refactor App layout and introduce new components for improved organization
- Replaced the main layout in App component with AppFrame for better structure.
- Introduced MainEditor component to encapsulate the main editing interface, including ProfileList and HistoryTable.
- Added ModelsTab component to manage model-related functionalities.
- Updated Sidebar to include a new Models tab for navigation.
- Removed unused components and streamlined the layout for enhanced user experience.
2026-01-27 16:38:37 -08:00
Jamie Pine f7cb219f6d Refactor AudioPlayer for improved native playback handling and debugging
- Introduced a stop flag mechanism to manage audio playback more effectively.
- Enhanced native playback logic to ensure proper stopping of existing streams before starting new playback.
- Updated error handling and logging for better visibility during playback operations.
- Refactored audio output handling in Rust to support stopping playback and outputting silence when required.
- Improved the integration of native playback with WaveSurfer for seamless audio visualization.
2026-01-27 16:25:38 -08:00
Jamie Pine 7f18c09628 Enhance AudioPlayer component for native playback and debugging
- Improved the useNativePlayback logic to include detailed console logging for better debugging.
- Updated auto-play functionality to fetch runtime profile channels and channels, ensuring accurate playback decisions.
- Refactored audio playback handling to support native audio routing with enhanced error handling and logging.
- Introduced a new MultiSelect component for improved channel selection in VoicesTab.
- Updated FloatingGenerateBox to include selectedProfileId in audio setting.
- Added new dependencies for audio processing in Cargo.toml and Cargo.lock.
2026-01-27 16:15:16 -08:00
Jamie Pine d9c7121c5b Merge branch 'main' into channels 2026-01-27 15:24:18 -08:00
Jamie Pine 008b58f91c put back gen files for now 2026-01-27 14:08:30 -08:00
Jamie Pine c7404411d5 Bump version: 0.1.2 → 0.1.3 2026-01-27 13:46:21 -08:00
Jamie Pine ac08c4fcf4 Remove unused .gitkeep files and adjust ProfileList component padding
- Deleted .gitkeep files from Generation, ServerSettings, VoiceProfiles, and hooks directories as they are no longer necessary.
- Updated the ProfileList component to increase bottom padding for improved layout consistency.
2026-01-27 13:46:08 -08:00
Jamie Pine 3ce7498495 Remove unused schema files and update .gitignore
- Deleted several JSON schema files related to capabilities and desktop configurations that are no longer needed.
- Updated .gitignore to exclude the entire generated assets directory instead of a specific file, improving version control management.
2026-01-27 13:42:35 -08:00
Jamie Pine ce2f09d29e Refactor useAutoUpdater hook for improved readability and update handling
- Rearranged import statements for better organization.
- Enhanced download progress handling logic for clarity and consistency.
- Ensured proper state updates during the download process, including setting download progress to 100% upon completion.
2026-01-27 13:18:33 -08:00
Jamie Pine f1be633dca Implement FloatingGenerateBox component and update App layout
- Introduced the FloatingGenerateBox component for audio generation, enhancing user interaction with voice profiles.
- Updated App component to integrate FloatingGenerateBox and removed the GenerationForm component.
- Enhanced the layout for better responsiveness and added functionality to manage audio playback state.
- Updated UpdateStatus component to include new update handling logic and improved UI feedback for update readiness.
2026-01-27 13:18:12 -08:00
Jamie Pine 5f58c4dc3d Refactor VoiceProfiles components and remove ProfileDetail
- Removed the ProfileDetail component to streamline the ProfileCard functionality.
- Updated ProfileCard to eliminate the detail view and associated state management.
- Enhanced ProfileForm to manage audio samples more effectively, including improved UI for sample management.
- Adjusted SampleList to ensure proper button types for better accessibility.
2026-01-26 22:28:29 -08:00
Jamie Pine 892f363e3a Update development server port and enhance GPU status reporting
- Changed the development server port in package.json from 17493 to 8000 for dev.
- Improved GPU availability checks in main.py to include support for MPS on Apple Silicon.
- Removed the Assets.car file from version control as it is no longer needed.
- Updated .gitignore to reflect the removal of Assets.car.
2026-01-26 22:18:59 -08:00
Jamie Pine 30ea627ae8 Implement audio channel management features
- Added new components for managing audio channels, including creation, updating, and deletion of channels.
- Introduced a new AudioTab for channel management and integrated it into the main application layout.
- Updated the API client to support audio channel operations and added corresponding backend endpoints.
- Enhanced the player store to handle audio playback routing through assigned channels.
- Refactored existing components to accommodate the new audio channel functionality, including updates to the HistoryTable and GenerationForm for profile-channel associations.
- Improved sidebar navigation to include new tabs for Voices and Audio management.
2026-01-26 19:20:20 -08:00
82 changed files with 9263 additions and 863 deletions
+4 -4
View File
@@ -1,5 +1,5 @@
[bumpversion]
current_version = 0.1.2
current_version = 0.1.8
commit = True
tag = True
tag_name = v{new_version}
@@ -34,6 +34,6 @@ replace = "version": "{new_version}"
search = "version": "{current_version}"
replace = "version": "{new_version}"
[bumpversion:file:backend/main.py]
search = "version": "{current_version}"
replace = "version": "{new_version}"
[bumpversion:file:backend/__init__.py]
search = __version__ = "{current_version}"
replace = __version__ = "{new_version}"
-2
View File
@@ -30,8 +30,6 @@ target/
*.swo
*~
tauri/src-tauri/gen/Assets.car
# OS
.DS_Store
Thumbs.db
+32 -9
View File
@@ -22,7 +22,7 @@
<p align="center">
<a href="https://voicebox.sh">
<img src=".github/assets/screenshot.webp" alt="Voicebox App Screenshot" width="800" />
<img src="landing/public/assets/app-screenshot-1.webp" alt="Voicebox App Screenshot" width="800" />
</a>
</p>
@@ -32,17 +32,29 @@
<br/>
## Why Voicebox?
<p align="center">
<img src="landing/public/assets/app-screenshot-2.webp" alt="Voicebox Screenshot 2" width="800" />
</p>
Voice AI is exploding, but most tools are either cloud-locked, expensive, or a nightmare to set up. Voicebox is different:
<p align="center">
<img src="landing/public/assets/app-screenshot-3.webp" alt="Voicebox Screenshot 3" width="800" />
</p>
- **100% Local** — Your voice data never leaves your machine
- **Lightweight** — No bloated Electron, native Tauri performance
- **Fast** — Near-instant on CUDA, optimized for Apple Silicon
- **Flexible** — Use the app, integrate the API, or both
- **Open Source** — No subscriptions, no limits, no lock-in
<br/>
Built with **Tauri** (Rust), **TypeScript**, **React**, and **Python**. Native performance meets modern DX.
## What is Voicebox?
Voicebox is a **local-first voice cloning studio** with DAW-like features for professional voice synthesis. Think of it as the **Ollama for voice** — download models, clone voices, and generate speech entirely on your machine.
Unlike cloud services that lock your voice data behind subscriptions, Voicebox gives you:
- **Complete privacy** — models and voice data stay on your machine
- **Professional tools** — multi-track timeline editor, audio trimming, conversation mixing
- **Model flexibility** — currently powered by Qwen3-TTS, with support for XTTS, Bark, and other models coming soon
- **API-first** — use the desktop app or integrate voice synthesis into your own projects
- **Native performance** — built with Tauri (Rust), not Electron
Download a voice model, clone any voice from a few seconds of audio, and compose multi-voice projects with studio-grade editing tools. No Python install required, no cloud dependency, no limits.
---
@@ -75,6 +87,7 @@ Powered by Alibaba's **Qwen3-TTS** — a breakthrough model that achieves near-p
- **Create profiles** from audio files or record directly in-app
- **Import/Export** profiles to share or backup
- **Multi-sample support** — combine multiple samples for higher quality cloning
- **Organize** with descriptions and language tags
### Speech Generation
@@ -83,9 +96,19 @@ Powered by Alibaba's **Qwen3-TTS** — a breakthrough model that achieves near-p
- **Batch generation** for long-form content
- **Smart caching** — regenerate instantly with voice prompt caching
### Stories Editor
Create multi-voice narratives, podcasts, and conversations with a timeline-based editor.
- **Multi-track composition** — arrange multiple voice tracks in a single project
- **Inline audio editing** — trim and split clips directly in the timeline
- **Auto-playback** — preview stories with synchronized playhead
- **Voice mixing** — build conversations with multiple participants
### Recording & Transcription
- **In-app recording** with waveform visualization
- **System audio capture** — record desktop audio on macOS and Windows
- **Automatic transcription** powered by Whisper
- **Export recordings** in multiple formats
+5 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@voicebox/app",
"version": "0.1.2",
"version": "0.1.8",
"private": true,
"type": "module",
"scripts": {
@@ -13,6 +13,9 @@
"check": "biome check --write src"
},
"dependencies": {
"@dnd-kit/core": "^6.3.1",
"@dnd-kit/sortable": "^10.0.0",
"@dnd-kit/utilities": "^3.2.2",
"@hookform/resolvers": "^3.9.0",
"@radix-ui/react-alert-dialog": "^1.1.1",
"@radix-ui/react-avatar": "^1.1.0",
@@ -30,6 +33,7 @@
"@radix-ui/react-toast": "^1.2.1",
"@tanstack/react-query": "^5.0.0",
"@tanstack/react-query-devtools": "^5.0.0",
"@tanstack/react-router": "^1.157.16",
"@tauri-apps/api": "^2.0.0",
"@tauri-apps/plugin-dialog": "^2.0.0",
"@tauri-apps/plugin-fs": "^2.0.0",
+17 -120
View File
@@ -1,31 +1,19 @@
import { useEffect, useState } from 'react';
import { useEffect, useRef, useState } from 'react';
import { RouterProvider } from '@tanstack/react-router';
import voiceboxLogo from '@/assets/voicebox-logo.png';
import { AudioPlayer } from '@/components/AudioPlayer/AudioPlayer';
import { GenerationForm } from '@/components/Generation/GenerationForm';
import { HistoryTable } from '@/components/History/HistoryTable';
import { ConnectionForm } from '@/components/ServerSettings/ConnectionForm';
import { ModelManagement } from '@/components/ServerSettings/ModelManagement';
import { ServerStatus } from '@/components/ServerSettings/ServerStatus';
import { UpdateStatus } from '@/components/ServerSettings/UpdateStatus';
import ShinyText from '@/components/ShinyText';
import { Sidebar } from '@/components/Sidebar';
import { TitleBarDragRegion } from '@/components/TitleBarDragRegion';
import { Toaster } from '@/components/ui/toaster';
import { ProfileList } from '@/components/VoiceProfiles/ProfileList';
import { useModelDownloadToast } from '@/lib/hooks/useModelDownloadToast';
import { MODEL_DISPLAY_NAMES, useRestoreActiveTasks } from '@/lib/hooks/useRestoreActiveTasks';
import { TOP_SAFE_AREA_PADDING } from '@/lib/constants/ui';
import {
isMacOS,
isTauri,
setKeepServerRunning,
setupWindowCloseHandler,
startServer,
} from '@/lib/tauri';
import { cn } from '@/lib/utils/cn';
import { router } from '@/router';
import { useServerStore } from '@/stores/serverStore';
// Track if server is starting to prevent duplicate starts
let serverStarting = false;
const LOADING_MESSAGES = [
'Warming up tensors...',
'Calibrating synthesizer engine...',
@@ -50,12 +38,9 @@ const LOADING_MESSAGES = [
];
function App() {
const [activeTab, setActiveTab] = useState('main');
const [serverReady, setServerReady] = useState(false);
const [loadingMessageIndex, setLoadingMessageIndex] = useState(0);
// Monitor active downloads/generations and show toasts for them
const activeDownloads = useRestoreActiveTasks();
const serverStartingRef = useRef(false);
// Sync stored setting to Rust on startup
useEffect(() => {
@@ -91,11 +76,11 @@ function App() {
}
// Auto-start server in production
if (serverStarting) {
if (serverStartingRef.current) {
return;
}
serverStarting = true;
serverStartingRef.current = true;
console.log('Production mode: Starting bundled server...');
startServer(false)
@@ -105,13 +90,11 @@ function App() {
useServerStore.getState().setServerUrl(serverUrl);
setServerReady(true);
// Mark that we started the server (so we know to stop it on close)
// @ts-expect-error - adding property to window
window.__voiceboxServerStartedByApp = true;
})
.catch((error) => {
console.error('Failed to auto-start server:', error);
serverStarting = false;
// @ts-expect-error - adding property to window
serverStartingRef.current = false;
window.__voiceboxServerStartedByApp = false;
});
@@ -119,7 +102,7 @@ function App() {
// Note: Window close is handled separately in Tauri Rust code
return () => {
// Window close event handles server shutdown based on setting
serverStarting = false;
serverStartingRef.current = false;
};
}, []);
@@ -139,7 +122,12 @@ function App() {
// Show loading screen while server is starting in Tauri
if (isTauri() && !serverReady) {
return (
<div className="min-h-screen bg-background flex items-center justify-center pt-12">
<div
className={cn(
'min-h-screen bg-background flex items-center justify-center',
TOP_SAFE_AREA_PADDING,
)}
>
<TitleBarDragRegion />
<div className="text-center space-y-6">
<div className="flex justify-center relative">
@@ -166,98 +154,7 @@ function App() {
);
}
return (
<div className="h-screen bg-background flex flex-col overflow-hidden pt-12">
<TitleBarDragRegion />
<div className="flex flex-1 min-h-0 overflow-hidden">
<Sidebar activeTab={activeTab} onTabChange={setActiveTab} isMacOS={isMacOS()} />
<main className="flex-1 ml-20 overflow-hidden flex flex-col">
<div className="container mx-auto px-8 max-w-[1800px] h-full overflow-hidden flex flex-col">
{activeTab === 'settings' ? (
<div className="space-y-4 overflow-y-auto flex flex-col">
<div className="grid gap-4 md:grid-cols-2">
<ConnectionForm />
<ServerStatus />
</div>
{isTauri() && <UpdateStatus />}
<ModelManagement />
<div className="py-8 text-center text-sm text-muted-foreground">
Created by{' '}
<a
href="https://github.com/jamiepine"
target="_blank"
rel="noopener noreferrer"
className="text-accent hover:underline"
>
Jamie Pine
</a>
</div>
</div>
) : (
// Main view: Profiles top left, Generator bottom left, History right
<div className="grid grid-cols-1 lg:grid-cols-2 gap-6 h-full min-h-0 overflow-hidden">
{/* Left Column */}
<div className="flex flex-col gap-6 min-h-0 overflow-y-auto pb-32">
{/* Profiles - Top Left */}
<div className="shrink-0 flex flex-col">
<ProfileList />
</div>
{/* Generator - Bottom Left */}
<div className="shrink-0">
<GenerationForm />
</div>
</div>
{/* Right Column - History */}
<div className="flex flex-col min-h-0 overflow-hidden">
<HistoryTable />
</div>
</div>
)}
</div>
</main>
</div>
{/* Audio Player - always visible except on settings */}
{activeTab !== 'settings' && <AudioPlayer />}
{/* Show download toasts for any active downloads (from anywhere) */}
{activeDownloads.map((download) => {
const displayName = MODEL_DISPLAY_NAMES[download.model_name] || download.model_name;
return (
<DownloadToastRestorer
key={download.model_name}
modelName={download.model_name}
displayName={displayName}
/>
);
})}
<Toaster />
</div>
);
}
/**
* Component that restores a download toast for a specific model.
*/
function DownloadToastRestorer({
modelName,
displayName,
}: {
modelName: string;
displayName: string;
}) {
// Use the download toast hook to restore the toast
useModelDownloadToast({
modelName,
displayName,
enabled: true,
});
return null;
return <RouterProvider router={router} />;
}
export default App;
+35
View File
@@ -0,0 +1,35 @@
import { useRouterState } from '@tanstack/react-router';
import { TitleBarDragRegion } from '@/components/TitleBarDragRegion';
import { AudioPlayer } from '@/components/AudioPlayer/AudioPlayer';
import { StoryTrackEditor } from '@/components/StoriesTab/StoryTrackEditor';
import { TOP_SAFE_AREA_PADDING } from '@/lib/constants/ui';
import { cn } from '@/lib/utils/cn';
import { useStoryStore } from '@/stores/storyStore';
import { useStory } from '@/lib/hooks/useStories';
interface AppFrameProps {
children: React.ReactNode;
}
export function AppFrame({ children }: AppFrameProps) {
const routerState = useRouterState();
const isStoriesRoute = routerState.location.pathname === '/stories';
const selectedStoryId = useStoryStore((state) => state.selectedStoryId);
const { data: story } = useStory(selectedStoryId);
// Show track editor when on stories route with a selected story that has items
const showTrackEditor = isStoriesRoute && selectedStoryId && story && story.items.length > 0;
return (
<div className={cn('h-screen bg-background flex flex-col overflow-hidden', TOP_SAFE_AREA_PADDING)}>
<TitleBarDragRegion />
{children}
{showTrackEditor ? (
<StoryTrackEditor storyId={story.id} items={story.items} />
) : (
<AudioPlayer />
)}
</div>
);
}
+454 -52
View File
@@ -1,15 +1,21 @@
import { Pause, Play, Repeat, Volume2, VolumeX } from 'lucide-react';
import { useEffect, useRef, useState } from 'react';
import { useQuery } from '@tanstack/react-query';
import { invoke } from '@tauri-apps/api/core';
import { Pause, Play, Repeat, Volume2, VolumeX, X } from 'lucide-react';
import { useEffect, useMemo, useRef, useState } from 'react';
import WaveSurfer from 'wavesurfer.js';
import { Button } from '@/components/ui/button';
import { Slider } from '@/components/ui/slider';
import { apiClient } from '@/lib/api/client';
import { isTauri } from '@/lib/tauri';
import { formatAudioDuration } from '@/lib/utils/audio';
import { debug } from '@/lib/utils/debug';
import { usePlayerStore } from '@/stores/playerStore';
export function AudioPlayer() {
const {
audioUrl,
audioId,
profileId,
title,
isPlaying,
currentTime,
@@ -23,13 +29,47 @@ export function AudioPlayer() {
setVolume,
toggleLoop,
clearRestartFlag,
reset,
} = usePlayerStore();
// Check if profile has assigned channels (for native audio routing)
const { data: profileChannels } = useQuery({
queryKey: ['profile-channels', profileId],
queryFn: () => {
if (!profileId) return { channel_ids: [] };
return apiClient.getProfileChannels(profileId);
},
enabled: !!profileId && isTauri(),
});
const { data: channels } = useQuery({
queryKey: ['channels'],
queryFn: () => apiClient.listChannels(),
enabled: !!profileChannels && profileChannels.channel_ids.length > 0,
});
// Determine if we should use native playback
const useNativePlayback = useMemo(() => {
if (!isTauri() || !profileChannels || !channels) {
return false;
}
const assignedChannels = channels.filter((ch) => profileChannels.channel_ids.includes(ch.id));
// Use native playback if any assigned channel has non-default devices
const shouldUseNative = assignedChannels.some(
(ch) => ch.device_ids.length > 0 && !ch.is_default,
);
return shouldUseNative;
}, [profileChannels, channels, profileId]);
const waveformRef = useRef<HTMLDivElement>(null);
const wavesurferRef = useRef<WaveSurfer | null>(null);
const loadingRef = useRef(false);
const previousAudioIdRef = useRef<string | null>(null);
const hasInitializedRef = useRef(false);
const isUsingNativePlaybackRef = useRef(false);
const [isLoading, setIsLoading] = useState(false);
const [error, setError] = useState<string | null>(null);
@@ -41,11 +81,11 @@ export function AudioPlayer() {
}
if (wavesurferRef.current) {
console.log('WaveSurfer already initialized, skipping');
debug.log('WaveSurfer already initialized, skipping');
return;
}
console.log('Creating NEW WaveSurfer instance');
debug.log('Creating NEW WaveSurfer instance');
// Wait for container to be properly rendered
const initWaveSurfer = () => {
@@ -71,7 +111,7 @@ export function AudioPlayer() {
return;
}
console.log('Initializing WaveSurfer...', {
debug.log('Initializing WaveSurfer...', {
container,
width: rect.width,
height: rect.height,
@@ -104,9 +144,9 @@ export function AudioPlayer() {
});
wavesurferRef.current = wavesurfer;
console.log('WaveSurfer created successfully');
debug.log('WaveSurfer created successfully');
} catch (error) {
console.error('Failed to create WaveSurfer:', error);
debug.error('Failed to create WaveSurfer:', error);
setError(
`Failed to initialize waveform: ${error instanceof Error ? error.message : String(error)}`,
);
@@ -122,47 +162,243 @@ export function AudioPlayer() {
});
// Update store when duration is loaded
wavesurfer.on('ready', () => {
wavesurfer.on('ready', async () => {
const dur = wavesurfer.getDuration();
setDuration(dur);
loadingRef.current = false;
setIsLoading(false);
setError(null);
console.log('Audio ready, duration:', dur);
console.log('Waveform should be visible now');
debug.log('Audio ready, duration:', dur);
debug.log('Waveform should be visible now');
// Ensure volume is set
const currentVolume = usePlayerStore.getState().volume;
wavesurfer.setVolume(currentVolume);
// Get the underlying audio element and ensure it's not muted
// (unless we're using native playback, which will be set later)
const mediaElement = wavesurfer.getMediaElement();
if (mediaElement) {
if (mediaElement && !isUsingNativePlaybackRef.current) {
mediaElement.volume = currentVolume;
mediaElement.muted = false;
console.log('Audio element volume:', mediaElement.volume, 'muted:', mediaElement.muted);
debug.log('Audio element volume:', mediaElement.volume, 'muted:', mediaElement.muted);
}
// Auto-play when ready
// Use a small delay to ensure audio element is fully ready
setTimeout(() => {
wavesurfer.play().catch((error) => {
console.error('Failed to autoplay:', error);
// Don't show error for autoplay failures (browser restrictions)
});
}, 100);
// Auto-play when ready - check if we should use native playback
// Get current values from the store and queries at runtime (not captured closure values)
const currentAudioUrl = usePlayerStore.getState().audioUrl;
const currentProfileId = usePlayerStore.getState().profileId;
debug.log('Auto-play check - capturing runtime values...');
// Fetch profile channels at runtime (not using captured value)
let runtimeProfileChannels = null;
let runtimeChannels = null;
if (isTauri() && currentProfileId) {
try {
runtimeProfileChannels = await apiClient.getProfileChannels(currentProfileId);
debug.log('Runtime profileChannels:', runtimeProfileChannels);
if (runtimeProfileChannels && runtimeProfileChannels.channel_ids.length > 0) {
runtimeChannels = await apiClient.listChannels();
debug.log('Runtime channels:', runtimeChannels);
}
} catch (error) {
debug.error('Failed to fetch runtime channel data:', error);
}
}
debug.log('Auto-play check:', {
isTauri: isTauri(),
currentAudioUrl,
currentProfileId,
hasProfileChannels: !!runtimeProfileChannels,
hasChannels: !!runtimeChannels,
});
if (
isTauri() &&
currentAudioUrl &&
currentProfileId &&
runtimeProfileChannels &&
runtimeChannels
) {
debug.log('Attempting native audio playback...');
// Stop any existing native playback first
if (isUsingNativePlaybackRef.current) {
try {
await invoke('stop_audio_playback');
debug.log('Stopped existing native playback before starting new one');
} catch (error) {
debug.error('Failed to stop existing playback:', error);
}
}
try {
// Collect all device IDs from assigned channels
const assignedChannels = runtimeChannels.filter((ch: any) =>
runtimeProfileChannels.channel_ids.includes(ch.id),
);
debug.log('Assigned channels for playback:', assignedChannels);
// Check if any assigned channel has non-default devices
const shouldUseNative = assignedChannels.some(
(ch: any) => ch.device_ids.length > 0 && !ch.is_default,
);
debug.log('Should use native playback:', shouldUseNative);
if (!shouldUseNative) {
debug.log('No custom devices assigned, falling back to WaveSurfer');
// Reset native playback flag and unmute WaveSurfer
isUsingNativePlaybackRef.current = false;
const mediaElement = wavesurfer.getMediaElement();
if (mediaElement) {
const currentVolume = usePlayerStore.getState().volume;
mediaElement.volume = currentVolume;
mediaElement.muted = false;
debug.log(
'WaveSurfer unmuted for normal playback - volume:',
mediaElement.volume,
'muted:',
mediaElement.muted,
);
}
} else {
const deviceIds = assignedChannels.flatMap((ch: any) => ch.device_ids);
debug.log('Device IDs to play to:', deviceIds);
if (deviceIds.length > 0) {
debug.log('Fetching audio data from:', currentAudioUrl);
// Fetch audio data
const response = await fetch(currentAudioUrl);
const audioData = new Uint8Array(await response.arrayBuffer());
debug.log('Audio data size:', audioData.length);
// Play via native audio
debug.log('Invoking play_audio_to_devices...');
try {
const result = await invoke('play_audio_to_devices', {
audioData: Array.from(audioData),
deviceIds: deviceIds,
});
debug.log('play_audio_to_devices completed successfully, result:', result);
// Mark that we're using native playback
isUsingNativePlaybackRef.current = true;
// Mute WaveSurfer's audio element to prevent UI audio output
// Keep WaveSurfer running for visualization
const mediaElement = wavesurfer.getMediaElement();
if (mediaElement) {
mediaElement.volume = 0;
mediaElement.muted = true;
debug.log(
'WaveSurfer muted for native playback - volume:',
mediaElement.volume,
'muted:',
mediaElement.muted,
);
}
// Start WaveSurfer playback for visualization (muted)
wavesurfer.play().catch((error) => {
debug.error('Failed to start WaveSurfer visualization:', error);
});
setIsPlaying(true);
debug.log('Auto-playing via native audio routing - SUCCESS');
return;
} catch (invokeError) {
debug.error('play_audio_to_devices invoke failed:', invokeError);
throw invokeError;
}
} else {
debug.log('No device IDs found, falling back to WaveSurfer');
}
}
} catch (error) {
debug.error(
'Native playback failed during auto-play, falling back to WaveSurfer:',
error,
);
// Reset native playback flag and unmute WaveSurfer
isUsingNativePlaybackRef.current = false;
const mediaElement = wavesurfer.getMediaElement();
if (mediaElement) {
const currentVolume = usePlayerStore.getState().volume;
mediaElement.volume = currentVolume;
mediaElement.muted = false;
debug.log(
'WaveSurfer unmuted after native playback failure - volume:',
mediaElement.volume,
'muted:',
mediaElement.muted,
);
}
// Fall through to WaveSurfer playback
}
} else {
debug.log('Not using native playback, using WaveSurfer');
// Reset native playback flag and unmute WaveSurfer
isUsingNativePlaybackRef.current = false;
const mediaElement = wavesurfer.getMediaElement();
if (mediaElement) {
const currentVolume = usePlayerStore.getState().volume;
mediaElement.volume = currentVolume;
mediaElement.muted = false;
debug.log(
'WaveSurfer unmuted for normal playback - volume:',
mediaElement.volume,
'muted:',
mediaElement.muted,
);
}
}
// Only auto-play if shouldAutoPlay flag is set (user explicitly clicked to play)
const shouldAutoPlayNow = usePlayerStore.getState().shouldAutoPlay;
if (shouldAutoPlayNow) {
// Clear the flag first
usePlayerStore.getState().clearAutoPlayFlag();
// Use a small delay to ensure audio element is fully ready
setTimeout(() => {
wavesurfer.play().catch((error) => {
debug.error('Failed to autoplay:', error);
// Don't show error for autoplay failures (browser restrictions)
});
}, 100);
} else {
debug.log('Skipping auto-play - shouldAutoPlay is false');
}
});
// Handle play/pause
wavesurfer.on('play', () => {
setIsPlaying(true);
// Ensure audio element is not muted when playing
// Ensure audio element volume is set correctly
const mediaElement = wavesurfer.getMediaElement();
if (mediaElement) {
mediaElement.muted = false;
const currentVolume = usePlayerStore.getState().volume;
mediaElement.volume = currentVolume;
console.log('Playing - volume:', mediaElement.volume, 'muted:', mediaElement.muted);
// Double-check: if using native playback, keep WaveSurfer muted
// Otherwise, ensure it's unmuted
if (isUsingNativePlaybackRef.current) {
mediaElement.volume = 0;
mediaElement.muted = true;
debug.log('Playing (native mode) - WaveSurfer muted for visualization only');
} else {
// Ensure WaveSurfer is unmuted for normal playback
const currentVolume = usePlayerStore.getState().volume;
mediaElement.volume = currentVolume;
mediaElement.muted = false;
debug.log(
'Playing (normal mode) - volume:',
mediaElement.volume,
'muted:',
mediaElement.muted,
);
}
}
});
wavesurfer.on('pause', () => setIsPlaying(false));
@@ -174,12 +410,17 @@ export function AudioPlayer() {
wavesurfer.play();
} else {
setIsPlaying(false);
// Trigger finish callback if set
const onFinish = usePlayerStore.getState().onFinish;
if (onFinish) {
onFinish();
}
}
});
// Handle errors
wavesurfer.on('error', (error) => {
console.error('WaveSurfer error:', error);
debug.error('WaveSurfer error:', error);
setIsLoading(false);
setError(`Audio error: ${error instanceof Error ? error.message : String(error)}`);
});
@@ -194,7 +435,7 @@ export function AudioPlayer() {
// Load audio immediately if audioUrl is already set
if (audioUrl) {
console.log('WaveSurfer ready, loading audio:', audioUrl);
debug.log('WaveSurfer ready, loading audio:', audioUrl);
loadingRef.current = true;
setIsLoading(true);
// Stop any current playback before loading new audio
@@ -204,11 +445,11 @@ export function AudioPlayer() {
wavesurfer
.load(audioUrl)
.then(() => {
console.log('Audio loaded into WaveSurfer');
debug.log('Audio loaded into WaveSurfer');
loadingRef.current = false;
})
.catch((error) => {
console.error('Failed to load audio into WaveSurfer:', error);
debug.error('Failed to load audio into WaveSurfer:', error);
loadingRef.current = false;
setIsLoading(false);
setError(
@@ -233,12 +474,12 @@ export function AudioPlayer() {
});
return () => {
console.log('Cleaning up WaveSurfer initialization effect');
debug.log('Cleaning up WaveSurfer initialization effect');
if (rafId1) cancelAnimationFrame(rafId1);
if (rafId2) cancelAnimationFrame(rafId2);
if (timeoutId) clearTimeout(timeoutId);
if (wavesurferRef.current) {
console.log('Destroying WaveSurfer instance');
debug.log('Destroying WaveSurfer instance');
try {
const mediaElement = wavesurferRef.current.getMediaElement();
if (mediaElement) {
@@ -247,7 +488,7 @@ export function AudioPlayer() {
}
wavesurferRef.current.destroy();
} catch (error) {
console.error('Error destroying WaveSurfer:', error);
debug.error('Error destroying WaveSurfer:', error);
}
wavesurferRef.current = null;
}
@@ -268,36 +509,61 @@ export function AudioPlayer() {
setDuration(0);
setCurrentTime(0);
setError(null);
// Reset native playback flag
isUsingNativePlaybackRef.current = false;
}
return;
}
// Stop native playback if it was active
if (isUsingNativePlaybackRef.current && isTauri()) {
(async () => {
try {
await invoke('stop_audio_playback');
debug.log('Stopped native audio playback');
} catch (error) {
debug.error('Failed to stop native playback:', error);
}
})();
}
// Reset native playback flag when loading new audio
// Also unmute WaveSurfer if it was muted
if (isUsingNativePlaybackRef.current) {
const mediaElement = wavesurfer.getMediaElement();
if (mediaElement) {
mediaElement.muted = false;
mediaElement.volume = usePlayerStore.getState().volume;
}
}
isUsingNativePlaybackRef.current = false;
// CRITICAL: Force stop any current playback and cancel any pending loads
// This must happen BEFORE any early returns
console.log('Audio URL changed to:', audioUrl);
debug.log('Audio URL changed to:', audioUrl);
// COMPLETELY stop and destroy the current audio
try {
// First pause if playing
if (wavesurfer.isPlaying()) {
console.log('Pausing current playback');
debug.log('Pausing current playback');
wavesurfer.pause();
}
// Stop the media element explicitly
const mediaElement = wavesurfer.getMediaElement();
if (mediaElement) {
console.log('Stopping media element');
debug.log('Stopping media element');
mediaElement.pause();
mediaElement.currentTime = 0;
mediaElement.src = '';
}
// Use empty() to completely destroy the waveform and media element
console.log('Calling wavesurfer.empty() to destroy audio');
debug.log('Calling wavesurfer.empty() to destroy audio');
wavesurfer.empty();
} catch (error) {
console.error('Error stopping previous audio:', error);
debug.error('Error stopping previous audio:', error);
// Continue anyway to load new audio
}
@@ -312,16 +578,16 @@ export function AudioPlayer() {
setDuration(0);
// Load new audio
console.log('Starting new audio load for:', audioUrl);
debug.log('Starting new audio load for:', audioUrl);
wavesurfer
.load(audioUrl)
.then(() => {
console.log('Audio load promise resolved');
debug.log('Audio load promise resolved');
// Don't set loading to false here - wait for 'ready' event
})
.catch((error) => {
console.error('Failed to load audio:', error);
console.error('Audio URL:', audioUrl);
debug.error('Failed to load audio:', error);
debug.error('Audio URL:', audioUrl);
loadingRef.current = false;
setIsLoading(false);
setError(`Failed to load audio: ${error instanceof Error ? error.message : String(error)}`);
@@ -336,7 +602,7 @@ export function AudioPlayer() {
if (isPlaying && wavesurferRef.current.isPlaying() === false) {
// Only auto-play if audio is ready
wavesurferRef.current.play().catch((error) => {
console.error('Failed to play:', error);
debug.error('Failed to play:', error);
setIsPlaying(false);
setError(`Playback error: ${error instanceof Error ? error.message : String(error)}`);
});
@@ -352,9 +618,16 @@ export function AudioPlayer() {
// Also ensure the underlying audio element volume is set
const mediaElement = wavesurferRef.current.getMediaElement();
if (mediaElement) {
mediaElement.volume = volume;
mediaElement.muted = volume === 0;
console.log('Volume synced:', volume, 'muted:', mediaElement.muted);
// If using native playback, keep WaveSurfer muted regardless of volume setting
if (isUsingNativePlaybackRef.current) {
mediaElement.volume = 0;
mediaElement.muted = true;
debug.log('Volume sync: Using native playback, keeping WaveSurfer muted');
} else {
mediaElement.volume = volume;
mediaElement.muted = volume === 0;
debug.log('Volume synced:', volume, 'muted:', mediaElement.muted);
}
}
}
}, [volume]);
@@ -381,38 +654,140 @@ export function AudioPlayer() {
}
// Reset to beginning and play
console.log('Restarting current audio from beginning');
debug.log('Restarting current audio from beginning');
wavesurfer.seekTo(0);
wavesurfer.play().catch((error) => {
console.error('Failed to play after restart:', error);
debug.error('Failed to play after restart:', error);
setIsPlaying(false);
setError(`Playback error: ${error instanceof Error ? error.message : String(error)}`);
});
// Clear the restart flag
clearRestartFlag();
}, [shouldRestart, duration, setIsPlaying, clearRestartFlag]);
// Handle shouldAutoPlay flag - for story mode auto-advance
const shouldAutoPlay = usePlayerStore((state) => state.shouldAutoPlay);
const clearAutoPlayFlag = usePlayerStore((state) => state.clearAutoPlayFlag);
useEffect(() => {
const wavesurfer = wavesurferRef.current;
if (!wavesurfer || !shouldAutoPlay || duration === 0) {
return;
}
// Auto-play the newly loaded audio
debug.log('Auto-playing next track in story mode');
wavesurfer.seekTo(0);
wavesurfer.play().catch((error) => {
debug.error('Failed to auto-play:', error);
setIsPlaying(false);
setError(`Playback error: ${error instanceof Error ? error.message : String(error)}`);
});
// Clear the auto-play flag
clearAutoPlayFlag();
}, [shouldAutoPlay, duration, setIsPlaying, clearAutoPlayFlag]);
// Handle loop - WaveSurfer handles this via the 'finish' event
const handlePlayPause = () => {
const handlePlayPause = async () => {
// Standard WaveSurfer playback (works for both normal and native playback modes)
// When using native playback, WaveSurfer is muted but still controls visualization
if (!wavesurferRef.current) {
console.error('WaveSurfer not initialized');
debug.error('WaveSurfer not initialized');
return;
}
// Check if audio is loaded
if (duration === 0 && !isLoading) {
console.error('Audio not loaded yet');
debug.error('Audio not loaded yet');
setError('Audio not loaded. Please wait...');
return;
}
// If using native playback
if (useNativePlayback && audioUrl && profileChannels && channels) {
if (isPlaying) {
// Pause: stop native playback and pause WaveSurfer visualization
try {
await invoke('stop_audio_playback');
debug.log('Stopped native audio playback');
} catch (error) {
debug.error('Failed to stop native playback:', error);
}
wavesurferRef.current.pause();
return;
}
// Play: trigger native playback
try {
// Stop any existing native playback first
try {
await invoke('stop_audio_playback');
} catch (_error) {
// Ignore errors when stopping (might not be playing)
debug.log('No existing playback to stop');
}
// Collect all device IDs from assigned channels
const assignedChannels = channels.filter((ch) =>
profileChannels.channel_ids.includes(ch.id),
);
const deviceIds = assignedChannels.flatMap((ch) => ch.device_ids);
if (deviceIds.length > 0) {
// Fetch audio data
const response = await fetch(audioUrl);
const audioData = new Uint8Array(await response.arrayBuffer());
// Play via native audio
await invoke('play_audio_to_devices', {
audioData: Array.from(audioData),
deviceIds: deviceIds,
});
// Mark that we're using native playback
isUsingNativePlaybackRef.current = true;
// Mute WaveSurfer and start it for visualization
const mediaElement = wavesurferRef.current.getMediaElement();
if (mediaElement) {
mediaElement.volume = 0;
mediaElement.muted = true;
}
// Start WaveSurfer for visualization (muted)
wavesurferRef.current.play().catch((error) => {
debug.error('Failed to start WaveSurfer visualization:', error);
setIsPlaying(false);
setError(`Playback error: ${error instanceof Error ? error.message : String(error)}`);
});
return;
}
} catch (error) {
debug.error('Native playback failed, falling back to WaveSurfer:', error);
// Fall through to WaveSurfer playback
isUsingNativePlaybackRef.current = false;
}
}
// Standard WaveSurfer playback (or fallback from native playback failure)
if (wavesurferRef.current.isPlaying()) {
wavesurferRef.current.pause();
} else {
// Ensure WaveSurfer is not muted if not using native playback
if (!isUsingNativePlaybackRef.current) {
const mediaElement = wavesurferRef.current.getMediaElement();
if (mediaElement) {
mediaElement.muted = false;
mediaElement.volume = volume;
}
}
wavesurferRef.current.play().catch((error) => {
console.error('Failed to play:', error);
debug.error('Failed to play:', error);
setIsPlaying(false);
setError(`Playback error: ${error instanceof Error ? error.message : String(error)}`);
});
@@ -429,6 +804,22 @@ export function AudioPlayer() {
setVolume(value[0] / 100);
};
const handleClose = () => {
// Stop any native playback
if (isUsingNativePlaybackRef.current && isTauri()) {
invoke('stop_audio_playback').catch((error) => {
debug.error('Failed to stop native playback:', error);
});
}
// Stop WaveSurfer
if (wavesurferRef.current) {
wavesurferRef.current.pause();
wavesurferRef.current.seekTo(0);
}
// Reset player state
reset();
};
// Don't render if no audio
if (!audioUrl) {
return null;
@@ -509,6 +900,17 @@ export function AudioPlayer() {
className="flex-1"
/>
</div>
{/* Close Button */}
<Button
variant="ghost"
size="icon"
onClick={handleClose}
className="shrink-0"
title="Close player"
>
<X className="h-5 w-5" />
</Button>
</div>
</div>
</div>
+672
View File
@@ -0,0 +1,672 @@
import { useMutation, useQuery, useQueryClient } from '@tanstack/react-query';
import { invoke } from '@tauri-apps/api/core';
import { Check, CheckCircle2, Edit, Plus, Speaker, Trash2 } from 'lucide-react';
import { useState } from 'react';
import { Badge } from '@/components/ui/badge';
import { Button } from '@/components/ui/button';
import {
Dialog,
DialogContent,
DialogDescription,
DialogFooter,
DialogHeader,
DialogTitle,
} from '@/components/ui/dialog';
import { Input } from '@/components/ui/input';
import { Label } from '@/components/ui/label';
import {
Select,
SelectContent,
SelectItem,
SelectTrigger,
SelectValue,
} from '@/components/ui/select';
import { apiClient } from '@/lib/api/client';
import { BOTTOM_SAFE_AREA_PADDING } from '@/lib/constants/ui';
import { isTauri } from '@/lib/tauri';
import { cn } from '@/lib/utils/cn';
import { usePlayerStore } from '@/stores/playerStore';
interface AudioDevice {
id: string;
name: string;
is_default: boolean;
}
export function AudioTab() {
const [createDialogOpen, setCreateDialogOpen] = useState(false);
const [editingChannel, setEditingChannel] = useState<string | null>(null);
const [selectedChannelId, setSelectedChannelId] = useState<string | null>(null);
const queryClient = useQueryClient();
const audioUrl = usePlayerStore((state) => state.audioUrl);
const isPlayerVisible = !!audioUrl;
const { data: channels, isLoading: channelsLoading } = useQuery({
queryKey: ['channels'],
queryFn: () => apiClient.listChannels(),
});
const { data: devices, isLoading: devicesLoading } = useQuery({
queryKey: ['audio-devices'],
queryFn: async () => {
if (!isTauri()) {
return [];
}
try {
const result = await invoke<AudioDevice[]>('list_audio_output_devices');
return result;
} catch (error) {
console.error('Failed to list audio devices:', error);
return [];
}
},
enabled: isTauri(),
});
const { data: profiles } = useQuery({
queryKey: ['profiles'],
queryFn: () => apiClient.listProfiles(),
});
const createChannel = useMutation({
mutationFn: (data: { name: string; device_ids: string[] }) => apiClient.createChannel(data),
onSuccess: () => {
queryClient.invalidateQueries({ queryKey: ['channels'] });
setCreateDialogOpen(false);
},
});
const updateChannel = useMutation({
mutationFn: ({
channelId,
data,
}: {
channelId: string;
data: { name?: string; device_ids?: string[] };
}) => apiClient.updateChannel(channelId, data),
onSuccess: () => {
queryClient.invalidateQueries({ queryKey: ['channels'] });
queryClient.invalidateQueries({ queryKey: ['profile-channels'] });
setEditingChannel(null);
},
});
const deleteChannel = useMutation({
mutationFn: (channelId: string) => apiClient.deleteChannel(channelId),
onSuccess: () => {
queryClient.invalidateQueries({ queryKey: ['channels'] });
queryClient.invalidateQueries({ queryKey: ['profile-channels'] });
},
});
const { data: channelVoices } = useQuery({
queryKey: ['channel-voices', editingChannel],
queryFn: async () => {
if (!editingChannel) return { profile_ids: [] };
return apiClient.getChannelVoices(editingChannel);
},
enabled: !!editingChannel,
});
const setChannelVoices = useMutation({
mutationFn: ({ channelId, profileIds }: { channelId: string; profileIds: string[] }) =>
apiClient.setChannelVoices(channelId, profileIds),
onSuccess: () => {
queryClient.invalidateQueries({ queryKey: ['channel-voices'] });
queryClient.invalidateQueries({ queryKey: ['profile-channels'] });
},
});
if (channelsLoading || devicesLoading) {
return (
<div className="flex items-center justify-center h-full">
<div className="text-muted-foreground">Loading...</div>
</div>
);
}
const allChannels = channels || [];
const allDevices = devices || [];
const selectedChannel = selectedChannelId
? allChannels.find((c) => c.id === selectedChannelId)
: null;
return (
<div className="h-full flex flex-col">
<div className="flex items-center justify-between mb-6 shrink-0">
<h2 className="text-2xl font-bold">Audio Channels</h2>
<Button onClick={() => setCreateDialogOpen(true)}>
<Plus className="h-4 w-4 mr-2" />
New Channel
</Button>
</div>
<div className="grid grid-cols-1 lg:grid-cols-2 gap-6 h-full min-h-0">
{/* Left Column - Channels */}
<div
className={cn(
'flex flex-col min-h-0 overflow-y-auto',
isPlayerVisible && BOTTOM_SAFE_AREA_PADDING,
)}
>
{allChannels.length === 0 ? (
<div className="flex flex-col items-center justify-center py-12 border-2 border-dashed border-muted rounded-md">
<Speaker className="h-12 w-12 text-muted-foreground mb-4" />
<p className="text-muted-foreground mb-4">
No audio channels yet. Create your first channel to route voices to specific
devices.
</p>
<Button onClick={() => setCreateDialogOpen(true)}>
<Plus className="h-4 w-4 mr-2" />
Create Channel
</Button>
</div>
) : (
<div className="space-y-3 p-2">
{allChannels.map((channel) => {
const isSelected = selectedChannelId === channel.id;
return (
<button
key={channel.id}
type="button"
className={cn(
'group border rounded-lg p-4 transition-colors cursor-pointer text-left w-full',
isSelected && 'ring-2 ring-primary bg-primary/5 border-primary',
)}
onClick={() => setSelectedChannelId(isSelected ? null : channel.id)}
>
<div className="flex items-start justify-between gap-4">
<div className="flex-1 min-w-0">
<div className="flex items-center gap-2 mb-3">
<div className="h-8 w-8 rounded-lg bg-muted flex items-center justify-center shrink-0">
<Speaker className="h-4 w-4 text-muted-foreground" />
</div>
<div className="flex items-center gap-2 min-w-0">
<h3 className="font-semibold text-base truncate">{channel.name}</h3>
</div>
</div>
<div className="space-y-2.5 ml-10">
<div>
<div className="text-xs font-medium text-muted-foreground mb-1">
Output Devices
</div>
<div className="flex flex-wrap gap-1.5">
{channel.device_ids.length > 0
? channel.device_ids.map((deviceId) => {
const device = allDevices.find((d) => d.id === deviceId);
return (
<Badge
key={deviceId}
variant="outline"
className="text-xs font-normal"
>
{device?.name || deviceId}
</Badge>
);
})
: (() => {
const defaultDevice = allDevices.find((d) => d.is_default);
return defaultDevice ? (
<Badge variant="outline" className="text-xs font-normal">
{defaultDevice.name}
</Badge>
) : null;
})()}
</div>
</div>
<div>
<div className="text-xs font-medium text-muted-foreground mb-1">
Assigned Voices
</div>
<ChannelVoicesList channelId={channel.id} />
</div>
</div>
</div>
{!channel.is_default && (
<div className="flex gap-1 shrink-0 opacity-0 group-hover:opacity-100 transition-opacity">
<Button
variant="ghost"
size="sm"
className="h-8 w-8 p-0"
onClick={(e) => {
e.stopPropagation();
setEditingChannel(channel.id);
}}
>
<Edit className="h-4 w-4" />
</Button>
<Button
variant="ghost"
size="sm"
className="h-8 w-8 p-0"
onClick={(e) => {
e.stopPropagation();
if (confirm('Delete this channel?')) {
deleteChannel.mutate(channel.id);
}
}}
>
<Trash2 className="h-4 w-4" />
</Button>
</div>
)}
</div>
</button>
);
})}
</div>
)}
</div>
{/* Right Column - Available Devices */}
<div
className={cn(
'flex flex-col min-h-0 overflow-y-auto',
isPlayerVisible && BOTTOM_SAFE_AREA_PADDING,
)}
>
<div className="shrink-0 mb-4">
<h3 className="text-lg font-semibold">Available Devices</h3>
<p className="text-sm text-muted-foreground mt-1">
{selectedChannelId
? selectedChannel?.is_default
? 'Default channel uses system default device'
: 'Click devices to add or remove them from the selected channel'
: 'Select a channel to assign devices'}
</p>
</div>
{allDevices.length > 0 ? (
<div className="space-y-2">
{allDevices.map((device) => {
const isConnected =
selectedChannelId &&
selectedChannel &&
(selectedChannel.device_ids.length === 0
? device.is_default
: selectedChannel.device_ids.includes(device.id));
const canToggle =
selectedChannelId && selectedChannel && !selectedChannel.is_default;
const handleDeviceClick = () => {
if (!canToggle || !selectedChannel) return;
const currentDeviceIds = selectedChannel.device_ids;
const newDeviceIds = isConnected
? currentDeviceIds.filter((id) => id !== device.id)
: [...currentDeviceIds, device.id];
updateChannel.mutate({
channelId: selectedChannelId,
data: { device_ids: newDeviceIds },
});
};
return (
<button
key={device.id}
type="button"
onClick={handleDeviceClick}
disabled={!canToggle}
className={cn(
'flex items-center gap-2 text-sm p-3 rounded-lg border transition-colors text-left w-full',
isConnected
? 'bg-primary/10 border-primary ring-1 ring-primary/20'
: 'hover:bg-muted/50',
!canToggle && 'cursor-default opacity-60',
canToggle && 'cursor-pointer',
)}
>
{canToggle ? (
<div
className={cn(
'h-4 w-4 rounded border-2 flex items-center justify-center shrink-0',
isConnected ? 'bg-accent border-accent' : 'border-muted-foreground/30',
)}
>
{isConnected && <Check className="h-3 w-3 text-accent-foreground" />}
</div>
) : device.is_default ? (
<CheckCircle2 className="h-4 w-4 text-primary shrink-0" />
) : null}
<span className={cn('truncate flex-1', device.is_default && 'font-medium')}>
{device.name}
</span>
</button>
);
})}
</div>
) : (
<div className="flex flex-col items-center justify-center py-12 border-2 border-dashed border-muted rounded-md">
<CheckCircle2 className="h-12 w-12 text-muted-foreground mb-4" />
<p className="text-muted-foreground text-center">
{isTauri() ? 'No audio devices found' : 'Audio device selection requires Tauri'}
</p>
</div>
)}
</div>
</div>
{/* Create Channel Dialog */}
<CreateChannelDialog
open={createDialogOpen}
onOpenChange={setCreateDialogOpen}
devices={devices || []}
onCreate={(name, deviceIds) => {
createChannel.mutate({ name, device_ids: deviceIds });
}}
/>
{/* Edit Channel Dialog */}
{editingChannel &&
(() => {
const channel = channels?.find((c) => c.id === editingChannel);
return channel ? (
<EditChannelDialog
open={!!editingChannel}
onOpenChange={(open) => !open && setEditingChannel(null)}
channel={channel}
devices={devices || []}
profiles={profiles || []}
channelVoices={channelVoices?.profile_ids || []}
onUpdate={(name, deviceIds) => {
updateChannel.mutate({
channelId: editingChannel,
data: { name, device_ids: deviceIds },
});
}}
onSetVoices={(profileIds) => {
setChannelVoices.mutate({
channelId: editingChannel,
profileIds,
});
}}
/>
) : null;
})()}
</div>
);
}
function ChannelVoicesList({ channelId }: { channelId: string }) {
const { data: voices } = useQuery({
queryKey: ['channel-voices', channelId],
queryFn: () => apiClient.getChannelVoices(channelId),
});
const { data: profiles } = useQuery({
queryKey: ['profiles'],
queryFn: () => apiClient.listProfiles(),
});
const voiceNames =
voices?.profile_ids.map((id) => profiles?.find((p) => p.id === id)?.name).filter(Boolean) || [];
return (
<div className="flex flex-wrap gap-1.5">
{voiceNames.length > 0 ? (
voiceNames.map((name) => (
<Badge key={name} variant="outline" className="text-xs font-normal">
{name}
</Badge>
))
) : (
<span className="text-sm text-muted-foreground">No voices assigned</span>
)}
</div>
);
}
interface CreateChannelDialogProps {
open: boolean;
onOpenChange: (open: boolean) => void;
devices: AudioDevice[];
onCreate: (name: string, deviceIds: string[]) => void;
}
function CreateChannelDialog({ open, onOpenChange, devices, onCreate }: CreateChannelDialogProps) {
const [name, setName] = useState('');
const [selectedDevices, setSelectedDevices] = useState<string[]>([]);
const handleSubmit = () => {
if (name.trim()) {
onCreate(name.trim(), selectedDevices);
setName('');
setSelectedDevices([]);
}
};
return (
<Dialog open={open} onOpenChange={onOpenChange}>
<DialogContent>
<DialogHeader>
<DialogTitle>Create Audio Channel</DialogTitle>
<DialogDescription>
Create a new audio channel (bus) to route voices to specific output devices.
</DialogDescription>
</DialogHeader>
<div className="space-y-4">
<div>
<Label htmlFor="channel-name">Channel Name</Label>
<Input
id="channel-name"
value={name}
onChange={(e) => setName(e.target.value)}
placeholder="e.g., Virtual Cable, Broadcast"
/>
</div>
<div>
<Label>Output Devices</Label>
<Select
value={selectedDevices[0] || ''}
onValueChange={(value) => {
if (value && !selectedDevices.includes(value)) {
setSelectedDevices([...selectedDevices, value]);
}
}}
>
<SelectTrigger>
<SelectValue placeholder="Select device" />
</SelectTrigger>
<SelectContent>
{devices.map((device) => (
<SelectItem key={device.id} value={device.id}>
{device.name} {device.is_default && '(default)'}
</SelectItem>
))}
</SelectContent>
</Select>
{selectedDevices.length > 0 && (
<div className="mt-2 space-y-1">
{selectedDevices.map((deviceId) => {
const device = devices.find((d) => d.id === deviceId);
return (
<div
key={deviceId}
className="flex items-center justify-between text-sm bg-muted p-2 rounded"
>
<span>{device?.name || deviceId}</span>
<Button
variant="ghost"
size="sm"
onClick={() =>
setSelectedDevices(selectedDevices.filter((id) => id !== deviceId))
}
>
<Trash2 className="h-3 w-3" />
</Button>
</div>
);
})}
</div>
)}
</div>
</div>
<DialogFooter>
<Button variant="outline" onClick={() => onOpenChange(false)}>
Cancel
</Button>
<Button onClick={handleSubmit} disabled={!name.trim()}>
Create
</Button>
</DialogFooter>
</DialogContent>
</Dialog>
);
}
interface EditChannelDialogProps {
open: boolean;
onOpenChange: (open: boolean) => void;
channel: {
id: string;
name: string;
device_ids: string[];
};
devices: AudioDevice[];
profiles: Array<{ id: string; name: string }>;
channelVoices: string[];
onUpdate: (name: string, deviceIds: string[]) => void;
onSetVoices: (profileIds: string[]) => void;
}
function EditChannelDialog({
open,
onOpenChange,
channel,
devices,
profiles,
channelVoices,
onUpdate,
onSetVoices,
}: EditChannelDialogProps) {
const [name, setName] = useState(channel.name);
const [selectedDevices, setSelectedDevices] = useState<string[]>(channel.device_ids);
const [selectedVoices, setSelectedVoices] = useState<string[]>(channelVoices);
const handleSubmit = () => {
if (name.trim()) {
onUpdate(name.trim(), selectedDevices);
onSetVoices(selectedVoices);
}
};
return (
<Dialog open={open} onOpenChange={onOpenChange}>
<DialogContent className="max-w-2xl">
<DialogHeader>
<DialogTitle>Edit Channel</DialogTitle>
<DialogDescription>Update channel settings and voice assignments.</DialogDescription>
</DialogHeader>
<div className="space-y-4">
<div>
<Label htmlFor="edit-channel-name">Channel Name</Label>
<Input id="edit-channel-name" value={name} onChange={(e) => setName(e.target.value)} />
</div>
<div>
<Label>Output Devices</Label>
<Select
value=""
onValueChange={(value) => {
if (value && !selectedDevices.includes(value)) {
setSelectedDevices([...selectedDevices, value]);
}
}}
>
<SelectTrigger>
<SelectValue placeholder="Add device" />
</SelectTrigger>
<SelectContent>
{devices.map((device) => (
<SelectItem key={device.id} value={device.id}>
{device.name} {device.is_default && '(default)'}
</SelectItem>
))}
</SelectContent>
</Select>
{selectedDevices.length > 0 && (
<div className="mt-2 space-y-1">
{selectedDevices.map((deviceId) => {
const device = devices.find((d) => d.id === deviceId);
return (
<div
key={deviceId}
className="flex items-center justify-between text-sm bg-muted p-2 rounded"
>
<span>{device?.name || deviceId}</span>
<Button
variant="ghost"
size="sm"
onClick={() =>
setSelectedDevices(selectedDevices.filter((id) => id !== deviceId))
}
>
<Trash2 className="h-3 w-3" />
</Button>
</div>
);
})}
</div>
)}
</div>
<div>
<Label>Assigned Voices</Label>
<Select
value=""
onValueChange={(value) => {
if (value && !selectedVoices.includes(value)) {
setSelectedVoices([...selectedVoices, value]);
}
}}
>
<SelectTrigger>
<SelectValue placeholder="Add voice" />
</SelectTrigger>
<SelectContent>
{profiles.map((profile) => (
<SelectItem key={profile.id} value={profile.id}>
{profile.name}
</SelectItem>
))}
</SelectContent>
</Select>
{selectedVoices.length > 0 && (
<div className="mt-2 space-y-1">
{selectedVoices.map((profileId) => {
const profile = profiles.find((p) => p.id === profileId);
return (
<div
key={profileId}
className="flex items-center justify-between text-sm bg-muted p-2 rounded"
>
<span>{profile?.name || profileId}</span>
<Button
variant="ghost"
size="sm"
onClick={() =>
setSelectedVoices(selectedVoices.filter((id) => id !== profileId))
}
>
<Trash2 className="h-3 w-3" />
</Button>
</div>
);
})}
</div>
)}
</div>
</div>
<DialogFooter>
<Button variant="outline" onClick={() => onOpenChange(false)}>
Cancel
</Button>
<Button onClick={handleSubmit} disabled={!name.trim()}>
Save
</Button>
</DialogFooter>
</DialogContent>
</Dialog>
);
}
-1
View File
@@ -1 +0,0 @@
# Voice generation components
@@ -0,0 +1,378 @@
import { useMatchRoute } from '@tanstack/react-router';
import { AnimatePresence, motion } from 'framer-motion';
import { Loader2, MessageSquare, Sparkles } from 'lucide-react';
import { useEffect, useRef, useState } from 'react';
import { Button } from '@/components/ui/button';
import { Form, FormControl, FormField, FormItem, FormMessage } from '@/components/ui/form';
import {
Select,
SelectContent,
SelectItem,
SelectTrigger,
SelectValue,
} from '@/components/ui/select';
import { Textarea } from '@/components/ui/textarea';
import { useToast } from '@/components/ui/use-toast';
import { LANGUAGE_OPTIONS } from '@/lib/constants/languages';
import { useGenerationForm } from '@/lib/hooks/useGenerationForm';
import { useProfile, useProfiles } from '@/lib/hooks/useProfiles';
import { useAddStoryItem, useStory } from '@/lib/hooks/useStories';
import { cn } from '@/lib/utils/cn';
import { useStoryStore } from '@/stores/storyStore';
import { useUIStore } from '@/stores/uiStore';
interface FloatingGenerateBoxProps {
isPlayerOpen?: boolean;
showVoiceSelector?: boolean;
}
export function FloatingGenerateBox({
isPlayerOpen = false,
showVoiceSelector = false,
}: FloatingGenerateBoxProps) {
const selectedProfileId = useUIStore((state) => state.selectedProfileId);
const setSelectedProfileId = useUIStore((state) => state.setSelectedProfileId);
const { data: selectedProfile } = useProfile(selectedProfileId || '');
const { data: profiles } = useProfiles();
const [isExpanded, setIsExpanded] = useState(false);
const [isInstructMode, setIsInstructMode] = useState(false);
const containerRef = useRef<HTMLDivElement>(null);
const textareaRef = useRef<HTMLTextAreaElement | null>(null);
const matchRoute = useMatchRoute();
const isStoriesRoute = matchRoute({ to: '/stories' });
const selectedStoryId = useStoryStore((state) => state.selectedStoryId);
const trackEditorHeight = useStoryStore((state) => state.trackEditorHeight);
const { data: currentStory } = useStory(selectedStoryId);
const addStoryItem = useAddStoryItem();
const { toast } = useToast();
// Calculate if track editor is visible (on stories route with items)
const hasTrackEditor = isStoriesRoute && currentStory && currentStory.items.length > 0;
const { form, handleSubmit, isPending } = useGenerationForm({
onSuccess: async (generationId) => {
setIsExpanded(false);
// If on stories route and a story is selected, add generation to story
if (isStoriesRoute && selectedStoryId && generationId) {
try {
await addStoryItem.mutateAsync({
storyId: selectedStoryId,
data: { generation_id: generationId },
});
toast({
title: 'Added to story',
description: `Generation added to "${currentStory?.name || 'story'}"`,
});
} catch (error) {
toast({
title: 'Failed to add to story',
description:
error instanceof Error ? error.message : 'Could not add generation to story',
variant: 'destructive',
});
}
}
},
});
// Click away handler to collapse the box
useEffect(() => {
function handleClickOutside(event: MouseEvent) {
const target = event.target as HTMLElement;
// Don't collapse if clicking inside the container
if (containerRef.current?.contains(target)) {
return;
}
// Don't collapse if clicking on a Select dropdown (which renders in a portal)
if (
target.closest('[role="listbox"]') ||
target.closest('[data-radix-popper-content-wrapper]')
) {
return;
}
setIsExpanded(false);
}
if (isExpanded) {
document.addEventListener('mousedown', handleClickOutside);
}
return () => {
document.removeEventListener('mousedown', handleClickOutside);
};
}, [isExpanded]);
// Set first voice as default if none selected
useEffect(() => {
if (!selectedProfileId && profiles && profiles.length > 0) {
setSelectedProfileId(profiles[0].id);
}
}, [selectedProfileId, profiles, setSelectedProfileId]);
// Get current form value to trigger resize when it changes
const formValue = form.watch(isInstructMode ? 'instruct' : 'text');
// Auto-resize textarea based on content (only when expanded)
useEffect(() => {
if (!isExpanded) {
// Reset textarea height after collapse animation completes
const timeoutId = setTimeout(() => {
const textarea = textareaRef.current;
if (textarea) {
textarea.style.height = '32px';
textarea.style.overflowY = 'hidden';
}
}, 200); // Wait for animation to complete
return () => clearTimeout(timeoutId);
}
const textarea = textareaRef.current;
if (!textarea) return;
const adjustHeight = () => {
textarea.style.height = 'auto';
const scrollHeight = textarea.scrollHeight;
const minHeight = 100; // Expanded minimum
const maxHeight = 300; // Max height in pixels
const targetHeight = Math.max(minHeight, Math.min(scrollHeight, maxHeight));
textarea.style.height = `${targetHeight}px`;
// Show scrollbar if content exceeds max height
if (scrollHeight > maxHeight) {
textarea.style.overflowY = 'auto';
} else {
textarea.style.overflowY = 'hidden';
}
};
// Small delay to let framer animation complete
const timeoutId = setTimeout(() => {
adjustHeight();
}, 200);
// Adjust on mount and when value changes
adjustHeight();
// Watch for input changes
textarea.addEventListener('input', adjustHeight);
return () => {
clearTimeout(timeoutId);
textarea.removeEventListener('input', adjustHeight);
};
}, [isExpanded]);
async function onSubmit(data: Parameters<typeof handleSubmit>[0]) {
await handleSubmit(data, selectedProfileId);
}
return (
<motion.div
ref={containerRef}
className={cn(
'fixed right-auto',
isStoriesRoute
? // Position aligned with story list: after sidebar + padding, width 360px
'left-[calc(5rem+2rem)] w-[360px]'
: 'left-[calc(5rem+2rem)] w-[calc((100%-5rem-4rem)/2-1rem)]',
)}
style={{
// On stories route: offset by track editor height when visible
// On other routes: offset by audio player height when visible
bottom: hasTrackEditor
? `${trackEditorHeight + 24}px`
: isPlayerOpen
? 'calc(7rem + 1.5rem)'
: '1.5rem',
}}
>
<motion.div
className="bg-background/30 backdrop-blur-2xl border border-accent/20 rounded-[2rem] shadow-2xl hover:bg-background/40 hover:border-accent/20 transition-all duration-300 overflow-hidden p-3"
transition={{ duration: 0.6, ease: 'easeInOut' }}
>
<Form {...form}>
<form onSubmit={form.handleSubmit(onSubmit)}>
<div className="flex gap-2">
<motion.div className="flex-1" transition={{ duration: 0.3, ease: 'easeOut' }}>
{isInstructMode && (
<span className="text-xs text-accent font-medium mb-1 block">
Delivery instructions:
</span>
)}
<FormField
control={form.control}
name={isInstructMode ? 'instruct' : 'text'}
render={({ field }) => (
<FormItem>
<FormControl>
<motion.div
animate={{
height: isExpanded ? 'auto' : '32px',
}}
transition={{ duration: 0.15, ease: 'easeOut' }}
style={{ overflow: 'hidden' }}
>
<Textarea
{...field}
ref={(node: HTMLTextAreaElement | null) => {
// Store ref for auto-resize
textareaRef.current = node;
// Forward ref to react-hook-form
if (typeof field.ref === 'function') {
field.ref(node);
}
}}
placeholder={
isInstructMode
? 'Add delivery instructions...'
: isStoriesRoute && currentStory
? `Generate speech for "${currentStory.name}"...`
: selectedProfile
? `Generate speech using ${selectedProfile.name}...`
: 'Select a voice profile above...'
}
className="resize-none bg-transparent border-none focus-visible:ring-0 focus-visible:ring-offset-0 focus:outline-none focus:ring-0 outline-none ring-0 rounded-2xl text-sm placeholder:text-muted-foreground/60 w-full"
style={{
minHeight: isExpanded ? '100px' : '32px',
maxHeight: '300px',
}}
disabled={!selectedProfileId}
onClick={() => setIsExpanded(true)}
onFocus={() => setIsExpanded(true)}
/>
</motion.div>
</FormControl>
<FormMessage className="text-xs" />
</FormItem>
)}
/>
</motion.div>
<div className="relative shrink-0">
<Button
type="submit"
disabled={isPending || !selectedProfileId}
className="h-10 w-10 rounded-full bg-accent hover:bg-accent/90 hover:scale-105 text-accent-foreground shadow-lg hover:shadow-accent/50 transition-all duration-200"
size="icon"
>
{isPending ? (
<Loader2 className="h-4 w-4 animate-spin" />
) : (
<Sparkles className="h-4 w-4" />
)}
</Button>
<AnimatePresence>
{isExpanded && (
<motion.div
initial={{ opacity: 0, scale: 0.8 }}
animate={{ opacity: 1, scale: 1 }}
exit={{ opacity: 0, scale: 0.8 }}
transition={{ duration: 0.2 }}
className="absolute top-0 right-[calc(100%+0.5rem)]"
>
<Button
type="button"
variant="ghost"
size="icon"
onClick={() => setIsInstructMode(!isInstructMode)}
className={`h-10 w-10 rounded-full bg-card border border-border hover:bg-background/50 transition-all duration-200 ${
isInstructMode ? 'text-accent' : ''
}`}
>
<MessageSquare className="h-4 w-4" />
</Button>
</motion.div>
)}
</AnimatePresence>
</div>
</div>
<AnimatePresence>
<motion.div
initial={{ height: 0, opacity: 0 }}
animate={{ height: 'auto', opacity: 1 }}
exit={{ height: 0, opacity: 0 }}
transition={{ duration: 0.3, ease: 'easeOut' }}
className=" mt-3"
>
<div className="flex items-center gap-2">
{showVoiceSelector && (
<div className="flex-1">
<Select
value={selectedProfileId || ''}
onValueChange={(value) => setSelectedProfileId(value || null)}
>
<SelectTrigger className="h-8 text-xs bg-card border-border rounded-full hover:bg-background/50 transition-all w-full">
<SelectValue placeholder="Select a voice..." />
</SelectTrigger>
<SelectContent>
{profiles?.map((profile) => (
<SelectItem key={profile.id} value={profile.id} className="text-xs">
{profile.name}
</SelectItem>
))}
</SelectContent>
</Select>
</div>
)}
<FormField
control={form.control}
name="language"
render={({ field }) => (
<FormItem className="flex-1 space-y-0">
<Select onValueChange={field.onChange} defaultValue={field.value}>
<FormControl>
<SelectTrigger className="h-8 text-xs bg-card border-border rounded-full hover:bg-background/50 transition-all">
<SelectValue />
</SelectTrigger>
</FormControl>
<SelectContent>
{LANGUAGE_OPTIONS.map((lang) => (
<SelectItem key={lang.value} value={lang.value} className="text-xs">
{lang.label}
</SelectItem>
))}
</SelectContent>
</Select>
<FormMessage className="text-xs" />
</FormItem>
)}
/>
<FormField
control={form.control}
name="modelSize"
render={({ field }) => (
<FormItem className="flex-1 space-y-0">
<Select onValueChange={field.onChange} defaultValue={field.value}>
<FormControl>
<SelectTrigger className="h-8 text-xs bg-card border-border rounded-full hover:bg-background/50 transition-all">
<SelectValue />
</SelectTrigger>
</FormControl>
<SelectContent>
<SelectItem value="1.7B" className="text-xs text-muted-foreground">
Qwen3-TTS 1.7B
</SelectItem>
<SelectItem value="0.6B" className="text-xs text-muted-foreground">
Qwen3-TTS 0.6B
</SelectItem>
</SelectContent>
</Select>
<FormMessage className="text-xs" />
</FormItem>
)}
/>
</div>
</motion.div>
</AnimatePresence>
</form>
</Form>
</motion.div>
</motion.div>
);
}
@@ -1,8 +1,4 @@
import { zodResolver } from '@hookform/resolvers/zod';
import { Loader2, Mic } from 'lucide-react';
import { useState } from 'react';
import { useForm } from 'react-hook-form';
import * as z from 'zod';
import { Button } from '@/components/ui/button';
import { Card, CardContent, CardHeader, CardTitle } from '@/components/ui/card';
import {
@@ -23,118 +19,19 @@ import {
SelectValue,
} from '@/components/ui/select';
import { Textarea } from '@/components/ui/textarea';
import { useToast } from '@/components/ui/use-toast';
import { apiClient } from '@/lib/api/client';
import { LANGUAGE_CODES, LANGUAGE_OPTIONS, type LanguageCode } from '@/lib/constants/languages';
import { useGeneration } from '@/lib/hooks/useGeneration';
import { useModelDownloadToast } from '@/lib/hooks/useModelDownloadToast';
import { LANGUAGE_OPTIONS } from '@/lib/constants/languages';
import { useGenerationForm } from '@/lib/hooks/useGenerationForm';
import { useProfile } from '@/lib/hooks/useProfiles';
import { useGenerationStore } from '@/stores/generationStore';
import { usePlayerStore } from '@/stores/playerStore';
import { useUIStore } from '@/stores/uiStore';
const generationSchema = z.object({
text: z.string().min(1, 'Text is required').max(5000),
language: z.enum(LANGUAGE_CODES as [LanguageCode, ...LanguageCode[]]),
seed: z.number().int().optional(),
modelSize: z.enum(['1.7B', '0.6B']).optional(),
instruct: z.string().max(500).optional(),
});
type GenerationFormValues = z.infer<typeof generationSchema>;
export function GenerationForm() {
const selectedProfileId = useUIStore((state) => state.selectedProfileId);
const { data: selectedProfile } = useProfile(selectedProfileId || '');
const generation = useGeneration();
const { toast } = useToast();
const setAudio = usePlayerStore((state) => state.setAudio);
const setIsGenerating = useGenerationStore((state) => state.setIsGenerating);
const [downloadingModelName, setDownloadingModelName] = useState<string | null>(null);
const [downloadingDisplayName, setDownloadingDisplayName] = useState<string | null>(null);
// Use the download toast hook to show progress when model is downloading
useModelDownloadToast({
modelName: downloadingModelName || '',
displayName: downloadingDisplayName || '',
enabled: !!downloadingModelName,
});
const { form, handleSubmit, isPending } = useGenerationForm();
const form = useForm<GenerationFormValues>({
resolver: zodResolver(generationSchema),
defaultValues: {
text: '',
language: 'en',
seed: undefined,
modelSize: '1.7B',
instruct: '',
},
});
async function onSubmit(data: GenerationFormValues) {
if (!selectedProfileId) {
toast({
title: 'No profile selected',
description: 'Please select a voice profile from the cards above.',
variant: 'destructive',
});
return;
}
try {
setIsGenerating(true);
// Determine model name and display name
const modelName = `qwen-tts-${data.modelSize}`;
const displayName = data.modelSize === '1.7B' ? 'Qwen TTS 1.7B' : 'Qwen TTS 0.6B';
// Check if model is downloaded before starting generation
try {
const modelStatus = await apiClient.getModelStatus();
const model = modelStatus.models.find((m) => m.model_name === modelName);
if (model && !model.downloaded) {
// Model is not downloaded, enable download toast
setDownloadingModelName(modelName);
setDownloadingDisplayName(displayName);
}
} catch (error) {
// If status check fails, continue anyway - generation will handle it
console.error('Failed to check model status:', error);
}
// Proceed with generation (which will trigger download if needed)
const result = await generation.mutateAsync({
profile_id: selectedProfileId,
text: data.text,
language: data.language,
seed: data.seed,
model_size: data.modelSize,
instruct: data.instruct || undefined,
});
toast({
title: 'Generation complete!',
description: `Audio generated (${result.duration.toFixed(2)}s)`,
});
// Autoplay the generated audio
const audioUrl = apiClient.getAudioUrl(result.id);
setAudio(audioUrl, result.id, data.text.substring(0, 50));
form.reset();
} catch (error) {
toast({
title: 'Generation failed',
description: error instanceof Error ? error.message : 'Failed to generate audio',
variant: 'destructive',
});
} finally {
setIsGenerating(false);
// Clear download state after generation completes
setDownloadingModelName(null);
setDownloadingDisplayName(null);
}
async function onSubmit(data: Parameters<typeof handleSubmit>[0]) {
await handleSubmit(data, selectedProfileId);
}
return (
@@ -276,9 +173,9 @@ export function GenerationForm() {
<Button
type="submit"
className="w-full"
disabled={generation.isPending || !selectedProfileId}
disabled={isPending || !selectedProfileId}
>
{generation.isPending ? (
{isPending ? (
<>
<Loader2 className="mr-2 h-4 w-4 animate-spin" />
Generating...
+44 -52
View File
@@ -16,7 +16,9 @@ import {
DropdownMenuTrigger,
} from '@/components/ui/dropdown-menu';
import { Textarea } from '@/components/ui/textarea';
import { useToast } from '@/components/ui/use-toast';
import { apiClient } from '@/lib/api/client';
import { BOTTOM_SAFE_AREA_PADDING } from '@/lib/constants/ui';
import {
useDeleteGeneration,
useExportGeneration,
@@ -33,13 +35,14 @@ import { usePlayerStore } from '@/stores/playerStore';
// NEW ALTERNATE HISTORY VIEW - FIXED HEIGHT ROWS
export function HistoryTable() {
const [page, setPage] = useState(0);
const [page, _setPage] = useState(0);
const [isScrolled, setIsScrolled] = useState(false);
const scrollRef = useRef<HTMLDivElement>(null);
const fileInputRef = useRef<HTMLInputElement>(null);
const [importDialogOpen, setImportDialogOpen] = useState(false);
const [selectedFile, setSelectedFile] = useState<File | null>(null);
const limit = 20;
const { toast } = useToast();
const { data: historyData, isLoading } = useHistory({
limit,
@@ -50,7 +53,7 @@ export function HistoryTable() {
const exportGeneration = useExportGeneration();
const exportGenerationAudio = useExportGenerationAudio();
const importGeneration = useImportGeneration();
const setAudio = usePlayerStore((state) => state.setAudio);
const setAudioWithAutoPlay = usePlayerStore((state) => state.setAudioWithAutoPlay);
const restartCurrentAudio = usePlayerStore((state) => state.restartCurrentAudio);
const currentAudioId = usePlayerStore((state) => state.audioId);
const isPlaying = usePlayerStore((state) => state.isPlaying);
@@ -69,14 +72,14 @@ export function HistoryTable() {
return () => scrollEl.removeEventListener('scroll', handleScroll);
}, []);
const handlePlay = (audioId: string, text: string) => {
const handlePlay = (audioId: string, text: string, profileId: string) => {
// If clicking the same audio, restart it from the beginning
if (currentAudioId === audioId) {
restartCurrentAudio();
} else {
// Otherwise, load the new audio
// Otherwise, load the new audio and auto-play it
const audioUrl = apiClient.getAudioUrl(audioId);
setAudio(audioUrl, audioId, text.substring(0, 50));
setAudioWithAutoPlay(audioUrl, audioId, profileId, text.substring(0, 50));
}
};
@@ -85,7 +88,11 @@ export function HistoryTable() {
{ generationId, text },
{
onError: (error) => {
alert(`Failed to download audio: ${error.message}`);
toast({
title: 'Failed to download audio',
description: error.message,
variant: 'destructive',
});
},
},
);
@@ -96,7 +103,11 @@ export function HistoryTable() {
{ generationId, text },
{
onError: (error) => {
alert(`Failed to export generation: ${error.message}`);
toast({
title: 'Failed to export generation',
description: error.message,
variant: 'destructive',
});
},
},
);
@@ -111,7 +122,11 @@ export function HistoryTable() {
if (file) {
// Validate file extension
if (!file.name.endsWith('.voicebox.zip')) {
alert('Please select a valid .voicebox.zip file');
toast({
title: 'Invalid file type',
description: 'Please select a valid .voicebox.zip file',
variant: 'destructive',
});
return;
}
setSelectedFile(file);
@@ -128,10 +143,17 @@ export function HistoryTable() {
if (fileInputRef.current) {
fileInputRef.current.value = '';
}
alert(data.message || 'Generation imported successfully');
toast({
title: 'Generation imported',
description: data.message || 'Generation imported successfully',
});
},
onError: (error) => {
alert(`Failed to import generation: ${error.message}`);
toast({
title: 'Failed to import generation',
description: error.message,
variant: 'destructive',
});
},
});
}
@@ -143,27 +165,10 @@ export function HistoryTable() {
const history = historyData?.items || [];
const total = historyData?.total || 0;
const hasMore = history.length === limit && (page + 1) * limit < total;
const _hasMore = history.length === limit && (page + 1) * limit < total;
return (
<div className="flex flex-col h-full min-h-0 relative">
{/* <div className="flex justify-between items-center mb-4 shrink-0">
<h2 className="text-2xl font-bold">History</h2>
<div className="flex gap-2">
<Button variant="outline" onClick={handleImportClick}>
<Upload className="mr-2 h-4 w-4" />
Import Generation
</Button>
<input
ref={fileInputRef}
type="file"
accept=".voicebox.zip"
onChange={handleFileChange}
className="hidden"
/>
</div>
</div> */}
{history.length === 0 ? (
<div className="text-center py-12 px-5 border-2 border-dashed mb-5 border-muted rounded-md text-muted-foreground flex-1 flex items-center justify-center">
No voice generations, yet...
@@ -176,8 +181,8 @@ export function HistoryTable() {
<div
ref={scrollRef}
className={cn(
'flex-1 min-h-0 overflow-y-auto space-y-2',
isPlayerVisible && 'max-h-[calc(100vh-117px)]',
'flex-1 min-h-0 overflow-y-auto space-y-2 pb-4',
isPlayerVisible && BOTTOM_SAFE_AREA_PADDING,
)}
>
{history.map((gen) => {
@@ -195,7 +200,7 @@ export function HistoryTable() {
if (target.closest('textarea') || window.getSelection()?.toString()) {
return;
}
handlePlay(gen.id, gen.text);
handlePlay(gen.id, gen.text, gen.profile_id);
}}
>
{/* Waveform icon */}
@@ -228,7 +233,11 @@ export function HistoryTable() {
</div>
{/* Far right - Ellipsis actions */}
<div className="w-10 shrink-0 flex justify-end">
<div
className="w-10 shrink-0 flex justify-end"
onMouseDown={(e) => e.stopPropagation()}
onClick={(e) => e.stopPropagation()}
>
<DropdownMenu>
<DropdownMenuTrigger asChild>
<Button
@@ -236,13 +245,14 @@ export function HistoryTable() {
size="icon"
className="h-8 w-8"
aria-label="Actions"
onClick={(e) => e.stopPropagation()}
>
<MoreHorizontal className="h-4 w-4" />
</Button>
</DropdownMenuTrigger>
<DropdownMenuContent align="end">
<DropdownMenuItem onClick={() => handlePlay(gen.id, gen.text)}>
<DropdownMenuItem
onClick={() => handlePlay(gen.id, gen.text, gen.profile_id)}
>
<Play className="mr-2 h-4 w-4" />
Play
</DropdownMenuItem>
@@ -275,24 +285,6 @@ export function HistoryTable() {
);
})}
</div>
{(total > limit || page > 0) && (
<div className="flex justify-between items-center mt-4 shrink-0">
<Button
variant="outline"
onClick={() => setPage((p) => Math.max(0, p - 1))}
disabled={page === 0}
>
Previous
</Button>
<div className="text-sm text-muted-foreground">
Page {page + 1} • {total} total
</div>
<Button variant="outline" onClick={() => setPage((p) => p + 1)} disabled={!hasMore}>
Next
</Button>
</div>
)}
</>
)}
@@ -0,0 +1,168 @@
import { Sparkles, Upload } from 'lucide-react';
import { useRef, useState } from 'react';
import { FloatingGenerateBox } from '@/components/Generation/FloatingGenerateBox';
import { HistoryTable } from '@/components/History/HistoryTable';
import { Button } from '@/components/ui/button';
import {
Dialog,
DialogContent,
DialogDescription,
DialogFooter,
DialogHeader,
DialogTitle,
} from '@/components/ui/dialog';
import { useToast } from '@/components/ui/use-toast';
import { ProfileList } from '@/components/VoiceProfiles/ProfileList';
import { BOTTOM_SAFE_AREA_PADDING } from '@/lib/constants/ui';
import { useImportProfile } from '@/lib/hooks/useProfiles';
import { cn } from '@/lib/utils/cn';
import { usePlayerStore } from '@/stores/playerStore';
import { useUIStore } from '@/stores/uiStore';
export function MainEditor() {
const audioUrl = usePlayerStore((state) => state.audioUrl);
const isPlayerVisible = !!audioUrl;
const scrollRef = useRef<HTMLDivElement>(null);
const setDialogOpen = useUIStore((state) => state.setProfileDialogOpen);
const importProfile = useImportProfile();
const fileInputRef = useRef<HTMLInputElement>(null);
const [importDialogOpen, setImportDialogOpen] = useState(false);
const [selectedFile, setSelectedFile] = useState<File | null>(null);
const { toast } = useToast();
const handleImportClick = () => {
fileInputRef.current?.click();
};
const handleFileChange = (e: React.ChangeEvent<HTMLInputElement>) => {
const file = e.target.files?.[0];
if (file) {
if (!file.name.endsWith('.voicebox.zip')) {
toast({
title: 'Invalid file type',
description: 'Please select a valid .voicebox.zip file',
variant: 'destructive',
});
return;
}
setSelectedFile(file);
setImportDialogOpen(true);
}
};
const handleImportConfirm = () => {
if (selectedFile) {
importProfile.mutate(selectedFile, {
onSuccess: () => {
setImportDialogOpen(false);
setSelectedFile(null);
if (fileInputRef.current) {
fileInputRef.current.value = '';
}
toast({
title: 'Profile imported',
description: 'Voice profile imported successfully',
});
},
onError: (error) => {
toast({
title: 'Failed to import profile',
description: error.message,
variant: 'destructive',
});
},
});
}
};
return (
// Main view: Profiles top left, Generator bottom left, History right
<div className="grid grid-cols-1 lg:grid-cols-2 gap-6 h-full min-h-0 overflow-hidden relative">
{/* Left Column */}
<div className="flex flex-col min-h-0 overflow-hidden relative">
{/* Scroll Mask - Always visible, behind content */}
<div className="absolute top-0 left-0 right-0 h-16 bg-gradient-to-b from-background to-transparent z-0 pointer-events-none" />
{/* Fixed Header */}
<div className="absolute top-0 left-0 right-0 z-10">
<div className="flex items-center justify-between mb-4 px-1">
<h2 className="text-2xl font-bold">Voicebox</h2>
<div className="flex gap-2">
<Button variant="outline" onClick={handleImportClick}>
<Upload className="mr-2 h-4 w-4" />
Import Voice
</Button>
<input
ref={fileInputRef}
type="file"
accept=".voicebox.zip"
onChange={handleFileChange}
className="hidden"
/>
<Button onClick={() => setDialogOpen(true)}>
<Sparkles className="mr-2 h-4 w-4" />
Create Voice
</Button>
</div>
</div>
</div>
{/* Scrollable Content */}
<div
ref={scrollRef}
className={cn(
'flex-1 min-h-0 overflow-y-auto pt-14',
isPlayerVisible ? BOTTOM_SAFE_AREA_PADDING : 'pb-4',
)}
>
<div className="flex flex-col gap-6">
<div className="shrink-0 flex flex-col">
<ProfileList />
</div>
</div>
</div>
</div>
{/* Right Column - History */}
<div className="flex flex-col min-h-0 overflow-hidden">
<HistoryTable />
</div>
{/* Floating Generate Box */}
<FloatingGenerateBox isPlayerOpen={!!audioUrl} />
{/* Import Dialog */}
<Dialog open={importDialogOpen} onOpenChange={setImportDialogOpen}>
<DialogContent>
<DialogHeader>
<DialogTitle>Import Profile</DialogTitle>
<DialogDescription>
Import the profile from "{selectedFile?.name}". This will create a new profile with
all samples.
</DialogDescription>
</DialogHeader>
<DialogFooter>
<Button
variant="outline"
onClick={() => {
setImportDialogOpen(false);
setSelectedFile(null);
if (fileInputRef.current) {
fileInputRef.current.value = '';
}
}}
>
Cancel
</Button>
<Button
onClick={handleImportConfirm}
disabled={importProfile.isPending || !selectedFile}
>
{importProfile.isPending ? 'Importing...' : 'Import'}
</Button>
</DialogFooter>
</DialogContent>
</Dialog>
</div>
);
}
@@ -0,0 +1,9 @@
import { ModelManagement } from '@/components/ServerSettings/ModelManagement';
export function ModelsTab() {
return (
<div className="space-y-4 overflow-y-auto flex flex-col">
<ModelManagement />
</div>
);
}
@@ -1 +0,0 @@
# Server settings and connection components
@@ -1,6 +1,6 @@
import { zodResolver } from '@hookform/resolvers/zod';
import { useForm } from 'react-hook-form';
import { useEffect } from 'react';
import { useForm } from 'react-hook-form';
import * as z from 'zod';
import { Button } from '@/components/ui/button';
import { Card, CardContent, CardHeader, CardTitle } from '@/components/ui/card';
@@ -1,13 +1,6 @@
import { useQuery, useMutation, useQueryClient } from '@tanstack/react-query';
import { useMutation, useQuery, useQueryClient } from '@tanstack/react-query';
import { Download, Loader2, Trash2 } from 'lucide-react';
import { useState } from 'react';
import { apiClient } from '@/lib/api/client';
import { Card, CardContent, CardHeader, CardTitle, CardDescription } from '@/components/ui/card';
import { Button } from '@/components/ui/button';
import { Badge } from '@/components/ui/badge';
import { Loader2, Download, CheckCircle2, Trash2 } from 'lucide-react';
import { ModelProgress } from './ModelProgress';
import { useToast } from '@/components/ui/use-toast';
import { useModelDownloadToast } from '@/lib/hooks/useModelDownloadToast';
import {
AlertDialog,
AlertDialogAction,
@@ -18,6 +11,13 @@ import {
AlertDialogHeader,
AlertDialogTitle,
} from '@/components/ui/alert-dialog';
import { Badge } from '@/components/ui/badge';
import { Button } from '@/components/ui/button';
import { Card, CardContent, CardDescription, CardHeader, CardTitle } from '@/components/ui/card';
import { useToast } from '@/components/ui/use-toast';
import { apiClient } from '@/lib/api/client';
import { useModelDownloadToast } from '@/lib/hooks/useModelDownloadToast';
import { ModelProgress } from './ModelProgress';
export function ModelManagement() {
const { toast } = useToast();
@@ -197,8 +197,8 @@ export function ModelManagement() {
{modelToDelete?.sizeMb && (
<>
{' '}
This will free up {formatSize(modelToDelete.sizeMb)} of disk space. The model
will need to be re-downloaded if you want to use it again.
This will free up {formatSize(modelToDelete.sizeMb)} of disk space. The model will
need to be re-downloaded if you want to use it again.
</>
)}
</AlertDialogDescription>
@@ -244,13 +244,7 @@ interface ModelItemProps {
formatSize: (sizeMb?: number) => string;
}
function ModelItem({
model,
onDownload,
onDelete,
isDownloading,
formatSize,
}: ModelItemProps) {
function ModelItem({ model, onDownload, onDelete, isDownloading, formatSize }: ModelItemProps) {
return (
<div className="flex items-center justify-between p-3 border rounded-lg">
<div className="flex-1">
@@ -277,14 +271,12 @@ function ModelItem({
{model.downloaded ? (
<div className="flex items-center gap-2">
<div className="flex items-center gap-1 text-sm text-muted-foreground">
<CheckCircle2 className="h-4 w-4 text-green-500" />
<span>Ready</span>
</div>
<Button
size="sm"
onClick={onDelete}
variant="outline"
className="text-destructive hover:text-destructive"
disabled={model.loaded}
title={model.loaded ? 'Unload model before deleting' : 'Delete model'}
>
@@ -1,9 +1,9 @@
import { Loader2, XCircle } from 'lucide-react';
import { useEffect, useState } from 'react';
import { Progress } from '@/components/ui/progress';
import { Card, CardContent, CardHeader, CardTitle } from '@/components/ui/card';
import { useServerStore } from '@/stores/serverStore';
import { Progress } from '@/components/ui/progress';
import type { ModelProgress as ModelProgressType } from '@/lib/api/types';
import { Loader2, CheckCircle2, XCircle } from 'lucide-react';
import { useServerStore } from '@/stores/serverStore';
interface ModelProgressProps {
modelName: string;
@@ -63,13 +63,11 @@ export function ModelProgress({ modelName, displayName }: ModelProgressProps) {
const k = 1024;
const sizes = ['B', 'KB', 'MB', 'GB'];
const i = Math.floor(Math.log(bytes) / Math.log(k));
return `${(bytes / Math.pow(k, i)).toFixed(1)} ${sizes[i]}`;
return `${(bytes / k ** i).toFixed(1)} ${sizes[i]}`;
};
const getStatusIcon = () => {
switch (progress.status) {
case 'complete':
return <CheckCircle2 className="h-4 w-4 text-green-500" />;
case 'error':
return <XCircle className="h-4 w-4 text-destructive" />;
case 'downloading':
@@ -1,4 +1,4 @@
import { CheckCircle2, Loader2, XCircle } from 'lucide-react';
import { Loader2, XCircle } from 'lucide-react';
import { Badge } from '@/components/ui/badge';
import { Card, CardContent, CardHeader, CardTitle } from '@/components/ui/card';
import { useServerHealth } from '@/lib/hooks/useServer';
@@ -43,7 +43,6 @@ export function ServerStatus() {
) : health ? (
<div className="space-y-2">
<div className="flex items-center gap-2">
<CheckCircle2 className="h-4 w-4 text-green-500" />
<span className="text-sm">Connected</span>
</div>
<div className="flex flex-wrap gap-2">
@@ -1,14 +1,14 @@
import { useState, useEffect } from 'react';
import { RefreshCw, Download, CheckCircle2, AlertCircle } from 'lucide-react';
import { getVersion } from '@tauri-apps/api/app';
import { AlertCircle, Download, RefreshCw } from 'lucide-react';
import { useEffect, useState } from 'react';
import { Badge } from '@/components/ui/badge';
import { Button } from '@/components/ui/button';
import { Card, CardContent, CardHeader, CardTitle } from '@/components/ui/card';
import { Badge } from '@/components/ui/badge';
import { Progress } from '@/components/ui/progress';
import { useAutoUpdater } from '@/hooks/useAutoUpdater';
import { getVersion } from '@tauri-apps/api/app';
export function UpdateStatus() {
const { status, checkForUpdates, downloadAndInstall } = useAutoUpdater(false);
const { status, checkForUpdates, downloadAndInstall, restartAndInstall } = useAutoUpdater(false);
const [currentVersion, setCurrentVersion] = useState<string>('');
useEffect(() => {
@@ -30,7 +30,7 @@ export function UpdateStatus() {
</div>
<Button
onClick={checkForUpdates}
disabled={status.checking || status.downloading || status.installing}
disabled={status.checking || status.downloading || status.readyToInstall}
variant="outline"
size="sm"
>
@@ -53,7 +53,7 @@ export function UpdateStatus() {
</div>
)}
{status.available && !status.downloading && !status.installing && (
{status.available && !status.downloading && !status.readyToInstall && (
<div className="space-y-3 p-4 border rounded-lg bg-primary/5">
<div className="flex items-center justify-between">
<div>
@@ -64,7 +64,7 @@ export function UpdateStatus() {
</div>
<Button onClick={downloadAndInstall} className="w-full" size="sm">
<Download className="h-4 w-4 mr-2" />
Install Update
Download Update
</Button>
</div>
)}
@@ -77,33 +77,44 @@ export function UpdateStatus() {
Downloading update...
</div>
{status.downloadProgress !== undefined && (
<span className="text-muted-foreground">
{status.downloadProgress}%
</span>
<span className="text-muted-foreground">{status.downloadProgress}%</span>
)}
</div>
<Progress value={status.downloadProgress} />
{status.downloadedBytes !== undefined && status.totalBytes !== undefined && status.totalBytes > 0 && (
<div className="text-xs text-muted-foreground">
{(status.downloadedBytes / 1024 / 1024).toFixed(1)} MB / {(status.totalBytes / 1024 / 1024).toFixed(1)} MB
</div>
)}
{status.downloadedBytes !== undefined &&
status.totalBytes !== undefined &&
status.totalBytes > 0 && (
<div className="text-xs text-muted-foreground">
{(status.downloadedBytes / 1024 / 1024).toFixed(1)} MB /{' '}
{(status.totalBytes / 1024 / 1024).toFixed(1)} MB
</div>
)}
</div>
)}
{status.installing && (
<div className="space-y-2">
<div className="flex items-center gap-2 text-sm">
<RefreshCw className="h-4 w-4 animate-spin" />
Installing update...
{status.readyToInstall && (
<div className="space-y-3 p-4 border rounded-lg bg-accent/30 border-accent/50">
<div className="flex items-center gap-2">
<div>
<div className="font-semibold">Update Ready to Install</div>
<div className="text-sm text-muted-foreground">
Version {status.version} has been downloaded
</div>
</div>
</div>
<div className="text-xs text-muted-foreground">App will restart automatically</div>
<div className="text-sm text-muted-foreground">
The app needs to restart to complete the installation. You can do this now or later at
your convenience.
</div>
<Button onClick={restartAndInstall} className="w-full" size="sm">
<RefreshCw className="h-4 w-4 mr-2" />
Restart Now
</Button>
</div>
)}
{!status.available && !status.checking && !status.error && status.checking === false && (
<div className="flex items-center gap-2 text-sm text-muted-foreground">
<CheckCircle2 className="h-4 w-4 text-green-500" />
You're up to date
</div>
)}
@@ -0,0 +1,27 @@
import { ConnectionForm } from '@/components/ServerSettings/ConnectionForm';
import { ServerStatus } from '@/components/ServerSettings/ServerStatus';
import { UpdateStatus } from '@/components/ServerSettings/UpdateStatus';
import { isTauri } from '@/lib/tauri';
export function ServerTab() {
return (
<div className="space-y-4 overflow-y-auto flex flex-col">
<div className="grid gap-4 md:grid-cols-2">
<ConnectionForm />
<ServerStatus />
</div>
{isTauri() && <UpdateStatus />}
<div className="py-8 text-center text-sm text-muted-foreground">
Created by{' '}
<a
href="https://github.com/jamiepine"
target="_blank"
rel="noopener noreferrer"
className="text-accent hover:underline"
>
Jamie Pine
</a>
</div>
</div>
);
}
+18 -11
View File
@@ -1,24 +1,28 @@
import { Loader2, Settings, Volume2 } from 'lucide-react';
import { Link, useMatchRoute } from '@tanstack/react-router';
import { Box, BookOpen, Loader2, Mic, Server, Speaker, Volume2 } from 'lucide-react';
import voiceboxLogo from '@/assets/voicebox-logo.png';
import { cn } from '@/lib/utils/cn';
import { useGenerationStore } from '@/stores/generationStore';
import { usePlayerStore } from '@/stores/playerStore';
interface SidebarProps {
activeTab: string;
onTabChange: (tab: string) => void;
isMacOS?: boolean;
}
const tabs = [
{ id: 'main', icon: Volume2, label: 'Main' },
{ id: 'settings', icon: Settings, label: 'Settings' },
{ id: 'main', path: '/', icon: Volume2, label: 'Generate' },
{ id: 'stories', path: '/stories', icon: BookOpen, label: 'Stories' },
{ id: 'voices', path: '/voices', icon: Mic, label: 'Voices' },
{ id: 'audio', path: '/audio', icon: Speaker, label: 'Audio' },
{ id: 'models', path: '/models', icon: Box, label: 'Models' },
{ id: 'server', path: '/server', icon: Server, label: 'Server' },
];
export function Sidebar({ activeTab, onTabChange, isMacOS }: SidebarProps) {
export function Sidebar({ isMacOS }: SidebarProps) {
const isGenerating = useGenerationStore((state) => state.isGenerating);
const audioUrl = usePlayerStore((state) => state.audioUrl);
const isPlayerVisible = !!audioUrl;
const matchRoute = useMatchRoute();
return (
<div
@@ -36,13 +40,16 @@ export function Sidebar({ activeTab, onTabChange, isMacOS }: SidebarProps) {
<div className="flex flex-col gap-3">
{tabs.map((tab) => {
const Icon = tab.icon;
const isActive = activeTab === tab.id;
// For index route, use exact match; for others, use default matching
const isActive =
tab.path === '/'
? matchRoute({ to: '/', exact: true })
: matchRoute({ to: tab.path });
return (
<button
<Link
key={tab.id}
type="button"
onClick={() => onTabChange(tab.id)}
to={tab.path}
className={cn(
'w-12 h-12 rounded-full flex items-center justify-center transition-all duration-200',
'hover:bg-muted/50',
@@ -52,7 +59,7 @@ export function Sidebar({ activeTab, onTabChange, isMacOS }: SidebarProps) {
aria-label={tab.label}
>
<Icon className="h-5 w-5" />
</button>
</Link>
);
})}
</div>
@@ -0,0 +1,25 @@
import { FloatingGenerateBox } from '@/components/Generation/FloatingGenerateBox';
import { StoryContent } from './StoryContent';
import { StoryList } from './StoryList';
export function StoriesTab() {
return (
<div className="flex flex-col h-full min-h-0 overflow-hidden">
{/* Main content area */}
<div className="flex-1 min-h-0 flex gap-6 overflow-hidden relative">
{/* Left Column - Story List */}
<div className="flex flex-col min-h-0 overflow-hidden w-full max-w-[360px] shrink-0">
<StoryList />
</div>
{/* Right Column - Story Content */}
<div className="flex flex-col min-h-0 overflow-hidden flex-1">
<StoryContent />
</div>
{/* Floating Generate Box - position is managed via storyStore.trackEditorHeight */}
<FloatingGenerateBox showVoiceSelector />
</div>
</div>
);
}
@@ -0,0 +1,148 @@
import { useSortable } from '@dnd-kit/sortable';
import { CSS } from '@dnd-kit/utilities';
import { GripVertical, Mic, MoreHorizontal, Play, Trash2 } from 'lucide-react';
import { Button } from '@/components/ui/button';
import {
DropdownMenu,
DropdownMenuContent,
DropdownMenuItem,
DropdownMenuTrigger,
} from '@/components/ui/dropdown-menu';
import { Textarea } from '@/components/ui/textarea';
import type { StoryItemDetail } from '@/lib/api/types';
import { cn } from '@/lib/utils/cn';
import { useStoryStore } from '@/stores/storyStore';
interface StoryChatItemProps {
item: StoryItemDetail;
storyId: string;
index: number;
onRemove: () => void;
currentTimeMs: number;
isPlaying: boolean;
dragHandleProps?: React.HTMLAttributes<HTMLButtonElement>;
isDragging?: boolean;
}
export function StoryChatItem({
item,
onRemove,
currentTimeMs,
isPlaying,
dragHandleProps,
isDragging,
}: StoryChatItemProps) {
const seek = useStoryStore((state) => state.seek);
// Check if this item is currently playing based on timecode
const itemStartMs = item.start_time_ms;
const itemEndMs = item.start_time_ms + item.duration * 1000;
const isCurrentlyPlaying = isPlaying && currentTimeMs >= itemStartMs && currentTimeMs < itemEndMs;
const handlePlay = () => {
// Seek to the start of this item
seek(itemStartMs);
};
const formatTime = (ms: number): string => {
const totalSeconds = Math.floor(ms / 1000);
const minutes = Math.floor(totalSeconds / 60);
const seconds = totalSeconds % 60;
const milliseconds = Math.floor((ms % 1000) / 100);
return `${minutes}:${seconds.toString().padStart(2, '0')}.${milliseconds}`;
};
return (
<div
className={cn(
'flex items-start gap-3 p-4 rounded-lg border transition-colors',
isCurrentlyPlaying && 'bg-muted/70 border-primary',
!isCurrentlyPlaying && 'hover:bg-muted/50',
isDragging && 'opacity-50 shadow-lg',
)}
>
{/* Drag Handle */}
{dragHandleProps && (
<button
type="button"
className="shrink-0 cursor-grab active:cursor-grabbing touch-none text-muted-foreground hover:text-foreground transition-colors"
{...dragHandleProps}
>
<GripVertical className="h-5 w-5" />
</button>
)}
{/* Voice Icon */}
<div className="shrink-0">
<div className="h-10 w-10 rounded-full bg-muted flex items-center justify-center">
<Mic className="h-5 w-5 text-muted-foreground" />
</div>
</div>
{/* Content */}
<div className="flex-1 min-w-0">
<div className="flex items-center gap-2 mb-2">
<span className="font-medium text-sm">{item.profile_name}</span>
<span className="text-xs text-muted-foreground">{item.language}</span>
<span className="text-xs text-muted-foreground tabular-nums ml-auto">
{formatTime(itemStartMs)}
</span>
</div>
<Textarea
value={item.text}
className="flex-1 resize-none text-sm text-muted-foreground select-text bg-card cursor-text"
readOnly
onDoubleClick={handlePlay}
/>
</div>
{/* Actions */}
<div className="shrink-0">
<DropdownMenu>
<DropdownMenuTrigger asChild>
<Button variant="ghost" size="icon" className="h-8 w-8" aria-label="Actions">
<MoreHorizontal className="h-4 w-4" />
</Button>
</DropdownMenuTrigger>
<DropdownMenuContent align="end">
<DropdownMenuItem onClick={handlePlay}>
<Play className="mr-2 h-4 w-4" />
Play from here
</DropdownMenuItem>
<DropdownMenuItem onClick={onRemove} className="text-destructive focus:text-destructive">
<Trash2 className="mr-2 h-4 w-4" />
Remove from Story
</DropdownMenuItem>
</DropdownMenuContent>
</DropdownMenu>
</div>
</div>
);
}
// Sortable wrapper component
export function SortableStoryChatItem(props: Omit<StoryChatItemProps, 'dragHandleProps' | 'isDragging'>) {
const {
attributes,
listeners,
setNodeRef,
transform,
transition,
isDragging,
} = useSortable({ id: props.item.generation_id });
const style = {
transform: CSS.Transform.toString(transform),
transition,
};
return (
<div ref={setNodeRef} style={style} {...attributes}>
<StoryChatItem
{...props}
dragHandleProps={listeners}
isDragging={isDragging}
/>
</div>
);
}
@@ -0,0 +1,376 @@
import {
closestCenter,
DndContext,
type DragEndEvent,
KeyboardSensor,
PointerSensor,
useSensor,
useSensors,
} from '@dnd-kit/core';
import {
arrayMove,
SortableContext,
sortableKeyboardCoordinates,
verticalListSortingStrategy,
} from '@dnd-kit/sortable';
import { Download, Plus } from 'lucide-react';
import { useEffect, useMemo, useRef, useState } from 'react';
import { Button } from '@/components/ui/button';
import { Input } from '@/components/ui/input';
import { Popover, PopoverContent, PopoverTrigger } from '@/components/ui/popover';
import { useToast } from '@/components/ui/use-toast';
import { useHistory } from '@/lib/hooks/useHistory';
import {
useAddStoryItem,
useExportStoryAudio,
useRemoveStoryItem,
useReorderStoryItems,
useStory,
} from '@/lib/hooks/useStories';
import { useStoryPlayback } from '@/lib/hooks/useStoryPlayback';
import { useStoryStore } from '@/stores/storyStore';
import { SortableStoryChatItem } from './StoryChatItem';
export function StoryContent() {
const selectedStoryId = useStoryStore((state) => state.selectedStoryId);
const { data: story, isLoading } = useStory(selectedStoryId);
const removeItem = useRemoveStoryItem();
const reorderItems = useReorderStoryItems();
const exportAudio = useExportStoryAudio();
const addStoryItem = useAddStoryItem();
const { toast } = useToast();
const scrollRef = useRef<HTMLDivElement>(null);
// Add generation popover state
const [searchQuery, setSearchQuery] = useState('');
const [isAddOpen, setIsAddOpen] = useState(false);
const { data: historyData } = useHistory();
// Filter generations not in story and matching search
const availableGenerations = useMemo(() => {
if (!historyData?.items || !story) return [];
const storyGenerationIds = new Set(story.items.map((i) => i.generation_id));
const query = searchQuery.toLowerCase();
return historyData.items.filter(
(gen) =>
!storyGenerationIds.has(gen.id) &&
(gen.text.toLowerCase().includes(query) ||
gen.profile_name.toLowerCase().includes(query)),
);
}, [historyData, story, searchQuery]);
// Get track editor height from store for dynamic padding
const trackEditorHeight = useStoryStore((state) => state.trackEditorHeight);
// Track editor is shown when story has items
const hasBottomBar = story && story.items.length > 0;
// Calculate dynamic bottom padding: track editor + gap
const bottomPadding = hasBottomBar ? trackEditorHeight + 24 : 0;
// Drag and drop sensors
const sensors = useSensors(
useSensor(PointerSensor, {
activationConstraint: {
distance: 8,
},
}),
useSensor(KeyboardSensor, {
coordinateGetter: sortableKeyboardCoordinates,
}),
);
// Playback state (for auto-scroll and item highlighting)
const isPlaying = useStoryStore((state) => state.isPlaying);
const currentTimeMs = useStoryStore((state) => state.currentTimeMs);
const playbackStoryId = useStoryStore((state) => state.playbackStoryId);
// Refs for auto-scrolling to playing item
const itemRefsMap = useRef<Map<string, HTMLDivElement>>(new Map());
const lastScrolledItemRef = useRef<string | null>(null);
// Use playback hook
useStoryPlayback(story?.items);
// Sort items by start_time_ms
const sortedItems = useMemo(() => {
if (!story?.items) return [];
return [...story.items].sort((a, b) => a.start_time_ms - b.start_time_ms);
}, [story?.items]);
// Find the currently playing item based on timecode
const currentlyPlayingItemId = useMemo(() => {
if (!isPlaying || playbackStoryId !== story?.id || !sortedItems.length) {
return null;
}
const playingItem = sortedItems.find((item) => {
const itemStart = item.start_time_ms;
const itemEnd = item.start_time_ms + item.duration * 1000;
return currentTimeMs >= itemStart && currentTimeMs < itemEnd;
});
return playingItem?.generation_id ?? null;
}, [isPlaying, playbackStoryId, story?.id, sortedItems, currentTimeMs]);
// Auto-scroll to the currently playing item
useEffect(() => {
if (!currentlyPlayingItemId || currentlyPlayingItemId === lastScrolledItemRef.current) {
return;
}
const element = itemRefsMap.current.get(currentlyPlayingItemId);
if (element && scrollRef.current) {
element.scrollIntoView({ behavior: 'smooth', block: 'start' });
lastScrolledItemRef.current = currentlyPlayingItemId;
}
}, [currentlyPlayingItemId]);
// Reset last scrolled item when playback stops
useEffect(() => {
if (!isPlaying) {
lastScrolledItemRef.current = null;
}
}, [isPlaying]);
const handleRemoveItem = (itemId: string) => {
if (!story) return;
removeItem.mutate(
{
storyId: story.id,
itemId,
},
{
onError: (error) => {
toast({
title: 'Failed to remove item',
description: error.message,
variant: 'destructive',
});
},
},
);
};
const handleDragEnd = (event: DragEndEvent) => {
const { active, over } = event;
if (!story || !over || active.id === over.id) return;
const oldIndex = sortedItems.findIndex((item) => item.generation_id === active.id);
const newIndex = sortedItems.findIndex((item) => item.generation_id === over.id);
if (oldIndex === -1 || newIndex === -1) return;
// Calculate the new order
const newOrder = arrayMove(sortedItems, oldIndex, newIndex);
const generationIds = newOrder.map((item) => item.generation_id);
// Send reorder request to backend
reorderItems.mutate(
{
storyId: story.id,
data: { generation_ids: generationIds },
},
{
onError: (error) => {
toast({
title: 'Failed to reorder items',
description: error.message,
variant: 'destructive',
});
},
},
);
};
const handleExportAudio = () => {
if (!story) return;
exportAudio.mutate(
{
storyId: story.id,
storyName: story.name,
},
{
onError: (error) => {
toast({
title: 'Failed to export audio',
description: error.message,
variant: 'destructive',
});
},
},
);
};
const handleAddGeneration = (generationId: string) => {
if (!story) return;
addStoryItem.mutate(
{
storyId: story.id,
data: { generation_id: generationId },
},
{
onSuccess: () => {
setIsAddOpen(false);
setSearchQuery('');
},
onError: (error) => {
toast({
title: 'Failed to add generation',
description: error.message,
variant: 'destructive',
});
},
},
);
};
if (!selectedStoryId) {
return (
<div className="flex items-center justify-center h-full text-muted-foreground">
<div className="text-center">
<p className="text-lg font-medium mb-2">Select a story</p>
<p className="text-sm">Choose a story from the list to view its content</p>
</div>
</div>
);
}
if (isLoading) {
return (
<div className="flex items-center justify-center h-full">
<div className="text-muted-foreground">Loading story...</div>
</div>
);
}
if (!story) {
return (
<div className="flex items-center justify-center h-full text-muted-foreground">
<div className="text-center">
<p className="text-lg font-medium mb-2">Story not found</p>
<p className="text-sm">The selected story could not be loaded</p>
</div>
</div>
);
}
return (
<div className="flex flex-col h-full min-h-0">
{/* Header */}
<div className="flex items-center justify-between mb-4 px-1">
<div>
<h2 className="text-2xl font-bold">{story.name}</h2>
{story.description && (
<p className="text-sm text-muted-foreground mt-1">{story.description}</p>
)}
</div>
<div className="flex gap-2">
<Popover open={isAddOpen} onOpenChange={setIsAddOpen}>
<PopoverTrigger asChild>
<Button variant="outline" size="sm">
<Plus className="mr-2 h-4 w-4" />
Add
</Button>
</PopoverTrigger>
<PopoverContent className="w-80 p-0" align="end">
<div className="p-2 border-b">
<Input
placeholder="Search by name or transcript..."
value={searchQuery}
onChange={(e) => setSearchQuery(e.target.value)}
autoFocus
/>
</div>
<div className="max-h-60 overflow-y-auto">
{availableGenerations.length === 0 ? (
<div className="p-4 text-center text-sm text-muted-foreground">
{searchQuery
? 'No matching generations found'
: 'No available generations'}
</div>
) : (
availableGenerations.map((gen) => (
<button
key={gen.id}
type="button"
className="w-full text-left px-3 py-2 hover:bg-muted transition-colors border-b last:border-b-0"
onClick={() => handleAddGeneration(gen.id)}
>
<div className="font-medium text-sm">{gen.profile_name}</div>
<div className="text-xs text-muted-foreground truncate">
{gen.text.length > 50 ? `${gen.text.substring(0, 50)}...` : gen.text}
</div>
</button>
))
)}
</div>
</PopoverContent>
</Popover>
{story.items.length > 0 && (
<Button
variant="outline"
size="sm"
onClick={handleExportAudio}
disabled={exportAudio.isPending}
>
<Download className="mr-2 h-4 w-4" />
Export Audio
</Button>
)}
</div>
</div>
{/* Content */}
<div
ref={scrollRef}
className="flex-1 min-h-0 overflow-y-auto space-y-3"
style={{ paddingBottom: bottomPadding > 0 ? `${bottomPadding}px` : undefined }}
>
{sortedItems.length === 0 ? (
<div className="text-center py-12 px-5 border-2 border-dashed border-muted rounded-md text-muted-foreground">
<p className="text-sm">No items in this story</p>
<p className="text-xs mt-2">Generate speech using the box below to add items</p>
</div>
) : (
<DndContext
sensors={sensors}
collisionDetection={closestCenter}
onDragEnd={handleDragEnd}
>
<SortableContext
items={sortedItems.map((item) => item.generation_id)}
strategy={verticalListSortingStrategy}
>
<div className="space-y-3">
{sortedItems.map((item, index) => (
<div
key={item.id}
ref={(el) => {
if (el) {
itemRefsMap.current.set(item.generation_id, el);
} else {
itemRefsMap.current.delete(item.generation_id);
}
}}
>
<SortableStoryChatItem
item={item}
storyId={story.id}
index={index}
onRemove={() => handleRemoveItem(item.id)}
currentTimeMs={currentTimeMs}
isPlaying={isPlaying && playbackStoryId === story.id}
/>
</div>
))}
</div>
</SortableContext>
</DndContext>
)}
</div>
</div>
);
}
+369
View File
@@ -0,0 +1,369 @@
import { Plus, BookOpen, MoreHorizontal, Pencil, Trash2 } from 'lucide-react';
import { useState } from 'react';
import { Button } from '@/components/ui/button';
import {
Dialog,
DialogContent,
DialogDescription,
DialogFooter,
DialogHeader,
DialogTitle,
} from '@/components/ui/dialog';
import {
AlertDialog,
AlertDialogAction,
AlertDialogCancel,
AlertDialogContent,
AlertDialogDescription,
AlertDialogFooter,
AlertDialogHeader,
AlertDialogTitle,
} from '@/components/ui/alert-dialog';
import {
DropdownMenu,
DropdownMenuContent,
DropdownMenuItem,
DropdownMenuTrigger,
} from '@/components/ui/dropdown-menu';
import { Input } from '@/components/ui/input';
import { Textarea } from '@/components/ui/textarea';
import { Label } from '@/components/ui/label';
import { useToast } from '@/components/ui/use-toast';
import {
useStories,
useCreateStory,
useUpdateStory,
useDeleteStory,
} from '@/lib/hooks/useStories';
import { useStoryStore } from '@/stores/storyStore';
import { cn } from '@/lib/utils/cn';
import { formatDate } from '@/lib/utils/format';
export function StoryList() {
const { data: stories, isLoading } = useStories();
const selectedStoryId = useStoryStore((state) => state.selectedStoryId);
const setSelectedStoryId = useStoryStore((state) => state.setSelectedStoryId);
const createStory = useCreateStory();
const updateStory = useUpdateStory();
const deleteStory = useDeleteStory();
const [createDialogOpen, setCreateDialogOpen] = useState(false);
const [editDialogOpen, setEditDialogOpen] = useState(false);
const [deleteDialogOpen, setDeleteDialogOpen] = useState(false);
const [editingStory, setEditingStory] = useState<{ id: string; name: string; description?: string } | null>(null);
const [deletingStoryId, setDeletingStoryId] = useState<string | null>(null);
const [newStoryName, setNewStoryName] = useState('');
const [newStoryDescription, setNewStoryDescription] = useState('');
const { toast } = useToast();
const handleCreateStory = () => {
if (!newStoryName.trim()) {
toast({
title: 'Name required',
description: 'Please enter a story name',
variant: 'destructive',
});
return;
}
createStory.mutate(
{
name: newStoryName.trim(),
description: newStoryDescription.trim() || undefined,
},
{
onSuccess: (story) => {
setSelectedStoryId(story.id);
setCreateDialogOpen(false);
setNewStoryName('');
setNewStoryDescription('');
toast({
title: 'Story created',
description: `"${story.name}" has been created`,
});
},
onError: (error) => {
toast({
title: 'Failed to create story',
description: error.message,
variant: 'destructive',
});
},
},
);
};
const handleEditClick = (story: { id: string; name: string; description?: string }) => {
setEditingStory(story);
setNewStoryName(story.name);
setNewStoryDescription(story.description || '');
setEditDialogOpen(true);
};
const handleUpdateStory = () => {
if (!editingStory || !newStoryName.trim()) {
toast({
title: 'Name required',
description: 'Please enter a story name',
variant: 'destructive',
});
return;
}
updateStory.mutate(
{
storyId: editingStory.id,
data: {
name: newStoryName.trim(),
description: newStoryDescription.trim() || undefined,
},
},
{
onSuccess: () => {
setEditDialogOpen(false);
setEditingStory(null);
setNewStoryName('');
setNewStoryDescription('');
},
onError: (error) => {
toast({
title: 'Failed to update story',
description: error.message,
variant: 'destructive',
});
},
},
);
};
const handleDeleteClick = (storyId: string) => {
setDeletingStoryId(storyId);
setDeleteDialogOpen(true);
};
const handleDeleteConfirm = () => {
if (!deletingStoryId) return;
deleteStory.mutate(deletingStoryId, {
onSuccess: () => {
// Clear selection if deleting the currently selected story
if (selectedStoryId === deletingStoryId) {
setSelectedStoryId(null);
}
setDeleteDialogOpen(false);
setDeletingStoryId(null);
},
onError: (error) => {
toast({
title: 'Failed to delete story',
description: error.message,
variant: 'destructive',
});
},
});
};
if (isLoading) {
return (
<div className="flex items-center justify-center h-full">
<div className="text-muted-foreground">Loading stories...</div>
</div>
);
}
const storyList = stories || [];
return (
<div className="flex flex-col h-full min-h-0">
{/* Header */}
<div className="flex items-center justify-between mb-4 px-1">
<h2 className="text-2xl font-bold">Stories</h2>
<Button onClick={() => setCreateDialogOpen(true)} size="sm">
<Plus className="mr-2 h-4 w-4" />
New Story
</Button>
</div>
{/* Story List */}
<div className="flex-1 min-h-0 overflow-y-auto space-y-2">
{storyList.length === 0 ? (
<div className="text-center py-12 px-5 border-2 border-dashed border-muted rounded-md text-muted-foreground">
<BookOpen className="h-12 w-12 mx-auto mb-4 opacity-50" />
<p className="text-sm">No stories yet</p>
<p className="text-xs mt-2">Create your first story to get started</p>
</div>
) : (
storyList.map((story) => (
<div
key={story.id}
className={cn(
'h-24 p-4 border rounded-md transition-colors group flex items-center',
selectedStoryId === story.id && 'bg-muted border-primary',
)}
>
<div className="flex items-start justify-between gap-2 w-full min-w-0">
<button
type="button"
className="flex-1 min-w-0 text-left cursor-pointer overflow-hidden"
onClick={() => setSelectedStoryId(story.id)}
>
<h3 className="font-medium truncate">{story.name}</h3>
{story.description && (
<p className="text-sm text-muted-foreground mt-1 truncate">
{story.description}
</p>
)}
<div className="flex items-center gap-3 mt-2 text-xs text-muted-foreground">
<span>{story.item_count} {story.item_count === 1 ? 'item' : 'items'}</span>
<span>•</span>
<span>{formatDate(story.updated_at)}</span>
</div>
</button>
<DropdownMenu>
<DropdownMenuTrigger asChild>
<Button
variant="ghost"
size="icon"
className="h-8 w-8 opacity-0 group-hover:opacity-100 transition-opacity"
onClick={(e) => e.stopPropagation()}
>
<MoreHorizontal className="h-4 w-4" />
</Button>
</DropdownMenuTrigger>
<DropdownMenuContent align="end">
<DropdownMenuItem onClick={() => handleEditClick(story)}>
<Pencil className="mr-2 h-4 w-4" />
Edit
</DropdownMenuItem>
<DropdownMenuItem
onClick={() => handleDeleteClick(story.id)}
className="text-destructive focus:text-destructive"
>
<Trash2 className="mr-2 h-4 w-4" />
Delete
</DropdownMenuItem>
</DropdownMenuContent>
</DropdownMenu>
</div>
</div>
))
)}
</div>
{/* Create Story Dialog */}
<Dialog open={createDialogOpen} onOpenChange={setCreateDialogOpen}>
<DialogContent>
<DialogHeader>
<DialogTitle>Create New Story</DialogTitle>
<DialogDescription>
Create a new story to organize your voice generations into conversations.
</DialogDescription>
</DialogHeader>
<div className="space-y-4 py-4">
<div className="space-y-2">
<Label htmlFor="story-name">Name</Label>
<Input
id="story-name"
placeholder="My Story"
value={newStoryName}
onChange={(e) => setNewStoryName(e.target.value)}
onKeyDown={(e) => {
if (e.key === 'Enter') {
handleCreateStory();
}
}}
/>
</div>
<div className="space-y-2">
<Label htmlFor="story-description">Description (optional)</Label>
<Textarea
id="story-description"
placeholder="A conversation between..."
value={newStoryDescription}
onChange={(e) => setNewStoryDescription(e.target.value)}
rows={3}
/>
</div>
</div>
<DialogFooter>
<Button variant="outline" onClick={() => setCreateDialogOpen(false)}>
Cancel
</Button>
<Button onClick={handleCreateStory} disabled={createStory.isPending}>
{createStory.isPending ? 'Creating...' : 'Create'}
</Button>
</DialogFooter>
</DialogContent>
</Dialog>
{/* Edit Story Dialog */}
<Dialog open={editDialogOpen} onOpenChange={setEditDialogOpen}>
<DialogContent>
<DialogHeader>
<DialogTitle>Edit Story</DialogTitle>
<DialogDescription>
Update the story name and description.
</DialogDescription>
</DialogHeader>
<div className="space-y-4 py-4">
<div className="space-y-2">
<Label htmlFor="edit-story-name">Name</Label>
<Input
id="edit-story-name"
placeholder="My Story"
value={newStoryName}
onChange={(e) => setNewStoryName(e.target.value)}
onKeyDown={(e) => {
if (e.key === 'Enter') {
handleUpdateStory();
}
}}
/>
</div>
<div className="space-y-2">
<Label htmlFor="edit-story-description">Description (optional)</Label>
<Textarea
id="edit-story-description"
placeholder="A conversation between..."
value={newStoryDescription}
onChange={(e) => setNewStoryDescription(e.target.value)}
rows={3}
/>
</div>
</div>
<DialogFooter>
<Button variant="outline" onClick={() => setEditDialogOpen(false)}>
Cancel
</Button>
<Button onClick={handleUpdateStory} disabled={updateStory.isPending}>
{updateStory.isPending ? 'Saving...' : 'Save'}
</Button>
</DialogFooter>
</DialogContent>
</Dialog>
{/* Delete Story Confirmation Dialog */}
<AlertDialog open={deleteDialogOpen} onOpenChange={setDeleteDialogOpen}>
<AlertDialogContent>
<AlertDialogHeader>
<AlertDialogTitle>Are you sure?</AlertDialogTitle>
<AlertDialogDescription>
This will permanently delete the story and all its items. This action cannot be undone.
</AlertDialogDescription>
</AlertDialogHeader>
<AlertDialogFooter>
<AlertDialogCancel>Cancel</AlertDialogCancel>
<AlertDialogAction asChild>
<Button
onClick={handleDeleteConfirm}
disabled={deleteStory.isPending}
className="bg-destructive text-destructive-foreground hover:bg-destructive/90"
>
{deleteStory.isPending ? 'Deleting...' : 'Delete'}
</Button>
</AlertDialogAction>
</AlertDialogFooter>
</AlertDialogContent>
</AlertDialog>
</div>
);
}
@@ -0,0 +1,953 @@
import {
Copy,
GripHorizontal,
Minus,
Pause,
Play,
Plus,
Scissors,
Square,
Trash2,
} from 'lucide-react';
import { useCallback, useEffect, useMemo, useRef, useState } from 'react';
import WaveSurfer from 'wavesurfer.js';
import { Button } from '@/components/ui/button';
import { useToast } from '@/components/ui/use-toast';
import { apiClient } from '@/lib/api/client';
import type { StoryItemDetail } from '@/lib/api/types';
import {
useDuplicateStoryItem,
useMoveStoryItem,
useRemoveStoryItem,
useSplitStoryItem,
useTrimStoryItem,
} from '@/lib/hooks/useStories';
import { cn } from '@/lib/utils/cn';
import { useStoryStore } from '@/stores/storyStore';
// Clip waveform component with trim support
function ClipWaveform({
generationId,
width,
trimStartMs,
trimEndMs,
duration,
}: {
generationId: string;
width: number;
trimStartMs: number;
trimEndMs: number;
duration: number;
}) {
const waveformRef = useRef<HTMLDivElement>(null);
const wavesurferRef = useRef<WaveSurfer | null>(null);
// Calculate the full waveform width based on the original duration
// The visible portion (width) represents the effective duration after trimming
const effectiveDurationMs = duration * 1000 - trimStartMs - trimEndMs;
const fullWaveformWidth =
effectiveDurationMs > 0 ? (width / effectiveDurationMs) * (duration * 1000) : width;
// Calculate how much to offset the waveform to hide the trimmed start
const offsetX =
effectiveDurationMs > 0 ? (trimStartMs / (duration * 1000)) * fullWaveformWidth : 0;
useEffect(() => {
if (!waveformRef.current || fullWaveformWidth < 20) return;
// Get CSS colors
const root = document.documentElement;
const getCSSVar = (varName: string) => {
const value = getComputedStyle(root).getPropertyValue(varName).trim();
return value ? `hsl(${value})` : '';
};
const waveColor = getCSSVar('--accent-foreground');
const wavesurfer = WaveSurfer.create({
container: waveformRef.current,
waveColor,
progressColor: waveColor,
cursorWidth: 0,
barWidth: 1,
barRadius: 1,
barGap: 1,
height: 28,
normalize: true,
interact: false,
});
wavesurferRef.current = wavesurfer;
const audioUrl = apiClient.getAudioUrl(generationId);
wavesurfer.load(audioUrl).catch(() => {
// Ignore load errors
});
return () => {
wavesurfer.destroy();
wavesurferRef.current = null;
};
}, [generationId, fullWaveformWidth]);
return (
<div className="w-full h-full opacity-60 overflow-hidden">
{/* Inner container that holds the full waveform, offset to show only visible portion */}
<div
ref={waveformRef}
style={{
width: `${fullWaveformWidth}px`,
transform: `translateX(-${offsetX}px)`,
}}
className="h-full"
/>
</div>
);
}
interface StoryTrackEditorProps {
storyId: string;
items: StoryItemDetail[];
}
const TRACK_HEIGHT = 48;
const MIN_PIXELS_PER_SECOND = 10;
const MAX_PIXELS_PER_SECOND = 200;
const DEFAULT_PIXELS_PER_SECOND = 50;
const DEFAULT_TRACKS = [1, 0, -1]; // Default 3 tracks
const MIN_EDITOR_HEIGHT = 120;
const MAX_EDITOR_HEIGHT = 500;
export function StoryTrackEditor({ storyId, items }: StoryTrackEditorProps) {
const [pixelsPerSecond, setPixelsPerSecond] = useState(DEFAULT_PIXELS_PER_SECOND);
const [draggingItem, setDraggingItem] = useState<string | null>(null);
const [dragOffset, setDragOffset] = useState({ x: 0, y: 0 });
const [dragPosition, setDragPosition] = useState({ x: 0, y: 0 });
const [isResizing, setIsResizing] = useState(false);
const [containerWidth, setContainerWidth] = useState(0);
const containerRef = useRef<HTMLDivElement>(null);
const tracksRef = useRef<HTMLDivElement>(null);
const resizeStartY = useRef(0);
const resizeStartHeight = useRef(0);
const moveItem = useMoveStoryItem();
const trimItem = useTrimStoryItem();
const splitItem = useSplitStoryItem();
const duplicateItem = useDuplicateStoryItem();
const removeItem = useRemoveStoryItem();
const { toast } = useToast();
// Selection state
const selectedClipId = useStoryStore((state) => state.selectedClipId);
const setSelectedClipId = useStoryStore((state) => state.setSelectedClipId);
// Trim state
const [trimmingItem, setTrimmingItem] = useState<string | null>(null);
const [trimSide, setTrimSide] = useState<'start' | 'end' | null>(null);
const [trimStartX, setTrimStartX] = useState(0);
const [tempTrimValues, setTempTrimValues] = useState<{
trim_start_ms: number;
trim_end_ms: number;
} | null>(null);
// Track editor height from store (shared with FloatingGenerateBox)
const editorHeight = useStoryStore((state) => state.trackEditorHeight);
const setEditorHeight = useStoryStore((state) => state.setTrackEditorHeight);
// Playback state
const isPlaying = useStoryStore((state) => state.isPlaying);
const currentTimeMs = useStoryStore((state) => state.currentTimeMs);
const playbackStoryId = useStoryStore((state) => state.playbackStoryId);
const play = useStoryStore((state) => state.play);
const pause = useStoryStore((state) => state.pause);
const stop = useStoryStore((state) => state.stop);
const seek = useStoryStore((state) => state.seek);
const setActiveStory = useStoryStore((state) => state.setActiveStory);
const isActiveStory = playbackStoryId === storyId;
const isCurrentlyPlaying = isPlaying && isActiveStory;
// Auto-activate this story when the editor is shown so playhead is visible
useEffect(() => {
if (items.length > 0 && !isActiveStory) {
const totalDuration = Math.max(
...items.map((item) => {
const trimStart = item.trim_start_ms || 0;
const trimEnd = item.trim_end_ms || 0;
const effectiveDuration = item.duration * 1000 - trimStart - trimEnd;
return item.start_time_ms + effectiveDuration;
}),
0,
);
setActiveStory(storyId, items, totalDuration);
}
}, [storyId, items, isActiveStory, setActiveStory]);
// Sort items by start time for play
const sortedItems = useMemo(() => {
return [...items].sort((a, b) => a.start_time_ms - b.start_time_ms);
}, [items]);
const handlePlayPause = () => {
if (isCurrentlyPlaying) {
pause();
} else {
play(storyId, sortedItems);
}
};
const handleStop = () => {
stop();
};
// Calculate unique tracks from items, always showing at least 3 default tracks
const tracks = useMemo(() => {
const trackSet = new Set([...DEFAULT_TRACKS, ...items.map((item) => item.track)]);
return Array.from(trackSet).sort((a, b) => b - a); // Higher tracks on top
}, [items]);
// Track container width for full-width minimum
useEffect(() => {
const container = tracksRef.current;
if (!container) return;
const observer = new ResizeObserver((entries) => {
for (const entry of entries) {
setContainerWidth(entry.contentRect.width);
}
});
observer.observe(container);
// Set initial width
setContainerWidth(container.clientWidth);
return () => observer.disconnect();
}, []);
// Calculate effective duration (accounting for trims)
const getEffectiveDuration = (item: StoryItemDetail) => {
return item.duration * 1000 - (item.trim_start_ms || 0) - (item.trim_end_ms || 0);
};
// Calculate total duration (using effective durations)
const totalDurationMs = useMemo(() => {
if (items.length === 0) return 10000; // Default 10 seconds
return Math.max(...items.map((item) => item.start_time_ms + getEffectiveDuration(item)), 10000);
}, [items, getEffectiveDuration]);
// Calculate timeline width - at least full container width
const contentWidth = (totalDurationMs / 1000) * pixelsPerSecond + 200; // Content width with padding
const timelineWidth = Math.max(contentWidth, containerWidth);
// Generate time markers
const timeMarkers = useMemo(() => {
const markers: number[] = [];
// Determine interval based on zoom level
let intervalMs = 5000; // 5 seconds
if (pixelsPerSecond > 100) intervalMs = 1000;
else if (pixelsPerSecond > 50) intervalMs = 2000;
else if (pixelsPerSecond < 20) intervalMs = 10000;
for (let ms = 0; ms <= totalDurationMs + intervalMs; ms += intervalMs) {
markers.push(ms);
}
return markers;
}, [totalDurationMs, pixelsPerSecond]);
const formatTime = (ms: number): string => {
const totalSeconds = Math.floor(ms / 1000);
const minutes = Math.floor(totalSeconds / 60);
const seconds = totalSeconds % 60;
return `${minutes}:${seconds.toString().padStart(2, '0')}`;
};
const msToPixels = useCallback((ms: number) => (ms / 1000) * pixelsPerSecond, [pixelsPerSecond]);
const pixelsToMs = useCallback((px: number) => (px / pixelsPerSecond) * 1000, [pixelsPerSecond]);
const handleZoomIn = () => {
setPixelsPerSecond((prev) => Math.min(prev * 1.5, MAX_PIXELS_PER_SECOND));
};
const handleZoomOut = () => {
setPixelsPerSecond((prev) => Math.max(prev / 1.5, MIN_PIXELS_PER_SECOND));
};
// Resize handlers
const handleResizeStart = useCallback(
(e: React.MouseEvent) => {
e.preventDefault();
setIsResizing(true);
resizeStartY.current = e.clientY;
resizeStartHeight.current = editorHeight;
},
[editorHeight],
);
const handleResizeMove = useCallback(
(e: MouseEvent) => {
if (!isResizing) return;
const deltaY = resizeStartY.current - e.clientY;
const newHeight = Math.min(
MAX_EDITOR_HEIGHT,
Math.max(MIN_EDITOR_HEIGHT, resizeStartHeight.current + deltaY),
);
setEditorHeight(newHeight);
},
[isResizing, setEditorHeight],
);
const handleResizeEnd = useCallback(() => {
setIsResizing(false);
}, []);
// Add global mouse listeners for resizing
useEffect(() => {
if (isResizing) {
window.addEventListener('mousemove', handleResizeMove);
window.addEventListener('mouseup', handleResizeEnd);
return () => {
window.removeEventListener('mousemove', handleResizeMove);
window.removeEventListener('mouseup', handleResizeEnd);
};
}
}, [isResizing, handleResizeMove, handleResizeEnd]);
const handleTimelineClick = (e: React.MouseEvent<HTMLDivElement>) => {
if (!tracksRef.current || draggingItem || trimmingItem) return;
const rect = tracksRef.current.getBoundingClientRect();
const x = e.clientX - rect.left + tracksRef.current.scrollLeft;
const timeMs = Math.max(0, pixelsToMs(x));
seek(timeMs);
// Deselect clip when clicking on timeline
setSelectedClipId(null);
};
const handleClipClick = (e: React.MouseEvent, item: StoryItemDetail) => {
e.stopPropagation();
if (draggingItem || trimmingItem) return;
setSelectedClipId(item.id);
};
const handleTrimStart = (e: React.MouseEvent, item: StoryItemDetail, side: 'start' | 'end') => {
e.stopPropagation();
if (!tracksRef.current) return;
setTrimmingItem(item.id);
setTrimSide(side);
setSelectedClipId(item.id);
setTrimStartX(e.clientX);
trimStartItemRef.current = {
item,
initialTrimStart: item.trim_start_ms || 0,
initialTrimEnd: item.trim_end_ms || 0,
};
};
const trimStartItemRef = useRef<{
item: StoryItemDetail;
initialTrimStart: number;
initialTrimEnd: number;
} | null>(null);
const handleTrimMove = useCallback(
(e: MouseEvent) => {
if (!trimmingItem || !trimSide || !trimStartItemRef.current) return;
const deltaX = e.clientX - trimStartX;
const deltaMs = pixelsToMs(deltaX); // Signed delta in milliseconds
const { item, initialTrimStart, initialTrimEnd } = trimStartItemRef.current;
const originalDurationMs = item.duration * 1000;
let newTrimStart = initialTrimStart;
let newTrimEnd = initialTrimEnd;
if (trimSide === 'start') {
// Moving right increases trim_start (trims more from start)
// Moving left decreases trim_start (restores from start)
newTrimStart = Math.round(
Math.max(
0,
Math.min(initialTrimStart + deltaMs, originalDurationMs - initialTrimEnd - 100),
),
);
} else {
// Moving right decreases trim_end (restores from end)
// Moving left increases trim_end (trims more from end)
newTrimEnd = Math.round(
Math.max(
0,
Math.min(initialTrimEnd - deltaMs, originalDurationMs - initialTrimStart - 100),
),
);
}
// Validate that we don't exceed duration
if (newTrimStart + newTrimEnd >= originalDurationMs - 100) {
return; // Don't allow trimming to less than 100ms
}
// Update temporary trim values for visual feedback
setTempTrimValues({
trim_start_ms: newTrimStart,
trim_end_ms: newTrimEnd,
});
},
[trimmingItem, trimSide, trimStartX, pixelsToMs],
);
const handleTrimEnd = useCallback(() => {
if (!trimmingItem || !trimSide || !trimStartItemRef.current) {
setTrimmingItem(null);
setTrimSide(null);
setTempTrimValues(null);
trimStartItemRef.current = null;
return;
}
const { initialTrimStart, initialTrimEnd } = trimStartItemRef.current;
// Use temporary trim values if available, otherwise use initial values
// Ensure values are integers for the backend
const finalTrimStart = Math.round(tempTrimValues?.trim_start_ms ?? initialTrimStart);
const finalTrimEnd = Math.round(tempTrimValues?.trim_end_ms ?? initialTrimEnd);
// Only update if values changed
if (finalTrimStart !== initialTrimStart || finalTrimEnd !== initialTrimEnd) {
trimItem.mutate(
{
storyId,
itemId: trimmingItem,
data: {
trim_start_ms: finalTrimStart,
trim_end_ms: finalTrimEnd,
},
},
{
onError: (error) => {
toast({
title: 'Failed to trim clip',
description: error instanceof Error ? error.message : String(error),
variant: 'destructive',
});
},
},
);
}
setTrimmingItem(null);
setTrimSide(null);
setTempTrimValues(null);
trimStartItemRef.current = null;
}, [trimmingItem, trimSide, tempTrimValues, storyId, trimItem, toast]);
const handleSplit = useCallback(() => {
if (!selectedClipId) return;
const item = items.find((i) => i.id === selectedClipId);
if (!item) return;
const splitTimeMs = currentTimeMs - item.start_time_ms;
const effectiveDuration = getEffectiveDuration(item);
if (splitTimeMs <= 0 || splitTimeMs >= effectiveDuration) {
toast({
title: 'Invalid split point',
description: 'Playhead must be within the selected clip',
variant: 'destructive',
});
return;
}
splitItem.mutate(
{
storyId,
itemId: selectedClipId,
data: { split_time_ms: splitTimeMs },
},
{
onSuccess: () => {
setSelectedClipId(null);
},
onError: (error) => {
toast({
title: 'Failed to split clip',
description: error instanceof Error ? error.message : String(error),
variant: 'destructive',
});
},
},
);
}, [
selectedClipId,
items,
currentTimeMs,
getEffectiveDuration,
storyId,
splitItem,
toast,
setSelectedClipId,
]);
const handleDuplicate = useCallback(() => {
if (!selectedClipId) return;
duplicateItem.mutate(
{
storyId,
itemId: selectedClipId,
},
{
onError: (error) => {
toast({
title: 'Failed to duplicate clip',
description: error instanceof Error ? error.message : String(error),
variant: 'destructive',
});
},
},
);
}, [selectedClipId, storyId, duplicateItem, toast]);
const handleDelete = useCallback(() => {
if (!selectedClipId) return;
removeItem.mutate(
{
storyId,
itemId: selectedClipId,
},
{
onSuccess: () => {
setSelectedClipId(null);
},
onError: (error) => {
toast({
title: 'Failed to delete clip',
description: error instanceof Error ? error.message : String(error),
variant: 'destructive',
});
},
},
);
}, [selectedClipId, storyId, removeItem, toast, setSelectedClipId]);
// Keyboard shortcuts
useEffect(() => {
const handleKeyDown = (e: KeyboardEvent) => {
// Only handle shortcuts when editor is focused or no input is focused
if (e.target instanceof HTMLInputElement || e.target instanceof HTMLTextAreaElement) {
return;
}
if (e.key === 'Escape') {
setSelectedClipId(null);
} else if (e.key === 's' || e.key === 'S') {
if (selectedClipId) {
e.preventDefault();
handleSplit();
}
} else if (e.key === 'd' || e.key === 'D') {
if (selectedClipId && (e.metaKey || e.ctrlKey)) {
e.preventDefault();
handleDuplicate();
}
} else if (e.key === 'Delete' || e.key === 'Backspace') {
if (selectedClipId) {
e.preventDefault();
handleDelete();
}
}
};
window.addEventListener('keydown', handleKeyDown);
return () => window.removeEventListener('keydown', handleKeyDown);
}, [selectedClipId, handleSplit, handleDuplicate, handleDelete, setSelectedClipId]);
// Add global mouse listeners for trimming
useEffect(() => {
if (trimmingItem) {
window.addEventListener('mousemove', handleTrimMove);
window.addEventListener('mouseup', handleTrimEnd);
return () => {
window.removeEventListener('mousemove', handleTrimMove);
window.removeEventListener('mouseup', handleTrimEnd);
};
}
}, [trimmingItem, handleTrimMove, handleTrimEnd]);
const handleDragStart = (e: React.MouseEvent, item: StoryItemDetail) => {
e.stopPropagation();
if (!tracksRef.current) return;
const rect = e.currentTarget.getBoundingClientRect();
setDragOffset({
x: e.clientX - rect.left,
y: e.clientY - rect.top,
});
setDragPosition({
x: rect.left - tracksRef.current.getBoundingClientRect().left + tracksRef.current.scrollLeft,
y: rect.top - tracksRef.current.getBoundingClientRect().top,
});
setDraggingItem(item.id);
};
const handleDragMove = useCallback(
(e: React.MouseEvent) => {
if (!draggingItem || !tracksRef.current) return;
const rect = tracksRef.current.getBoundingClientRect();
const x = e.clientX - rect.left + tracksRef.current.scrollLeft - dragOffset.x;
const y = e.clientY - rect.top - dragOffset.y;
setDragPosition({ x: Math.max(0, x), y });
},
[draggingItem, dragOffset],
);
const handleDragEnd = useCallback(() => {
if (!draggingItem || !tracksRef.current) {
setDraggingItem(null);
return;
}
const item = items.find((i) => i.id === draggingItem);
if (!item) {
setDraggingItem(null);
return;
}
// Calculate new time from x position
const newTimeMs = Math.max(0, Math.round(pixelsToMs(dragPosition.x)));
// Calculate new track from y position
const trackIndex = Math.floor(dragPosition.y / TRACK_HEIGHT);
const clampedTrackIndex = Math.max(0, Math.min(trackIndex, tracks.length - 1));
const newTrack = tracks[clampedTrackIndex] ?? 0;
// Check if position changed
if (newTimeMs !== item.start_time_ms || newTrack !== item.track) {
moveItem.mutate(
{
storyId,
itemId: item.id,
data: {
start_time_ms: newTimeMs,
track: newTrack,
},
},
{
onError: (error) => {
toast({
title: 'Failed to move item',
description: error instanceof Error ? error.message : String(error),
variant: 'destructive',
});
},
},
);
}
setDraggingItem(null);
}, [draggingItem, dragPosition, items, tracks, pixelsToMs, storyId, moveItem, toast]);
// Get track index for rendering
const getTrackIndex = (trackNumber: number) => tracks.indexOf(trackNumber);
// Calculate clip position and dimensions
const getClipStyle = (item: StoryItemDetail) => {
const isDragging = draggingItem === item.id;
const trackIndex = getTrackIndex(item.track);
const effectiveDuration = getEffectiveDuration(item);
const width = msToPixels(effectiveDuration);
const left = isDragging ? dragPosition.x : msToPixels(item.start_time_ms);
const top = isDragging ? dragPosition.y : trackIndex * TRACK_HEIGHT;
return {
width: `${width}px`,
left: `${left}px`,
top: `${top}px`,
height: `${TRACK_HEIGHT - 4}px`,
};
};
// Playhead position
const playheadLeft = msToPixels(currentTimeMs);
// Calculate tracks area height
const tracksAreaHeight = tracks.length * TRACK_HEIGHT;
const timelineContainerHeight = editorHeight - 40; // Subtract toolbar height
if (items.length === 0) {
return null;
}
return (
<div className="fixed bottom-0 left-0 right-0 border-t bg-background/95 backdrop-blur supports-backdrop-filter:bg-background/60 z-50">
<div
className="border-t bg-background/30 backdrop-blur-2xl overflow-hidden relative"
ref={containerRef}
>
{/* Resize handle at top */}
<button
type="button"
className="absolute top-0 left-0 right-0 h-2 cursor-ns-resize flex items-center justify-center hover:bg-muted/50 transition-colors z-20 group"
onMouseDown={handleResizeStart}
aria-label="Resize track editor"
>
<GripHorizontal className="h-3 w-3 text-muted-foreground/50 group-hover:text-muted-foreground" />
</button>
{/* Toolbar */}
<div className="flex items-center justify-between px-3 py-2 border-b bg-muted/30 mt-2">
{/* Play controls - left side */}
<div className="flex items-center gap-2">
<Button variant="ghost" size="icon" className="h-7 w-7" onClick={handlePlayPause}>
{isCurrentlyPlaying ? <Pause className="h-4 w-4" /> : <Play className="h-4 w-4" />}
</Button>
<Button
variant="ghost"
size="icon"
className="h-7 w-7"
onClick={handleStop}
disabled={!isCurrentlyPlaying}
>
<Square className="h-3 w-3" />
</Button>
<span className="text-xs text-muted-foreground tabular-nums ml-2">
{formatTime(currentTimeMs)} / {formatTime(totalDurationMs)}
</span>
</div>
{/* Clip editing controls - center */}
{selectedClipId && (
<div className="flex items-center gap-1">
<Button
variant="ghost"
size="icon"
className="h-7 w-7"
onClick={handleSplit}
title="Split at playhead (S)"
>
<Scissors className="h-4 w-4" />
</Button>
<Button
variant="ghost"
size="icon"
className="h-7 w-7"
onClick={handleDuplicate}
title="Duplicate (Cmd/Ctrl+D)"
>
<Copy className="h-4 w-4" />
</Button>
<Button
variant="ghost"
size="icon"
className="h-7 w-7"
onClick={handleDelete}
title="Delete (Delete/Backspace)"
>
<Trash2 className="h-4 w-4" />
</Button>
</div>
)}
{/* Zoom controls - right side */}
<div className="flex items-center gap-2">
<span className="text-xs text-muted-foreground">Zoom:</span>
<Button variant="ghost" size="icon" className="h-6 w-6" onClick={handleZoomOut}>
<Minus className="h-3 w-3" />
</Button>
<Button variant="ghost" size="icon" className="h-6 w-6" onClick={handleZoomIn}>
<Plus className="h-3 w-3" />
</Button>
</div>
</div>
{/* Timeline container with track labels sidebar */}
<div className="flex" style={{ height: `${timelineContainerHeight}px` }}>
{/* Track labels sidebar - fixed width */}
<div className="w-16 shrink-0 border-r bg-muted/20 overflow-hidden">
{/* Spacer for time ruler */}
<div className="h-6 border-b bg-muted/30" />
{/* Track labels */}
<div style={{ height: `${tracksAreaHeight}px` }}>
{tracks.map((trackNumber, index) => (
<div
key={trackNumber}
className={cn(
'border-b flex items-center justify-center',
index % 2 === 0 ? 'bg-background' : 'bg-muted/10',
)}
style={{ height: `${TRACK_HEIGHT}px` }}
>
<span className="text-[10px] text-muted-foreground select-none">
{trackNumber}
</span>
</div>
))}
</div>
</div>
{/* Scrollable timeline area */}
{/* biome-ignore lint/a11y/noStaticElementInteractions: Container handles drag events for child clips */}
<div
ref={tracksRef}
className="overflow-auto relative flex-1"
onMouseMove={draggingItem ? handleDragMove : undefined}
onMouseUp={draggingItem ? handleDragEnd : undefined}
onMouseLeave={draggingItem ? handleDragEnd : undefined}
>
{/* Time ruler - clickable to seek */}
<button
type="button"
className="h-6 border-b bg-muted/20 sticky top-0 z-10 cursor-pointer text-left"
style={{ width: `${timelineWidth}px` }}
onClick={handleTimelineClick}
aria-label="Seek timeline"
>
{timeMarkers.map((ms) => (
<div
key={ms}
className="absolute top-0 h-full flex flex-col justify-end pointer-events-none"
style={{ left: `${msToPixels(ms)}px` }}
>
<div className="h-2 w-px bg-border" />
<span className="text-[10px] text-muted-foreground ml-1 select-none">
{formatTime(ms)}
</span>
</div>
))}
</button>
{/* Tracks area */}
<div
className="relative"
style={{ width: `${timelineWidth}px`, height: `${tracksAreaHeight}px` }}
>
{/* Track backgrounds - pointer-events-none to allow clicks to pass through */}
{tracks.map((trackNumber, index) => (
<div
key={trackNumber}
className={cn(
'absolute left-0 right-0 border-b pointer-events-none',
index % 2 === 0 ? 'bg-background' : 'bg-muted/10',
)}
style={{
top: `${index * TRACK_HEIGHT}px`,
height: `${TRACK_HEIGHT}px`,
}}
/>
))}
{/* Click area for seeking - z-index lower than clips */}
<button
type="button"
className="absolute inset-0 z-0 cursor-pointer"
onClick={handleTimelineClick}
aria-label="Seek timeline"
/>
{/* Audio clips */}
{items.map((item) => {
const isDragging = draggingItem === item.id;
const isSelected = selectedClipId === item.id;
const isTrimming = trimmingItem === item.id;
// Use temporary trim values during trimming for visual feedback
const displayTrimStart =
isTrimming && tempTrimValues
? tempTrimValues.trim_start_ms
: item.trim_start_ms || 0;
const displayTrimEnd =
isTrimming && tempTrimValues ? tempTrimValues.trim_end_ms : item.trim_end_ms || 0;
const effectiveDuration = item.duration * 1000 - displayTrimStart - displayTrimEnd;
const style = getClipStyle({
...item,
trim_start_ms: displayTrimStart,
trim_end_ms: displayTrimEnd,
});
const clipWidth = msToPixels(effectiveDuration);
return (
<div
key={item.id}
className={cn(
'absolute rounded select-none overflow-visible z-10',
isSelected && 'ring-2 ring-primary ring-offset-1',
isTrimming && 'ring-2 ring-accent',
)}
style={style}
>
<button
type="button"
className={cn(
'w-full h-full rounded cursor-move overflow-hidden',
'bg-accent/80 hover:bg-accent border border-accent-foreground/20',
'flex flex-col justify-center',
isDragging && 'opacity-80 shadow-lg z-20',
!isDragging && 'transition-all duration-100',
)}
onClick={(e) => handleClipClick(e, item)}
onMouseDown={(e) => {
// Only start drag if not clicking on trim handles
if (!(e.target as HTMLElement).closest('.trim-handle')) {
handleDragStart(e, item);
}
}}
>
{/* Clip label */}
<div className="absolute top-0 left-1 right-1 z-10">
<p className="text-[9px] font-medium text-accent-foreground truncate">
{item.profile_name}
</p>
</div>
{/* Waveform */}
<div className="absolute inset-0 top-3">
<ClipWaveform
generationId={item.generation_id}
width={clipWidth}
trimStartMs={displayTrimStart}
trimEndMs={displayTrimEnd}
duration={item.duration}
/>
</div>
</button>
{/* Trim handles */}
{isSelected && (
<>
{/* Left trim handle */}
<button
type="button"
className="trim-handle absolute left-0 top-0 bottom-0 w-2 cursor-ew-resize hover:bg-primary/30 bg-primary/20 z-30 rounded-l"
onMouseDown={(e) => handleTrimStart(e, item, 'start')}
aria-label="Trim start"
/>
{/* Right trim handle */}
<button
type="button"
className="trim-handle absolute right-0 top-0 bottom-0 w-2 cursor-ew-resize hover:bg-primary/30 bg-primary/20 z-30 rounded-r"
onMouseDown={(e) => handleTrimStart(e, item, 'end')}
aria-label="Trim end"
/>
</>
)}
</div>
);
})}
{/* Playhead - always visible */}
<div
className="absolute top-0 bottom-0 w-1 bg-accent z-30 pointer-events-none rounded-full"
style={{ left: `${playheadLeft}px` }}
>
<div className="absolute -top-1 left-1/2 -translate-x-1/2 w-3 h-3 bg-accent rounded-full" />
</div>
</div>
</div>
</div>
</div>
</div>
);
}
@@ -1 +0,0 @@
# Voice profile management components
@@ -1,4 +1,4 @@
import { Download, Edit, Eye, Mic, Trash2 } from 'lucide-react';
import { Download, Edit, Mic, Trash2 } from 'lucide-react';
import { useState } from 'react';
import { Badge } from '@/components/ui/badge';
import { Button } from '@/components/ui/button';
@@ -16,14 +16,12 @@ import type { VoiceProfileResponse } from '@/lib/api/types';
import { useDeleteProfile, useExportProfile } from '@/lib/hooks/useProfiles';
import { cn } from '@/lib/utils/cn';
import { useUIStore } from '@/stores/uiStore';
import { ProfileDetail } from './ProfileDetail';
interface ProfileCardProps {
profile: VoiceProfileResponse;
}
export function ProfileCard({ profile }: ProfileCardProps) {
const [detailOpen, setDetailOpen] = useState(false);
const [deleteDialogOpen, setDeleteDialogOpen] = useState(false);
const deleteProfile = useDeleteProfile();
const exportProfile = useExportProfile();
@@ -85,14 +83,6 @@ export function ProfileCard({ profile }: ProfileCardProps) {
</Badge>
</div>
<div className="flex gap-0.5 justify-end items-end mt-auto">
<CircleButton
icon={Eye}
onClick={(e) => {
e.stopPropagation();
setDetailOpen(true);
}}
aria-label="View details"
/>
<CircleButton
icon={Download}
onClick={handleExport}
@@ -117,8 +107,6 @@ export function ProfileCard({ profile }: ProfileCardProps) {
</CardContent>
</Card>
<ProfileDetail profileId={profile.id} open={detailOpen} onOpenChange={setDetailOpen} />
<Dialog open={deleteDialogOpen} onOpenChange={setDeleteDialogOpen}>
<DialogContent>
<DialogHeader>
@@ -1,66 +0,0 @@
import { Badge } from '@/components/ui/badge';
import {
Dialog,
DialogContent,
DialogDescription,
DialogHeader,
DialogTitle,
} from '@/components/ui/dialog';
import { useProfile } from '@/lib/hooks/useProfiles';
import { formatDate } from '@/lib/utils/format';
import { SampleList } from './SampleList';
interface ProfileDetailProps {
profileId: string;
open: boolean;
onOpenChange: (open: boolean) => void;
}
export function ProfileDetail({ profileId, open, onOpenChange }: ProfileDetailProps) {
const { data: profile, isLoading } = useProfile(profileId);
if (isLoading) {
return (
<Dialog open={open} onOpenChange={onOpenChange}>
<DialogContent>
<div className="text-muted-foreground">Loading profile...</div>
</DialogContent>
</Dialog>
);
}
if (!profile) {
return null;
}
return (
<Dialog open={open} onOpenChange={onOpenChange}>
<DialogContent className="max-w-3xl max-h-[90vh] overflow-y-auto">
<DialogHeader>
<DialogTitle>{profile.name}</DialogTitle>
<DialogDescription>Manage samples and view profile details</DialogDescription>
</DialogHeader>
<div className="space-y-4">
{profile.description && (
<div>
<h3 className="text-sm font-medium mb-1">Description</h3>
<p className="text-sm text-muted-foreground">{profile.description}</p>
</div>
)}
<div className="flex gap-2">
<Badge variant="outline">{profile.language}</Badge>
<span className="text-xs text-muted-foreground">
Created {formatDate(profile.created_at)}
</span>
</div>
<div className="border-t pt-4">
<SampleList profileId={profileId} />
</div>
</div>
</DialogContent>
</Dialog>
);
}
+125 -147
View File
@@ -42,43 +42,12 @@ import {
import { useSystemAudioCapture } from '@/lib/hooks/useSystemAudioCapture';
import { useTranscription } from '@/lib/hooks/useTranscription';
import { isTauri } from '@/lib/tauri';
import { formatAudioDuration } from '@/lib/utils/audio';
import { formatAudioDuration, getAudioDuration } from '@/lib/utils/audio';
import { useUIStore } from '@/stores/uiStore';
import { AudioSampleRecording } from './AudioSampleRecording';
import { AudioSampleSystem } from './AudioSampleSystem';
import { AudioSampleUpload } from './AudioSampleUpload';
// Helper function to get audio duration from File
async function getAudioDuration(file: File & { recordedDuration?: number }): Promise<number> {
// If the file has a recordedDuration property (from our recording hooks),
// use that instead of trying to read metadata. This fixes issues on Windows
// where WebM files from MediaRecorder don't have proper duration metadata.
if (file.recordedDuration !== undefined && Number.isFinite(file.recordedDuration)) {
return file.recordedDuration;
}
return new Promise((resolve, reject) => {
const audio = new Audio();
const url = URL.createObjectURL(file);
audio.addEventListener('loadedmetadata', () => {
URL.revokeObjectURL(url);
// Check if duration is valid (not Infinity or NaN)
if (Number.isFinite(audio.duration) && audio.duration > 0) {
resolve(audio.duration);
} else {
reject(new Error('Audio file has invalid duration metadata'));
}
});
audio.addEventListener('error', () => {
URL.revokeObjectURL(url);
reject(new Error('Failed to load audio file'));
});
audio.src = url;
});
}
import { SampleList } from './SampleList';
const MAX_AUDIO_DURATION_SECONDS = 30;
@@ -186,7 +155,7 @@ export function ProfileForm() {
stopRecording,
cancelRecording,
} = useAudioRecording({
maxDurationSeconds: 30,
maxDurationSeconds: 29,
onRecordingComplete: (blob, recordedDuration) => {
const file = new File([blob], `recording-${Date.now()}.webm`, {
type: blob.type || 'audio/webm',
@@ -212,7 +181,7 @@ export function ProfileForm() {
stopRecording: stopSystemRecording,
cancelRecording: cancelSystemRecording,
} = useSystemAudioCapture({
maxDurationSeconds: 30,
maxDurationSeconds: 29,
onRecordingComplete: (blob, recordedDuration) => {
const file = new File([blob], `system-audio-${Date.now()}.wav`, {
type: blob.type || 'audio/wav',
@@ -325,7 +294,7 @@ export function ProfileForm() {
},
});
toast({
title: 'Profile updated',
title: 'Voice updated',
description: `"${data.name}" has been updated successfully.`,
});
} else {
@@ -446,17 +415,17 @@ export function ProfileForm() {
<Dialog open={open} onOpenChange={handleOpenChange}>
<DialogContent className="max-w-4xl">
<DialogHeader>
<DialogTitle>{editingProfileId ? 'Edit Profile' : 'Create Voice Profile'}</DialogTitle>
<DialogTitle>{editingProfileId ? 'Edit Voice' : 'Create Voice Profile'}</DialogTitle>
<DialogDescription>
{editingProfileId
? 'Update your voice profile details.'
? 'Update your voice profile details and manage samples.'
: 'Create a new voice profile with an audio sample to clone the voice.'}
</DialogDescription>
</DialogHeader>
<Form {...form}>
<form onSubmit={form.handleSubmit(onSubmit)}>
<div className={`grid gap-6 ${isCreating ? 'grid-cols-2' : 'grid-cols-1'}`}>
<div className="grid gap-6 grid-cols-2">
{/* Left column: Profile info */}
<div className="space-y-4">
<FormField
@@ -513,104 +482,83 @@ export function ProfileForm() {
/>
</div>
{/* Right column: Sample upload section - only show when creating */}
{isCreating && (
<div className="space-y-4 border-l pl-6">
<div>
<h3 className="text-sm font-medium mb-2">Add Sample</h3>
<p className="text-sm text-muted-foreground mb-4">
Provide an audio sample to clone the voice. You can add more samples later.
</p>
</div>
{/* Right column: Sample management */}
<div className="space-y-4 border-l pl-6">
{isCreating ? (
<>
<div>
<h3 className="text-sm font-medium mb-2">Add Sample</h3>
<p className="text-sm text-muted-foreground mb-4">
Provide an audio sample to clone the voice. You can add more samples later.
</p>
</div>
<Tabs
value={sampleMode}
onValueChange={(v) => {
const newMode = v as 'upload' | 'record' | 'system';
// Cancel any active recordings when switching modes
if (isRecording && newMode !== 'record') {
cancelRecording();
}
if (isSystemRecording && newMode !== 'system') {
cancelSystemRecording();
}
setSampleMode(newMode);
}}
>
<TabsList
className={`grid w-full ${isTauri() && isSystemAudioSupported ? 'grid-cols-3' : 'grid-cols-2'}`}
<Tabs
value={sampleMode}
onValueChange={(v) => {
const newMode = v as 'upload' | 'record' | 'system';
// Cancel any active recordings when switching modes
if (isRecording && newMode !== 'record') {
cancelRecording();
}
if (isSystemRecording && newMode !== 'system') {
cancelSystemRecording();
}
setSampleMode(newMode);
}}
>
<TabsTrigger value="upload" className="flex items-center gap-2">
<Upload className="h-4 w-4 shrink-0" />
Upload
</TabsTrigger>
<TabsTrigger value="record" className="flex items-center gap-2">
<Mic className="h-4 w-4 shrink-0" />
Record
</TabsTrigger>
{isTauri() && isSystemAudioSupported && (
<TabsTrigger value="system" className="flex items-center gap-2">
<Monitor className="h-4 w-4 shrink-0" />
System Audio
<TabsList
className={`grid w-full ${isTauri() && isSystemAudioSupported ? 'grid-cols-3' : 'grid-cols-2'}`}
>
<TabsTrigger value="upload" className="flex items-center gap-2">
<Upload className="h-4 w-4 shrink-0" />
Upload
</TabsTrigger>
)}
</TabsList>
<TabsContent value="upload" className="space-y-4">
<FormField
control={form.control}
name="sampleFile"
render={({ field: { onChange, name } }) => (
<AudioSampleUpload
file={selectedFile}
onFileChange={onChange}
onTranscribe={handleTranscribe}
onPlayPause={handlePlayPause}
isPlaying={isPlaying}
isValidating={isValidatingAudio}
isTranscribing={transcribe.isPending}
isDisabled={
audioDuration !== null && audioDuration > MAX_AUDIO_DURATION_SECONDS
}
fieldName={name}
/>
<TabsTrigger value="record" className="flex items-center gap-2">
<Mic className="h-4 w-4 shrink-0" />
Record
</TabsTrigger>
{isTauri() && isSystemAudioSupported && (
<TabsTrigger value="system" className="flex items-center gap-2">
<Monitor className="h-4 w-4 shrink-0" />
System Audio
</TabsTrigger>
)}
/>
</TabsContent>
</TabsList>
<TabsContent value="record" className="space-y-4">
<FormField
control={form.control}
name="sampleFile"
render={() => (
<AudioSampleRecording
file={selectedFile}
isRecording={isRecording}
duration={duration}
onStart={startRecording}
onStop={stopRecording}
onCancel={handleCancelRecording}
onTranscribe={handleTranscribe}
onPlayPause={handlePlayPause}
isPlaying={isPlaying}
isTranscribing={transcribe.isPending}
/>
)}
/>
</TabsContent>
<TabsContent value="upload" className="space-y-4">
<FormField
control={form.control}
name="sampleFile"
render={({ field: { onChange, name } }) => (
<AudioSampleUpload
file={selectedFile}
onFileChange={onChange}
onTranscribe={handleTranscribe}
onPlayPause={handlePlayPause}
isPlaying={isPlaying}
isValidating={isValidatingAudio}
isTranscribing={transcribe.isPending}
isDisabled={
audioDuration !== null && audioDuration > MAX_AUDIO_DURATION_SECONDS
}
fieldName={name}
/>
)}
/>
</TabsContent>
{isTauri() && isSystemAudioSupported && (
<TabsContent value="system" className="space-y-4">
<TabsContent value="record" className="space-y-4">
<FormField
control={form.control}
name="sampleFile"
render={() => (
<AudioSampleSystem
<AudioSampleRecording
file={selectedFile}
isRecording={isSystemRecording}
duration={systemDuration}
onStart={startSystemRecording}
onStop={stopSystemRecording}
isRecording={isRecording}
duration={duration}
onStart={startRecording}
onStop={stopRecording}
onCancel={handleCancelRecording}
onTranscribe={handleTranscribe}
onPlayPause={handlePlayPause}
@@ -620,28 +568,58 @@ export function ProfileForm() {
)}
/>
</TabsContent>
)}
</Tabs>
<FormField
control={form.control}
name="referenceText"
render={({ field }) => (
<FormItem>
<FormLabel>Reference Text</FormLabel>
<FormControl>
<Textarea
placeholder="Enter the exact text spoken in the audio..."
className="min-h-[100px]"
{...field}
{isTauri() && isSystemAudioSupported && (
<TabsContent value="system" className="space-y-4">
<FormField
control={form.control}
name="sampleFile"
render={() => (
<AudioSampleSystem
file={selectedFile}
isRecording={isSystemRecording}
duration={systemDuration}
onStart={startSystemRecording}
onStop={stopSystemRecording}
onCancel={handleCancelRecording}
onTranscribe={handleTranscribe}
onPlayPause={handlePlayPause}
isPlaying={isPlaying}
isTranscribing={transcribe.isPending}
/>
)}
/>
</FormControl>
<FormMessage />
</FormItem>
)}
/>
</div>
)}
</TabsContent>
)}
</Tabs>
<FormField
control={form.control}
name="referenceText"
render={({ field }) => (
<FormItem>
<FormLabel>Reference Text</FormLabel>
<FormControl>
<Textarea
placeholder="Enter the exact text spoken in the audio..."
className="min-h-[100px]"
{...field}
/>
</FormControl>
<FormMessage />
</FormItem>
)}
/>
</>
) : (
// Show sample list when editing
editingProfileId && (
<div>
<SampleList profileId={editingProfileId} />
</div>
)
)}
</div>
</div>
<div className="flex gap-2 justify-end mt-6 pt-4 border-t">
@@ -655,7 +633,7 @@ export function ProfileForm() {
{createProfile.isPending || updateProfile.isPending || addSample.isPending
? 'Saving...'
: editingProfileId
? 'Update Profile'
? 'Save Changes'
: 'Create Profile'}
</Button>
</div>
@@ -1,16 +1,7 @@
import { Mic, Sparkles, Upload } from 'lucide-react';
import { useRef, useState } from 'react';
import { Mic, Sparkles } from 'lucide-react';
import { Button } from '@/components/ui/button';
import { Card, CardContent } from '@/components/ui/card';
import {
Dialog,
DialogContent,
DialogDescription,
DialogFooter,
DialogHeader,
DialogTitle,
} from '@/components/ui/dialog';
import { useImportProfile, useProfiles } from '@/lib/hooks/useProfiles';
import { useProfiles } from '@/lib/hooks/useProfiles';
import { useUIStore } from '@/stores/uiStore';
import { ProfileCard } from './ProfileCard';
import { ProfileForm } from './ProfileForm';
@@ -18,44 +9,6 @@ import { ProfileForm } from './ProfileForm';
export function ProfileList() {
const { data: profiles, isLoading, error } = useProfiles();
const setDialogOpen = useUIStore((state) => state.setProfileDialogOpen);
const importProfile = useImportProfile();
const fileInputRef = useRef<HTMLInputElement>(null);
const [importDialogOpen, setImportDialogOpen] = useState(false);
const [selectedFile, setSelectedFile] = useState<File | null>(null);
const handleImportClick = () => {
fileInputRef.current?.click();
};
const handleFileChange = (e: React.ChangeEvent<HTMLInputElement>) => {
const file = e.target.files?.[0];
if (file) {
// Validate file extension
if (!file.name.endsWith('.voicebox.zip')) {
alert('Please select a valid .voicebox.zip file');
return;
}
setSelectedFile(file);
setImportDialogOpen(true);
}
};
const handleImportConfirm = () => {
if (selectedFile) {
importProfile.mutate(selectedFile, {
onSuccess: () => {
setImportDialogOpen(false);
setSelectedFile(null);
if (fileInputRef.current) {
fileInputRef.current.value = '';
}
},
onError: (error) => {
alert(`Failed to import profile: ${error.message}`);
},
});
}
};
if (isLoading) {
return null;
@@ -73,27 +26,6 @@ export function ProfileList() {
return (
<div className="flex flex-col">
<div className="flex items-center justify-between mb-4 shrink-0">
<h2 className="text-2xl font-bold">Voicebox</h2>
<div className="flex gap-2">
<Button variant="outline" onClick={handleImportClick}>
<Upload className="mr-2 h-4 w-4" />
Import Voice
</Button>
<input
ref={fileInputRef}
type="file"
accept=".voicebox.zip"
onChange={handleFileChange}
className="hidden"
/>
<Button onClick={() => setDialogOpen(true)}>
<Sparkles className="mr-2 h-4 w-4" />
Create Voice
</Button>
</div>
</div>
<div className="shrink-0">
{allProfiles.length === 0 ? (
<Card>
@@ -109,7 +41,7 @@ export function ProfileList() {
</CardContent>
</Card>
) : (
<div className="grid gap-4 grid-cols-3 auto-rows-auto p-1">
<div className="grid gap-4 grid-cols-3 auto-rows-auto p-1 pb-[150px]">
{allProfiles.map((profile) => (
<ProfileCard key={profile.id} profile={profile} />
))}
@@ -118,38 +50,6 @@ export function ProfileList() {
</div>
<ProfileForm />
<Dialog open={importDialogOpen} onOpenChange={setImportDialogOpen}>
<DialogContent>
<DialogHeader>
<DialogTitle>Import Profile</DialogTitle>
<DialogDescription>
Import the profile from "{selectedFile?.name}". This will create a new profile with
all samples.
</DialogDescription>
</DialogHeader>
<DialogFooter>
<Button
variant="outline"
onClick={() => {
setImportDialogOpen(false);
setSelectedFile(null);
if (fileInputRef.current) {
fileInputRef.current.value = '';
}
}}
>
Cancel
</Button>
<Button
onClick={handleImportConfirm}
disabled={importProfile.isPending || !selectedFile}
>
{importProfile.isPending ? 'Importing...' : 'Import'}
</Button>
</DialogFooter>
</DialogContent>
</Dialog>
</div>
);
}
@@ -1,9 +1,9 @@
import { Plus, Trash2, Play } from 'lucide-react';
import { useState } from 'react';
import { Button } from '@/components/ui/button';
import { apiClient } from '@/lib/api/client';
import { useDeleteSample, useProfileSamples } from '@/lib/hooks/useProfiles';
import { usePlayerStore } from '@/stores/playerStore';
import { apiClient } from '@/lib/api/client';
import { SampleUpload } from './SampleUpload';
interface SampleListProps {
@@ -14,7 +14,7 @@ export function SampleList({ profileId }: SampleListProps) {
const { data: samples, isLoading } = useProfileSamples(profileId);
const deleteSample = useDeleteSample();
const [uploadOpen, setUploadOpen] = useState(false);
const setAudio = usePlayerStore((state) => state.setAudio);
const setAudioWithAutoPlay = usePlayerStore((state) => state.setAudioWithAutoPlay);
const currentAudioId = usePlayerStore((state) => state.audioId);
const isPlaying = usePlayerStore((state) => state.isPlaying);
@@ -26,7 +26,7 @@ export function SampleList({ profileId }: SampleListProps) {
const handlePlay = (referenceText: string, sampleId: string) => {
const audioUrl = apiClient.getSampleUrl(sampleId);
setAudio(audioUrl, sampleId, referenceText.substring(0, 50));
setAudioWithAutoPlay(audioUrl, sampleId, null, referenceText.substring(0, 50));
};
if (isLoading) {
@@ -37,7 +37,7 @@ export function SampleList({ profileId }: SampleListProps) {
<div className="space-y-4">
<div className="flex items-center justify-between">
<h3 className="text-lg font-semibold">Audio Samples</h3>
<Button size="sm" onClick={() => setUploadOpen(true)}>
<Button type="button" size="sm" onClick={() => setUploadOpen(true)}>
<Plus className="mr-2 h-4 w-4" />
Add Sample
</Button>
@@ -60,6 +60,7 @@ export function SampleList({ profileId }: SampleListProps) {
</div>
<div className="flex gap-2">
<Button
type="button"
variant="ghost"
size="sm"
onClick={() => handlePlay(sample.reference_text, sample.id)}
@@ -69,6 +70,7 @@ export function SampleList({ profileId }: SampleListProps) {
Play
</Button>
<Button
type="button"
variant="ghost"
size="sm"
onClick={() => handleDelete(sample.id)}
@@ -1,6 +1,6 @@
import { zodResolver } from '@hookform/resolvers/zod';
import { Mic, Monitor, Upload } from 'lucide-react';
import { useState, useEffect } from 'react';
import { useEffect, useState } from 'react';
import { useForm } from 'react-hook-form';
import * as z from 'zod';
import { Button } from '@/components/ui/button';
@@ -73,7 +73,7 @@ export function SampleUpload({ profileId, open, onOpenChange }: SampleUploadProp
stopRecording,
cancelRecording,
} = useAudioRecording({
maxDurationSeconds: 30,
maxDurationSeconds: 29,
onRecordingComplete: (blob, recordedDuration) => {
// Convert blob to File object
const file = new File([blob], `recording-${Date.now()}.webm`, {
@@ -100,7 +100,7 @@ export function SampleUpload({ profileId, open, onOpenChange }: SampleUploadProp
stopRecording: stopSystemRecording,
cancelRecording: cancelSystemRecording,
} = useSystemAudioCapture({
maxDurationSeconds: 30,
maxDurationSeconds: 29,
onRecordingComplete: (blob, recordedDuration) => {
// Convert blob to File object
const file = new File([blob], `system-audio-${Date.now()}.wav`, {
+234
View File
@@ -0,0 +1,234 @@
import { useQuery, useQueryClient } from '@tanstack/react-query';
import { Edit, MoreHorizontal, Plus, Trash2, Mic } from 'lucide-react';
import { useMemo, useRef } from 'react';
import { Button } from '@/components/ui/button';
import {
DropdownMenu,
DropdownMenuContent,
DropdownMenuItem,
DropdownMenuTrigger,
} from '@/components/ui/dropdown-menu';
import { MultiSelect } from '@/components/ui/multi-select';
import {
Table,
TableBody,
TableCell,
TableHead,
TableHeader,
TableRow,
} from '@/components/ui/table';
import { ProfileForm } from '@/components/VoiceProfiles/ProfileForm';
import { apiClient } from '@/lib/api/client';
import type { VoiceProfileResponse } from '@/lib/api/types';
import { BOTTOM_SAFE_AREA_PADDING } from '@/lib/constants/ui';
import { useHistory } from '@/lib/hooks/useHistory';
import { useDeleteProfile, useProfileSamples, useProfiles } from '@/lib/hooks/useProfiles';
import { cn } from '@/lib/utils/cn';
import { usePlayerStore } from '@/stores/playerStore';
import { useUIStore } from '@/stores/uiStore';
export function VoicesTab() {
const { data: profiles, isLoading } = useProfiles();
const { data: historyData } = useHistory({ limit: 1000 });
const queryClient = useQueryClient();
const setDialogOpen = useUIStore((state) => state.setProfileDialogOpen);
const setEditingProfileId = useUIStore((state) => state.setEditingProfileId);
const deleteProfile = useDeleteProfile();
const scrollRef = useRef<HTMLDivElement>(null);
const audioUrl = usePlayerStore((state) => state.audioUrl);
const isPlayerVisible = !!audioUrl;
// Get generation counts per profile
const generationCounts = useMemo(() => {
const counts: Record<string, number> = {};
if (historyData?.items) {
historyData.items.forEach((item) => {
counts[item.profile_id] = (counts[item.profile_id] || 0) + 1;
});
}
return counts;
}, [historyData]);
// Get channel assignments for each profile
const { data: channelAssignments } = useQuery({
queryKey: ['profile-channels'],
queryFn: async () => {
if (!profiles) return {};
const assignments: Record<string, string[]> = {};
for (const profile of profiles) {
try {
const result = await apiClient.getProfileChannels(profile.id);
assignments[profile.id] = result.channel_ids;
} catch {
assignments[profile.id] = [];
}
}
return assignments;
},
enabled: !!profiles,
});
// Get all channels
const { data: channels } = useQuery({
queryKey: ['channels'],
queryFn: () => apiClient.listChannels(),
});
const handleEdit = (profileId: string) => {
setEditingProfileId(profileId);
setDialogOpen(true);
};
const handleDelete = (profileId: string) => {
if (confirm('Are you sure you want to delete this profile?')) {
deleteProfile.mutate(profileId);
}
};
const handleChannelChange = async (profileId: string, channelIds: string[]) => {
try {
await apiClient.setProfileChannels(profileId, channelIds);
queryClient.invalidateQueries({ queryKey: ['profile-channels'] });
} catch (error) {
console.error('Failed to update channels:', error);
}
};
if (isLoading) {
return (
<div className="flex items-center justify-center h-full">
<div className="text-muted-foreground">Loading voices...</div>
</div>
);
}
return (
<div className="h-full flex flex-col relative overflow-hidden">
{/* Scroll Mask - Always visible, behind content */}
<div className="absolute top-0 left-0 right-0 h-16 bg-gradient-to-b from-background to-transparent z-10 pointer-events-none" />
{/* Fixed Header */}
<div className="absolute top-0 left-0 right-0 z-20">
<div className="flex items-center justify-between mb-6">
<h1 className="text-2xl font-bold">Voices</h1>
<Button onClick={() => setDialogOpen(true)}>
<Plus className="h-4 w-4 mr-2" />
New Voice
</Button>
</div>
</div>
{/* Scrollable Content */}
<div
ref={scrollRef}
className={cn(
'flex-1 overflow-y-auto pt-16 relative z-0',
isPlayerVisible && BOTTOM_SAFE_AREA_PADDING,
)}
>
<Table>
<TableHeader>
<TableRow>
<TableHead>Name</TableHead>
<TableHead>Language</TableHead>
<TableHead>Generations</TableHead>
<TableHead>Samples</TableHead>
<TableHead>Channels</TableHead>
<TableHead className="w-[50px]"></TableHead>
</TableRow>
</TableHeader>
<TableBody>
{profiles?.map((profile) => (
<VoiceRow
key={profile.id}
profile={profile}
generationCount={generationCounts[profile.id] || 0}
channelIds={channelAssignments?.[profile.id] || []}
channels={channels || []}
onChannelChange={(channelIds) => handleChannelChange(profile.id, channelIds)}
onEdit={() => handleEdit(profile.id)}
onDelete={() => handleDelete(profile.id)}
/>
))}
</TableBody>
</Table>
</div>
<ProfileForm />
</div>
);
}
interface VoiceRowProps {
profile: VoiceProfileResponse;
generationCount: number;
channelIds: string[];
channels: Array<{ id: string; name: string; is_default: boolean }>;
onChannelChange: (channelIds: string[]) => void;
onEdit: () => void;
onDelete: () => void;
}
function VoiceRow({
profile,
generationCount,
channelIds,
channels,
onChannelChange,
onEdit,
onDelete,
}: VoiceRowProps) {
const { data: samples } = useProfileSamples(profile.id);
return (
<TableRow className="cursor-pointer" onClick={onEdit}>
<TableCell>
<div className="flex items-center gap-2">
<div className="h-8 w-8 rounded-lg bg-muted flex items-center justify-center shrink-0">
<Mic className="h-4 w-4 text-muted-foreground" />
</div>
<div>
<div className="font-medium">{profile.name}</div>
{profile.description && (
<div className="text-sm text-muted-foreground">{profile.description}</div>
)}
</div>
</div>
</TableCell>
<TableCell onClick={(e) => e.stopPropagation()}>{profile.language}</TableCell>
<TableCell onClick={(e) => e.stopPropagation()}>{generationCount}</TableCell>
<TableCell onClick={(e) => e.stopPropagation()}>{samples?.length || 0}</TableCell>
<TableCell onClick={(e) => e.stopPropagation()}>
<MultiSelect
options={channels.map((ch) => ({
value: ch.id,
label: `${ch.name}${ch.is_default ? ' (Default)' : ''}`,
}))}
value={channelIds}
onChange={onChannelChange}
placeholder="Select channels..."
className="min-w-[200px]"
/>
</TableCell>
<TableCell onClick={(e) => e.stopPropagation()}>
<DropdownMenu>
<DropdownMenuTrigger asChild>
<Button variant="ghost" size="icon">
<MoreHorizontal className="h-4 w-4" />
</Button>
</DropdownMenuTrigger>
<DropdownMenuContent>
<DropdownMenuItem onClick={onEdit}>
<Edit className="h-4 w-4 mr-2" />
Edit
</DropdownMenuItem>
<DropdownMenuItem onClick={onDelete} className="text-destructive">
<Trash2 className="h-4 w-4 mr-2" />
Delete
</DropdownMenuItem>
</DropdownMenuContent>
</DropdownMenu>
</TableCell>
</TableRow>
);
}
+27 -19
View File
@@ -1,33 +1,41 @@
import * as React from 'react';
import { Check } from 'lucide-react';
import { cn } from '@/lib/utils/cn';
export interface CheckboxProps extends React.InputHTMLAttributes<HTMLInputElement> {
export interface CheckboxProps {
checked?: boolean;
onCheckedChange?: (checked: boolean) => void;
disabled?: boolean;
className?: string;
id?: string;
}
const Checkbox = React.forwardRef<HTMLInputElement, CheckboxProps>(
({ className, onCheckedChange, ...props }, ref) => {
const handleChange = (e: React.ChangeEvent<HTMLInputElement>) => {
if (onCheckedChange) {
onCheckedChange(e.target.checked);
}
// Call original onChange if provided
if (props.onChange) {
props.onChange(e);
}
};
const Checkbox = React.forwardRef<HTMLButtonElement, CheckboxProps>(
({ checked = false, onCheckedChange, disabled = false, className, id, ...props }, ref) => {
return (
<input
type="checkbox"
<button
type="button"
ref={ref}
id={id}
role="checkbox"
aria-checked={checked}
disabled={disabled}
onClick={() => {
if (!disabled && onCheckedChange) {
onCheckedChange(!checked);
}
}}
className={cn(
'h-4 w-4 rounded border-gray-300 text-primary focus:ring-2 focus:ring-primary focus:ring-offset-2 disabled:cursor-not-allowed disabled:opacity-50',
'h-4 w-4 rounded border-2 flex items-center justify-center shrink-0 transition-colors',
checked ? 'bg-accent border-accent' : 'border-muted-foreground/30',
disabled && 'opacity-50 cursor-not-allowed',
!disabled && 'cursor-pointer',
className,
)}
ref={ref}
onChange={handleChange}
{...props}
/>
>
{checked && <Check className="h-3 w-3 text-accent-foreground" />}
</button>
);
},
);
+102
View File
@@ -0,0 +1,102 @@
import * as React from 'react';
import { ChevronDown, Check } from 'lucide-react';
import { cn } from '@/lib/utils/cn';
import {
DropdownMenu,
DropdownMenuContent,
DropdownMenuTrigger,
} from '@/components/ui/dropdown-menu';
import * as DropdownMenuPrimitive from '@radix-ui/react-dropdown-menu';
export interface MultiSelectOption {
value: string;
label: string;
}
export interface MultiSelectProps {
options: MultiSelectOption[];
value: string[];
onChange: (value: string[]) => void;
placeholder?: string;
className?: string;
}
const MultiSelectCheckboxItem = React.forwardRef<
React.ElementRef<typeof DropdownMenuPrimitive.CheckboxItem>,
React.ComponentPropsWithoutRef<typeof DropdownMenuPrimitive.CheckboxItem>
>(({ className, children, checked, ...props }, ref) => (
<DropdownMenuPrimitive.CheckboxItem
ref={ref}
className={cn(
'relative flex cursor-default select-none items-center rounded-sm py-1.5 pl-8 pr-2 text-xs outline-none focus:bg-accent focus:text-accent-foreground data-disabled:pointer-events-none data-disabled:opacity-50',
className,
)}
checked={checked}
{...props}
>
<span className="absolute left-2 flex h-3.5 w-3.5 items-center justify-center">
<DropdownMenuPrimitive.ItemIndicator>
<Check className="h-4 w-4" />
</DropdownMenuPrimitive.ItemIndicator>
</span>
{children}
</DropdownMenuPrimitive.CheckboxItem>
));
MultiSelectCheckboxItem.displayName = DropdownMenuPrimitive.CheckboxItem.displayName;
export function MultiSelect({
options,
value,
onChange,
placeholder = 'Select...',
className,
}: MultiSelectProps) {
const [open, setOpen] = React.useState(false);
const handleSelect = (optionValue: string) => {
const newValue = value.includes(optionValue)
? value.filter((v) => v !== optionValue)
: [...value, optionValue];
onChange(newValue);
};
const displayText =
value.length === 0
? placeholder
: value.length === 1
? options.find((opt) => opt.value === value[0])?.label || placeholder
: `${value.length} selected`;
return (
<DropdownMenu open={open} onOpenChange={setOpen}>
<DropdownMenuTrigger asChild>
<button
type="button"
className={cn(
'flex h-8 w-full items-center justify-between rounded-full border border-border bg-card px-3 py-2 text-xs ring-offset-background placeholder:text-muted-foreground focus:outline-none focus:ring-2 focus:ring-ring focus:ring-offset-2 disabled:cursor-not-allowed disabled:opacity-50 hover:bg-background/50 transition-all',
className,
)}
>
<span className="line-clamp-1">{displayText}</span>
<ChevronDown className="h-4 w-4 opacity-50" />
</button>
</DropdownMenuTrigger>
<DropdownMenuContent
className="max-h-96 overflow-auto"
align="start"
onCloseAutoFocus={(e) => e.preventDefault()}
>
{options.map((option) => (
<MultiSelectCheckboxItem
key={option.value}
checked={value.includes(option.value)}
onSelect={() => handleSelect(option.value)}
onCheckedChange={() => handleSelect(option.value)}
>
{option.label}
</MultiSelectCheckboxItem>
))}
</DropdownMenuContent>
</DropdownMenu>
);
}
+28
View File
@@ -0,0 +1,28 @@
import * as PopoverPrimitive from '@radix-ui/react-popover';
import * as React from 'react';
import { cn } from '@/lib/utils/cn';
const Popover = PopoverPrimitive.Root;
const PopoverTrigger = PopoverPrimitive.Trigger;
const PopoverContent = React.forwardRef<
React.ElementRef<typeof PopoverPrimitive.Content>,
React.ComponentPropsWithoutRef<typeof PopoverPrimitive.Content>
>(({ className, align = 'center', sideOffset = 4, ...props }, ref) => (
<PopoverPrimitive.Portal>
<PopoverPrimitive.Content
ref={ref}
align={align}
sideOffset={sideOffset}
className={cn(
'z-50 w-72 rounded-md border bg-popover p-4 text-popover-foreground shadow-md outline-none data-[state=open]:animate-in data-[state=closed]:animate-out data-[state=closed]:fade-out-0 data-[state=open]:fade-in-0 data-[state=closed]:zoom-out-95 data-[state=open]:zoom-in-95 data-[side=bottom]:slide-in-from-top-2 data-[side=left]:slide-in-from-right-2 data-[side=right]:slide-in-from-left-2 data-[side=top]:slide-in-from-bottom-2',
className,
)}
{...props}
/>
</PopoverPrimitive.Portal>
));
PopoverContent.displayName = PopoverPrimitive.Content.displayName;
export { Popover, PopoverTrigger, PopoverContent };
+3
View File
@@ -0,0 +1,3 @@
interface Window {
__voiceboxServerStartedByApp?: boolean;
}
+52 -14
View File
@@ -1,6 +1,6 @@
import { useCallback, useEffect, useState } from 'react';
import { check, type Update } from '@tauri-apps/plugin-updater';
import { relaunch } from '@tauri-apps/plugin-process';
import { check, type Update } from '@tauri-apps/plugin-updater';
import { useCallback, useEffect, useState } from 'react';
export interface UpdateStatus {
checking: boolean;
@@ -8,12 +8,18 @@ export interface UpdateStatus {
version?: string;
downloading: boolean;
installing: boolean;
readyToInstall: boolean;
error?: string;
downloadProgress?: number; // 0-100 percentage
downloadedBytes?: number;
totalBytes?: number;
}
// Check if we're on Windows (NSIS installer handles restart automatically)
const isWindows = () => {
return navigator.userAgent.includes('Windows');
};
const isTauri = () => {
return '__TAURI_INTERNALS__' in window;
};
@@ -24,6 +30,7 @@ export function useAutoUpdater(checkOnMount = false) {
available: false,
downloading: false,
installing: false,
readyToInstall: false,
});
const [update, setUpdate] = useState<Update | null>(null);
@@ -46,6 +53,7 @@ export function useAutoUpdater(checkOnMount = false) {
version: foundUpdate.version,
downloading: false,
installing: false,
readyToInstall: false,
});
} else {
setStatus({
@@ -53,6 +61,7 @@ export function useAutoUpdater(checkOnMount = false) {
available: false,
downloading: false,
installing: false,
readyToInstall: false,
});
}
} catch (error) {
@@ -61,11 +70,13 @@ export function useAutoUpdater(checkOnMount = false) {
available: false,
downloading: false,
installing: false,
readyToInstall: false,
error: error instanceof Error ? error.message : 'Failed to check for updates',
});
}
}, []);
// Download the update (but don't install yet)
const downloadAndInstall = async () => {
if (!update || !isTauri()) return;
@@ -75,28 +86,28 @@ export function useAutoUpdater(checkOnMount = false) {
let downloadedBytes = 0;
let totalBytes = 0;
await update.downloadAndInstall((event) => {
// Just download the update
await update.download((event) => {
switch (event.event) {
case 'Started':
totalBytes = event.data.contentLength || 0;
downloadedBytes = 0;
setStatus((prev) => ({
...prev,
setStatus((prev) => ({
...prev,
downloading: true,
totalBytes,
downloadedBytes: 0,
downloadProgress: 0
downloadProgress: 0,
}));
break;
case 'Progress': {
downloadedBytes += event.data.chunkLength;
const progress = totalBytes > 0
? Math.round((downloadedBytes / totalBytes) * 100)
: undefined;
const progress =
totalBytes > 0 ? Math.round((downloadedBytes / totalBytes) * 100) : undefined;
setStatus((prev) => ({
...prev,
downloadedBytes,
downloadProgress: progress
downloadProgress: progress,
}));
break;
}
@@ -104,22 +115,48 @@ export function useAutoUpdater(checkOnMount = false) {
setStatus((prev) => ({
...prev,
downloading: false,
installing: true,
downloadProgress: 100
readyToInstall: true,
downloadProgress: 100,
}));
break;
}
});
await relaunch();
} catch (error) {
setStatus((prev) => ({
...prev,
downloading: false,
installing: false,
readyToInstall: false,
downloadProgress: undefined,
downloadedBytes: undefined,
totalBytes: undefined,
error: error instanceof Error ? error.message : 'Failed to download update',
}));
}
};
// Install the downloaded update and restart the app
const restartAndInstall = async () => {
if (!update || !isTauri()) return;
try {
setStatus((prev) => ({ ...prev, installing: true, error: undefined }));
// Install the update
await update.install();
// On Windows with NSIS, the installer handles the restart automatically.
// The process will be killed by the NSIS installer, so we won't reach here.
// On macOS/Linux, we need to manually relaunch.
if (!isWindows()) {
await relaunch();
}
// If we're on Windows and somehow still running, the NSIS installer
// should have already handled everything. Just wait for the process to end.
} catch (error) {
setStatus((prev) => ({
...prev,
installing: false,
error: error instanceof Error ? error.message : 'Failed to install update',
}));
}
@@ -135,5 +172,6 @@ export function useAutoUpdater(checkOnMount = false) {
status,
checkForUpdates,
downloadAndInstall,
restartAndInstall,
};
}
+189
View File
@@ -13,6 +13,16 @@ import type {
ModelStatusListResponse,
ModelDownloadRequest,
ActiveTasksResponse,
StoryCreate,
StoryResponse,
StoryDetailResponse,
StoryItemCreate,
StoryItemDetail,
StoryItemBatchUpdate,
StoryItemReorder,
StoryItemMove,
StoryItemTrim,
StoryItemSplit,
} from './types';
class ApiClient {
@@ -279,6 +289,185 @@ class ApiClient {
async getActiveTasks(): Promise<ActiveTasksResponse> {
return this.request<ActiveTasksResponse>('/tasks/active');
}
// Audio Channels
async listChannels(): Promise<
Array<{
id: string;
name: string;
is_default: boolean;
device_ids: string[];
created_at: string;
}>
> {
return this.request('/channels');
}
async createChannel(data: {
name: string;
device_ids: string[];
}): Promise<{
id: string;
name: string;
is_default: boolean;
device_ids: string[];
created_at: string;
}> {
return this.request('/channels', {
method: 'POST',
body: JSON.stringify(data),
});
}
async updateChannel(
channelId: string,
data: {
name?: string;
device_ids?: string[];
},
): Promise<{
id: string;
name: string;
is_default: boolean;
device_ids: string[];
created_at: string;
}> {
return this.request(`/channels/${channelId}`, {
method: 'PUT',
body: JSON.stringify(data),
});
}
async deleteChannel(channelId: string): Promise<{ message: string }> {
return this.request(`/channels/${channelId}`, {
method: 'DELETE',
});
}
async getChannelVoices(channelId: string): Promise<{ profile_ids: string[] }> {
return this.request(`/channels/${channelId}/voices`);
}
async setChannelVoices(
channelId: string,
profileIds: string[],
): Promise<{ message: string }> {
return this.request(`/channels/${channelId}/voices`, {
method: 'PUT',
body: JSON.stringify({ profile_ids: profileIds }),
});
}
async getProfileChannels(profileId: string): Promise<{ channel_ids: string[] }> {
return this.request(`/profiles/${profileId}/channels`);
}
async setProfileChannels(
profileId: string,
channelIds: string[],
): Promise<{ message: string }> {
return this.request(`/profiles/${profileId}/channels`, {
method: 'PUT',
body: JSON.stringify({ channel_ids: channelIds }),
});
}
// Stories
async listStories(): Promise<StoryResponse[]> {
return this.request<StoryResponse[]>('/stories');
}
async createStory(data: StoryCreate): Promise<StoryResponse> {
return this.request<StoryResponse>('/stories', {
method: 'POST',
body: JSON.stringify(data),
});
}
async getStory(storyId: string): Promise<StoryDetailResponse> {
return this.request<StoryDetailResponse>(`/stories/${storyId}`);
}
async updateStory(storyId: string, data: StoryCreate): Promise<StoryResponse> {
return this.request<StoryResponse>(`/stories/${storyId}`, {
method: 'PUT',
body: JSON.stringify(data),
});
}
async deleteStory(storyId: string): Promise<void> {
await this.request<void>(`/stories/${storyId}`, {
method: 'DELETE',
});
}
async addStoryItem(storyId: string, data: StoryItemCreate): Promise<StoryItemDetail> {
return this.request<StoryItemDetail>(`/stories/${storyId}/items`, {
method: 'POST',
body: JSON.stringify(data),
});
}
async removeStoryItem(storyId: string, itemId: string): Promise<void> {
await this.request<void>(`/stories/${storyId}/items/${itemId}`, {
method: 'DELETE',
});
}
async updateStoryItemTimes(storyId: string, data: StoryItemBatchUpdate): Promise<void> {
await this.request<void>(`/stories/${storyId}/items/times`, {
method: 'PUT',
body: JSON.stringify(data),
});
}
async reorderStoryItems(storyId: string, data: StoryItemReorder): Promise<StoryItemDetail[]> {
return this.request<StoryItemDetail[]>(`/stories/${storyId}/items/reorder`, {
method: 'PUT',
body: JSON.stringify(data),
});
}
async moveStoryItem(storyId: string, itemId: string, data: StoryItemMove): Promise<StoryItemDetail> {
return this.request<StoryItemDetail>(`/stories/${storyId}/items/${itemId}/move`, {
method: 'PUT',
body: JSON.stringify(data),
});
}
async trimStoryItem(storyId: string, itemId: string, data: StoryItemTrim): Promise<StoryItemDetail> {
return this.request<StoryItemDetail>(`/stories/${storyId}/items/${itemId}/trim`, {
method: 'PUT',
body: JSON.stringify(data),
});
}
async splitStoryItem(storyId: string, itemId: string, data: StoryItemSplit): Promise<StoryItemDetail[]> {
return this.request<StoryItemDetail[]>(`/stories/${storyId}/items/${itemId}/split`, {
method: 'POST',
body: JSON.stringify(data),
});
}
async duplicateStoryItem(storyId: string, itemId: string): Promise<StoryItemDetail> {
return this.request<StoryItemDetail>(`/stories/${storyId}/items/${itemId}/duplicate`, {
method: 'POST',
});
}
async exportStoryAudio(storyId: string): Promise<Blob> {
const url = `${this.getBaseUrl()}/stories/${storyId}/export-audio`;
const response = await fetch(url);
if (!response.ok) {
const error = await response.json().catch(() => ({
detail: response.statusText,
}));
throw new Error(error.detail || `HTTP error! status: ${response.status}`);
}
return response.blob();
}
}
export const apiClient = new ApiClient();
+76
View File
@@ -123,3 +123,79 @@ export interface ActiveTasksResponse {
downloads: ActiveDownloadTask[];
generations: ActiveGenerationTask[];
}
export interface StoryCreate {
name: string;
description?: string;
}
export interface StoryResponse {
id: string;
name: string;
description?: string;
created_at: string;
updated_at: string;
item_count: number;
}
export interface StoryItemDetail {
id: string;
story_id: string;
generation_id: string;
start_time_ms: number;
track: number;
trim_start_ms: number;
trim_end_ms: number;
created_at: string;
profile_id: string;
profile_name: string;
text: string;
language: string;
audio_path: string;
duration: number;
seed?: number;
instruct?: string;
generation_created_at: string;
}
export interface StoryDetailResponse {
id: string;
name: string;
description?: string;
created_at: string;
updated_at: string;
items: StoryItemDetail[];
}
export interface StoryItemCreate {
generation_id: string;
start_time_ms?: number;
track?: number;
}
export interface StoryItemUpdateTime {
generation_id: string;
start_time_ms: number;
}
export interface StoryItemBatchUpdate {
updates: StoryItemUpdateTime[];
}
export interface StoryItemReorder {
generation_ids: string[];
}
export interface StoryItemMove {
start_time_ms: number;
track: number;
}
export interface StoryItemTrim {
trim_start_ms: number;
trim_end_ms: number;
}
export interface StoryItemSplit {
split_time_ms: number;
}
+15
View File
@@ -0,0 +1,15 @@
/**
* UI layout constants for safe area padding
*/
/**
* Top safe area padding - height of the drag region bar
* Corresponds to Tailwind's pt-12 (3rem / 48px)
*/
export const TOP_SAFE_AREA_PADDING = 'pt-12';
/**
* Bottom safe area padding - height of the audio player
* Corresponds to Tailwind's pb-32 (8rem / 128px)
*/
export const BOTTOM_SAFE_AREA_PADDING = 'pb-32';
-1
View File
@@ -1 +0,0 @@
# React Query hooks will be placed here
+1 -1
View File
@@ -8,7 +8,7 @@ interface UseAudioRecordingOptions {
}
export function useAudioRecording({
maxDurationSeconds = 30,
maxDurationSeconds = 29,
onRecordingComplete,
}: UseAudioRecordingOptions = {}) {
const [isRecording, setIsRecording] = useState(false);
+122
View File
@@ -0,0 +1,122 @@
import { zodResolver } from '@hookform/resolvers/zod';
import { useState } from 'react';
import { useForm } from 'react-hook-form';
import * as z from 'zod';
import { useToast } from '@/components/ui/use-toast';
import { apiClient } from '@/lib/api/client';
import { LANGUAGE_CODES, type LanguageCode } from '@/lib/constants/languages';
import { useGeneration } from '@/lib/hooks/useGeneration';
import { useModelDownloadToast } from '@/lib/hooks/useModelDownloadToast';
import { useGenerationStore } from '@/stores/generationStore';
import { usePlayerStore } from '@/stores/playerStore';
const generationSchema = z.object({
text: z.string().min(1, 'Text is required').max(5000),
language: z.enum(LANGUAGE_CODES as [LanguageCode, ...LanguageCode[]]),
seed: z.number().int().optional(),
modelSize: z.enum(['1.7B', '0.6B']).optional(),
instruct: z.string().max(500).optional(),
});
export type GenerationFormValues = z.infer<typeof generationSchema>;
interface UseGenerationFormOptions {
onSuccess?: (generationId: string) => void;
defaultValues?: Partial<GenerationFormValues>;
}
export function useGenerationForm(options: UseGenerationFormOptions = {}) {
const { toast } = useToast();
const generation = useGeneration();
const setAudioWithAutoPlay = usePlayerStore((state) => state.setAudioWithAutoPlay);
const setIsGenerating = useGenerationStore((state) => state.setIsGenerating);
const [downloadingModelName, setDownloadingModelName] = useState<string | null>(null);
const [downloadingDisplayName, setDownloadingDisplayName] = useState<string | null>(null);
useModelDownloadToast({
modelName: downloadingModelName || '',
displayName: downloadingDisplayName || '',
enabled: !!downloadingModelName,
});
const form = useForm<GenerationFormValues>({
resolver: zodResolver(generationSchema),
defaultValues: {
text: '',
language: 'en',
seed: undefined,
modelSize: '1.7B',
instruct: '',
...options.defaultValues,
},
});
async function handleSubmit(
data: GenerationFormValues,
selectedProfileId: string | null,
): Promise<void> {
if (!selectedProfileId) {
toast({
title: 'No profile selected',
description: 'Please select a voice profile from the cards above.',
variant: 'destructive',
});
return;
}
try {
setIsGenerating(true);
const modelName = `qwen-tts-${data.modelSize}`;
const displayName = data.modelSize === '1.7B' ? 'Qwen TTS 1.7B' : 'Qwen TTS 0.6B';
try {
const modelStatus = await apiClient.getModelStatus();
const model = modelStatus.models.find((m) => m.model_name === modelName);
if (model && !model.downloaded) {
setDownloadingModelName(modelName);
setDownloadingDisplayName(displayName);
}
} catch (error) {
console.error('Failed to check model status:', error);
}
const result = await generation.mutateAsync({
profile_id: selectedProfileId,
text: data.text,
language: data.language,
seed: data.seed,
model_size: data.modelSize,
instruct: data.instruct || undefined,
});
toast({
title: 'Generation complete!',
description: `Audio generated (${result.duration.toFixed(2)}s)`,
});
const audioUrl = apiClient.getAudioUrl(result.id);
setAudioWithAutoPlay(audioUrl, result.id, selectedProfileId, data.text.substring(0, 50));
form.reset();
options.onSuccess?.(result.id);
} catch (error) {
toast({
title: 'Generation failed',
description: error instanceof Error ? error.message : 'Failed to generate audio',
variant: 'destructive',
});
} finally {
setIsGenerating(false);
setDownloadingModelName(null);
setDownloadingDisplayName(null);
}
}
return {
form,
handleSubmit,
isPending: generation.isPending,
};
}
+216
View File
@@ -0,0 +1,216 @@
import { useMutation, useQuery, useQueryClient } from '@tanstack/react-query';
import { apiClient } from '@/lib/api/client';
import type { StoryCreate, StoryItemCreate, StoryItemBatchUpdate, StoryItemReorder, StoryItemMove, StoryItemTrim, StoryItemSplit } from '@/lib/api/types';
import { isTauri } from '@/lib/tauri';
export function useStories() {
return useQuery({
queryKey: ['stories'],
queryFn: () => apiClient.listStories(),
});
}
export function useStory(storyId: string | null) {
return useQuery({
queryKey: ['stories', storyId],
queryFn: () => apiClient.getStory(storyId!),
enabled: !!storyId,
});
}
export function useCreateStory() {
const queryClient = useQueryClient();
return useMutation({
mutationFn: (data: StoryCreate) => apiClient.createStory(data),
onSuccess: () => {
queryClient.invalidateQueries({ queryKey: ['stories'] });
},
});
}
export function useUpdateStory() {
const queryClient = useQueryClient();
return useMutation({
mutationFn: ({ storyId, data }: { storyId: string; data: StoryCreate }) =>
apiClient.updateStory(storyId, data),
onSuccess: (_, variables) => {
queryClient.invalidateQueries({ queryKey: ['stories'] });
queryClient.invalidateQueries({ queryKey: ['stories', variables.storyId] });
},
});
}
export function useDeleteStory() {
const queryClient = useQueryClient();
return useMutation({
mutationFn: (storyId: string) => apiClient.deleteStory(storyId),
onSuccess: () => {
queryClient.invalidateQueries({ queryKey: ['stories'] });
},
});
}
export function useAddStoryItem() {
const queryClient = useQueryClient();
return useMutation({
mutationFn: ({ storyId, data }: { storyId: string; data: StoryItemCreate }) =>
apiClient.addStoryItem(storyId, data),
onSuccess: (_, variables) => {
queryClient.invalidateQueries({ queryKey: ['stories'] });
queryClient.invalidateQueries({ queryKey: ['stories', variables.storyId] });
},
});
}
export function useRemoveStoryItem() {
const queryClient = useQueryClient();
return useMutation({
mutationFn: ({ storyId, itemId }: { storyId: string; itemId: string }) =>
apiClient.removeStoryItem(storyId, itemId),
onSuccess: (_, variables) => {
queryClient.invalidateQueries({ queryKey: ['stories'] });
queryClient.invalidateQueries({ queryKey: ['stories', variables.storyId] });
},
});
}
export function useUpdateStoryItemTimes() {
const queryClient = useQueryClient();
return useMutation({
mutationFn: ({ storyId, data }: { storyId: string; data: StoryItemBatchUpdate }) =>
apiClient.updateStoryItemTimes(storyId, data),
onSuccess: (_, variables) => {
queryClient.invalidateQueries({ queryKey: ['stories'] });
queryClient.invalidateQueries({ queryKey: ['stories', variables.storyId] });
},
});
}
export function useReorderStoryItems() {
const queryClient = useQueryClient();
return useMutation({
mutationFn: ({ storyId, data }: { storyId: string; data: StoryItemReorder }) =>
apiClient.reorderStoryItems(storyId, data),
onSuccess: (_, variables) => {
queryClient.invalidateQueries({ queryKey: ['stories'] });
queryClient.invalidateQueries({ queryKey: ['stories', variables.storyId] });
},
});
}
export function useMoveStoryItem() {
const queryClient = useQueryClient();
return useMutation({
mutationFn: ({ storyId, itemId, data }: { storyId: string; itemId: string; data: StoryItemMove }) =>
apiClient.moveStoryItem(storyId, itemId, data),
onSuccess: (_, variables) => {
queryClient.invalidateQueries({ queryKey: ['stories'] });
queryClient.invalidateQueries({ queryKey: ['stories', variables.storyId] });
},
});
}
export function useTrimStoryItem() {
const queryClient = useQueryClient();
return useMutation({
mutationFn: ({ storyId, itemId, data }: { storyId: string; itemId: string; data: StoryItemTrim }) =>
apiClient.trimStoryItem(storyId, itemId, data),
onSuccess: (_, variables) => {
queryClient.invalidateQueries({ queryKey: ['stories'] });
queryClient.invalidateQueries({ queryKey: ['stories', variables.storyId] });
},
});
}
export function useSplitStoryItem() {
const queryClient = useQueryClient();
return useMutation({
mutationFn: ({ storyId, itemId, data }: { storyId: string; itemId: string; data: StoryItemSplit }) =>
apiClient.splitStoryItem(storyId, itemId, data),
onSuccess: (_, variables) => {
queryClient.invalidateQueries({ queryKey: ['stories'] });
queryClient.invalidateQueries({ queryKey: ['stories', variables.storyId] });
},
});
}
export function useDuplicateStoryItem() {
const queryClient = useQueryClient();
return useMutation({
mutationFn: ({ storyId, itemId }: { storyId: string; itemId: string }) =>
apiClient.duplicateStoryItem(storyId, itemId),
onSuccess: (_, variables) => {
queryClient.invalidateQueries({ queryKey: ['stories'] });
queryClient.invalidateQueries({ queryKey: ['stories', variables.storyId] });
},
});
}
export function useExportStoryAudio() {
return useMutation({
mutationFn: async ({ storyId, storyName }: { storyId: string; storyName: string }) => {
const blob = await apiClient.exportStoryAudio(storyId);
// Create safe filename
const safeName = storyName.substring(0, 50).replace(/[^a-z0-9]/gi, '-').toLowerCase();
const filename = `${safeName || 'story'}.wav`;
if (isTauri()) {
// Use Tauri's native save dialog
try {
const { save } = await import('@tauri-apps/plugin-dialog');
const filePath = await save({
defaultPath: filename,
filters: [
{
name: 'Audio File',
extensions: ['wav'],
},
],
});
if (filePath) {
// Write file using Tauri's filesystem API
const { writeBinaryFile } = await import('@tauri-apps/plugin-fs');
const arrayBuffer = await blob.arrayBuffer();
await writeBinaryFile(filePath, new Uint8Array(arrayBuffer));
}
} catch (error) {
console.error('Failed to use Tauri dialog, falling back to browser download:', error);
// Fall back to browser download if Tauri dialog fails
const url = window.URL.createObjectURL(blob);
const a = document.createElement('a');
a.href = url;
a.download = filename;
document.body.appendChild(a);
a.click();
window.URL.revokeObjectURL(url);
document.body.removeChild(a);
}
} else {
// Browser: trigger download
const url = window.URL.createObjectURL(blob);
const a = document.createElement('a');
a.href = url;
a.download = filename;
document.body.appendChild(a);
a.click();
window.URL.revokeObjectURL(url);
document.body.removeChild(a);
}
return blob;
},
});
}
+389
View File
@@ -0,0 +1,389 @@
import { useCallback, useEffect, useRef } from 'react';
import { apiClient } from '@/lib/api/client';
import type { StoryItemDetail } from '@/lib/api/types';
import { useStoryStore } from '@/stores/storyStore';
interface ActiveSource {
source: AudioBufferSourceNode;
itemId: string;
generationId: string;
startTimeMs: number;
endTimeMs: number;
}
/**
* Hook for managing timecode-based story playback using Web Audio API.
* Supports multiple simultaneous audio sources for overlapping clips on different tracks.
* Uses AudioContext for sample-accurate timing synchronization.
*/
export function useStoryPlayback(items: StoryItemDetail[] | undefined) {
const isPlaying = useStoryStore((state) => state.isPlaying);
const playbackItems = useStoryStore((state) => state.playbackItems);
const playbackStartContextTime = useStoryStore((state) => state.playbackStartContextTime);
const playbackStartStoryTime = useStoryStore((state) => state.playbackStartStoryTime);
const setPlaybackTiming = useStoryStore((state) => state.setPlaybackTiming);
// AudioContext instance (created once)
const audioContextRef = useRef<AudioContext | null>(null);
// Master gain for volume control
const masterGainRef = useRef<GainNode | null>(null);
// Preloaded AudioBuffers by generation_id (audio file is shared between split clips)
const audioBuffersRef = useRef<Map<string, AudioBuffer>>(new Map());
// Currently playing AudioBufferSourceNodes by item.id (unique per clip)
const activeSourcesRef = useRef<Map<string, ActiveSource>>(new Map());
// Animation frame for syncing visual playhead
const animationFrameRef = useRef<number | null>(null);
// Get or create AudioContext and audio graph
const getAudioContext = useCallback(() => {
if (!audioContextRef.current) {
audioContextRef.current = new AudioContext();
console.log(
'[StoryPlayback] Created AudioContext, sample rate:',
audioContextRef.current.sampleRate,
);
// Create master gain node for volume control
masterGainRef.current = audioContextRef.current.createGain();
masterGainRef.current.gain.value = 1;
masterGainRef.current.connect(audioContextRef.current.destination);
}
// Resume context if suspended (browser autoplay policy)
if (audioContextRef.current.state === 'suspended') {
audioContextRef.current.resume().catch(() => {
// Ignore resume errors
});
}
return audioContextRef.current;
}, []);
// Stop a source by item id
const stopSource = useCallback((itemId: string) => {
const activeSource = activeSourcesRef.current.get(itemId);
if (activeSource) {
try {
activeSource.source.stop();
} catch {
// Source may have already stopped
}
activeSourcesRef.current.delete(itemId);
}
}, []);
// Preload audio files as AudioBuffers
useEffect(() => {
if (!items || items.length === 0) {
// Clear preloaded buffers when no items
audioBuffersRef.current.clear();
return;
}
const currentIds = new Set(items.map((item) => item.generation_id));
const audioContext = getAudioContext();
// Remove buffers for items that no longer exist
for (const [id] of audioBuffersRef.current) {
if (!currentIds.has(id)) {
audioBuffersRef.current.delete(id);
}
}
// Preload audio for new items
const preloadPromises: Promise<void>[] = [];
for (const item of items) {
if (!audioBuffersRef.current.has(item.generation_id)) {
const audioUrl = apiClient.getAudioUrl(item.generation_id);
console.log('[StoryPlayback] Preloading audio buffer:', item.generation_id);
const preloadPromise = fetch(audioUrl)
.then((response) => response.arrayBuffer())
.then((arrayBuffer) => audioContext.decodeAudioData(arrayBuffer))
.then((audioBuffer) => {
audioBuffersRef.current.set(item.generation_id, audioBuffer);
console.log(
'[StoryPlayback] Preloaded buffer:',
item.generation_id,
'duration:',
audioBuffer.duration,
);
})
.catch((err) => {
console.error('[StoryPlayback] Failed to preload audio:', item.generation_id, err);
});
preloadPromises.push(preloadPromise);
}
}
Promise.all(preloadPromises).then(() => {
console.log('[StoryPlayback] Preloaded', audioBuffersRef.current.size, 'audio buffers');
});
}, [items, getAudioContext]);
// Cleanup AudioContext on unmount
useEffect(() => {
return () => {
// Stop all sources
for (const [itemId] of activeSourcesRef.current) {
stopSource(itemId);
}
activeSourcesRef.current.clear();
// Clean up audio graph
if (masterGainRef.current) {
masterGainRef.current.disconnect();
masterGainRef.current = null;
}
if (audioContextRef.current && audioContextRef.current.state !== 'closed') {
audioContextRef.current.close().catch(() => {
// Ignore errors when closing
});
audioContextRef.current = null;
}
if (animationFrameRef.current !== null) {
cancelAnimationFrame(animationFrameRef.current);
}
};
}, [stopSource]);
// Find ALL items that should be playing at a given story time
const findActiveItems = useCallback(
(storyTimeMs: number, itemList: StoryItemDetail[]): StoryItemDetail[] => {
return itemList.filter((item) => {
const itemStart = item.start_time_ms;
// Use effective duration (accounting for trims)
const trimStartMs = item.trim_start_ms || 0;
const trimEndMs = item.trim_end_ms || 0;
const effectiveDurationMs = item.duration * 1000 - trimStartMs - trimEndMs;
const itemEnd = item.start_time_ms + effectiveDurationMs;
return storyTimeMs >= itemStart && storyTimeMs < itemEnd;
});
},
[],
);
// Convert AudioContext time to story time (ms)
const contextTimeToStoryTime = useCallback(
(contextTime: number): number => {
if (playbackStartContextTime === null || playbackStartStoryTime === null) {
return 0;
}
const elapsedContextTime = contextTime - playbackStartContextTime;
return playbackStartStoryTime + elapsedContextTime * 1000;
},
[playbackStartContextTime, playbackStartStoryTime],
);
// Convert story time (ms) to AudioContext time
const storyTimeToContextTime = useCallback(
(storyTimeMs: number): number => {
if (playbackStartContextTime === null || playbackStartStoryTime === null) {
return 0;
}
const elapsedStoryTime = (storyTimeMs - playbackStartStoryTime) / 1000;
return playbackStartContextTime + elapsedStoryTime;
},
[playbackStartContextTime, playbackStartStoryTime],
);
// Stop all sources
const stopAllSources = useCallback(() => {
console.log('[StoryPlayback] Stopping all sources');
for (const [itemId] of activeSourcesRef.current) {
stopSource(itemId);
}
activeSourcesRef.current.clear();
}, [stopSource]);
// Schedule playback for all items that should be playing
const schedulePlayback = useCallback(
(storyTimeMs: number, itemList: StoryItemDetail[]) => {
const audioContext = getAudioContext();
const currentContextTime = audioContext.currentTime;
// Find all items that should be playing
const shouldBePlaying = findActiveItems(storyTimeMs, itemList);
const shouldBePlayingIds = new Set(shouldBePlaying.map((item) => item.id));
// Stop sources that shouldn't be playing anymore
for (const [itemId] of activeSourcesRef.current) {
if (!shouldBePlayingIds.has(itemId)) {
stopSource(itemId);
}
}
// Schedule new sources for items that should be playing
for (const item of shouldBePlaying) {
if (!activeSourcesRef.current.has(item.id)) {
const buffer = audioBuffersRef.current.get(item.generation_id);
if (!buffer) {
console.warn('[StoryPlayback] Buffer not loaded for:', item.generation_id);
continue;
}
// Calculate when this item should start in AudioContext time
const itemStartContextTime = storyTimeToContextTime(item.start_time_ms);
// Calculate effective duration and trim offsets
const trimStartSec = (item.trim_start_ms || 0) / 1000;
const trimEndSec = (item.trim_end_ms || 0) / 1000;
const effectiveDuration = item.duration - trimStartSec - trimEndSec;
const itemEndStoryTime = item.start_time_ms + effectiveDuration * 1000;
// Calculate offset into the buffer (if seeking mid-way)
// Offset is relative to the trimmed start of the clip
const offsetIntoEffectiveClip = Math.max(0, (storyTimeMs - item.start_time_ms) / 1000);
const offsetIntoBuffer = trimStartSec + offsetIntoEffectiveClip;
const duration = effectiveDuration - offsetIntoEffectiveClip;
// If the item should have already started, schedule it to start immediately
const startAtContextTime = Math.max(currentContextTime, itemStartContextTime);
console.log('[StoryPlayback] Scheduling source:', {
itemId: item.id,
generationId: item.generation_id,
storyTimeMs,
itemStart: item.start_time_ms,
offsetIntoBuffer,
startAtContextTime,
duration,
});
const source = audioContext.createBufferSource();
source.buffer = buffer;
source.connect(masterGainRef.current || audioContext.destination);
const activeSource: ActiveSource = {
source,
itemId: item.id,
generationId: item.generation_id,
startTimeMs: item.start_time_ms,
endTimeMs: itemEndStoryTime,
};
activeSourcesRef.current.set(item.id, activeSource);
// Schedule playback
source.start(startAtContextTime, offsetIntoBuffer, duration);
// Clean up when source ends
source.onended = () => {
console.log('[StoryPlayback] Source ended:', item.id);
activeSourcesRef.current.delete(item.id);
};
}
}
},
[getAudioContext, findActiveItems, storyTimeToContextTime, stopSource],
);
// Sync visual playhead from AudioContext time
useEffect(() => {
if (!isPlaying || playbackStartContextTime === null || playbackStartStoryTime === null) {
if (animationFrameRef.current !== null) {
cancelAnimationFrame(animationFrameRef.current);
animationFrameRef.current = null;
}
return;
}
const audioContext = getAudioContext();
const itemList = playbackItems || [];
const syncPlayhead = () => {
if (!useStoryStore.getState().isPlaying) {
return;
}
const currentContextTime = audioContext.currentTime;
const currentStoryTime = contextTimeToStoryTime(currentContextTime);
const totalDuration = useStoryStore.getState().totalDurationMs;
// Update store with current story time
useStoryStore.setState({ currentTimeMs: Math.min(currentStoryTime, totalDuration) });
// Schedule any items that should be playing
schedulePlayback(currentStoryTime, itemList);
// Check if we've reached the end
if (currentStoryTime >= totalDuration) {
// Check if all sources have ended
if (activeSourcesRef.current.size === 0) {
console.log('[StoryPlayback] Reached end');
useStoryStore.getState().stop();
return;
}
}
// Continue sync loop
animationFrameRef.current = requestAnimationFrame(syncPlayhead);
};
// Initial sync
const currentContextTime = audioContext.currentTime;
const currentStoryTime = contextTimeToStoryTime(currentContextTime);
schedulePlayback(currentStoryTime, itemList);
// Start sync loop
animationFrameRef.current = requestAnimationFrame(syncPlayhead);
return () => {
if (animationFrameRef.current !== null) {
cancelAnimationFrame(animationFrameRef.current);
animationFrameRef.current = null;
}
};
}, [
isPlaying,
playbackItems,
playbackStartContextTime,
playbackStartStoryTime,
getAudioContext,
contextTimeToStoryTime,
schedulePlayback,
]);
// Handle play/pause changes - stop sources when paused
useEffect(() => {
if (!isPlaying) {
console.log('[StoryPlayback] Stopping playback');
stopAllSources();
}
}, [isPlaying, stopAllSources]);
// Handle seek - reset timing anchors when they become null (triggered by seek)
useEffect(() => {
if (!isPlaying || !playbackItems || playbackItems.length === 0) {
return;
}
// Only run when timing anchors are null (after a seek)
if (playbackStartContextTime !== null && playbackStartStoryTime !== null) {
return;
}
const audioContext = getAudioContext();
const currentContextTime = audioContext.currentTime;
const currentStoryTime = useStoryStore.getState().currentTimeMs;
console.log('[StoryPlayback] Setting timing anchors after seek:', {
contextTime: currentContextTime,
storyTime: currentStoryTime,
});
setPlaybackTiming(currentContextTime, currentStoryTime);
// Stop all existing sources and reschedule from new position
stopAllSources();
schedulePlayback(currentStoryTime, playbackItems);
}, [
isPlaying,
playbackItems,
playbackStartContextTime,
playbackStartStoryTime,
getAudioContext,
stopAllSources,
schedulePlayback,
setPlaybackTiming,
]);
}
+1 -1
View File
@@ -12,7 +12,7 @@ interface UseSystemAudioCaptureOptions {
* Uses ScreenCaptureKit on macOS and WASAPI loopback on Windows.
*/
export function useSystemAudioCapture({
maxDurationSeconds = 30,
maxDurationSeconds = 29,
onRecordingComplete,
}: UseSystemAudioCaptureOptions = {}) {
const [isRecording, setIsRecording] = useState(false);
+35
View File
@@ -17,6 +17,41 @@ export function formatAudioDuration(seconds: number): string {
return `${mins}:${secs.toString().padStart(2, '0')}`;
}
/**
* Get audio duration from a File.
* If the file has a recordedDuration property (from recording hooks),
* use that instead of trying to read metadata. This fixes issues on Windows
* where WebM files from MediaRecorder don't have proper duration metadata.
*/
export async function getAudioDuration(
file: File & { recordedDuration?: number },
): Promise<number> {
if (file.recordedDuration !== undefined && Number.isFinite(file.recordedDuration)) {
return file.recordedDuration;
}
return new Promise((resolve, reject) => {
const audio = new Audio();
const url = URL.createObjectURL(file);
audio.addEventListener('loadedmetadata', () => {
URL.revokeObjectURL(url);
if (Number.isFinite(audio.duration) && audio.duration > 0) {
resolve(audio.duration);
} else {
reject(new Error('Audio file has invalid duration metadata'));
}
});
audio.addEventListener('error', () => {
URL.revokeObjectURL(url);
reject(new Error('Failed to load audio file'));
});
audio.src = url;
});
}
/**
* Convert any audio blob to WAV format using Web Audio API.
* This ensures compatibility without requiring ffmpeg on the backend.
+19
View File
@@ -0,0 +1,19 @@
const DEBUG = import.meta.env.DEV;
export const debug = {
log: (...args: unknown[]) => {
if (DEBUG) {
console.log(...args);
}
},
error: (...args: unknown[]) => {
if (DEBUG) {
console.error(...args);
}
},
warn: (...args: unknown[]) => {
if (DEBUG) {
console.warn(...args);
}
},
};
+134
View File
@@ -0,0 +1,134 @@
import { createRootRoute, createRoute, createRouter, Outlet } from '@tanstack/react-router';
import { AppFrame } from '@/components/AppFrame/AppFrame';
import { AudioTab } from '@/components/AudioTab/AudioTab';
import { MainEditor } from '@/components/MainEditor/MainEditor';
import { ModelsTab } from '@/components/ModelsTab/ModelsTab';
import { ServerTab } from '@/components/ServerTab/ServerTab';
import { Sidebar } from '@/components/Sidebar';
import { StoriesTab } from '@/components/StoriesTab/StoriesTab';
import { Toaster } from '@/components/ui/toaster';
import { VoicesTab } from '@/components/VoicesTab/VoicesTab';
import { useModelDownloadToast } from '@/lib/hooks/useModelDownloadToast';
import { MODEL_DISPLAY_NAMES, useRestoreActiveTasks } from '@/lib/hooks/useRestoreActiveTasks';
import { isMacOS } from '@/lib/tauri';
// Root layout component
function RootLayout() {
// Monitor active downloads/generations and show toasts for them
const activeDownloads = useRestoreActiveTasks();
return (
<AppFrame>
<div className="flex flex-1 min-h-0 overflow-hidden">
<Sidebar isMacOS={isMacOS()} />
<main className="flex-1 ml-20 overflow-hidden flex flex-col">
<div className="container mx-auto px-8 max-w-[1800px] h-full overflow-hidden flex flex-col">
<Outlet />
</div>
</main>
</div>
{/* Show download toasts for any active downloads (from anywhere) */}
{activeDownloads.map((download) => {
const displayName = MODEL_DISPLAY_NAMES[download.model_name] || download.model_name;
return (
<DownloadToastRestorer
key={download.model_name}
modelName={download.model_name}
displayName={displayName}
/>
);
})}
<Toaster />
</AppFrame>
);
}
/**
* Component that restores a download toast for a specific model.
*/
function DownloadToastRestorer({
modelName,
displayName,
}: {
modelName: string;
displayName: string;
}) {
// Use the download toast hook to restore the toast
useModelDownloadToast({
modelName,
displayName,
enabled: true,
});
return null;
}
// Root route with layout
const rootRoute = createRootRoute({
component: RootLayout,
});
// Index route (main/generate)
const indexRoute = createRoute({
getParentRoute: () => rootRoute,
path: '/',
component: MainEditor,
});
// Stories route
const storiesRoute = createRoute({
getParentRoute: () => rootRoute,
path: '/stories',
component: StoriesTab,
});
// Voices route
const voicesRoute = createRoute({
getParentRoute: () => rootRoute,
path: '/voices',
component: VoicesTab,
});
// Audio route
const audioRoute = createRoute({
getParentRoute: () => rootRoute,
path: '/audio',
component: AudioTab,
});
// Models route
const modelsRoute = createRoute({
getParentRoute: () => rootRoute,
path: '/models',
component: ModelsTab,
});
// Server route
const serverRoute = createRoute({
getParentRoute: () => rootRoute,
path: '/server',
component: ServerTab,
});
// Route tree
const routeTree = rootRoute.addChildren([
indexRoute,
storiesRoute,
voicesRoute,
audioRoute,
modelsRoute,
serverRoute,
]);
// Create router
export const router = createRouter({ routeTree });
// Register router for type safety
declare module '@tanstack/react-router' {
interface Register {
router: typeof router;
}
}
+42
View File
@@ -0,0 +1,42 @@
import { create } from 'zustand';
import { persist } from 'zustand/middleware';
export interface AudioChannel {
id: string;
name: string;
is_default: boolean;
device_ids: string[];
created_at: string;
}
interface AudioChannelStore {
channels: AudioChannel[];
setChannels: (channels: AudioChannel[]) => void;
addChannel: (channel: AudioChannel) => void;
updateChannel: (id: string, channel: Partial<AudioChannel>) => void;
removeChannel: (id: string) => void;
}
export const useAudioChannelStore = create<AudioChannelStore>()(
persist(
(set) => ({
channels: [],
setChannels: (channels) => set({ channels }),
addChannel: (channel) =>
set((state) => ({
channels: [...state.channels, channel],
})),
updateChannel: (id, updates) =>
set((state) => ({
channels: state.channels.map((ch) => (ch.id === id ? { ...ch, ...updates } : ch)),
})),
removeChannel: (id) =>
set((state) => ({
channels: state.channels.filter((ch) => ch.id !== id),
})),
}),
{
name: 'voicebox-audio-channels',
},
),
);
+29 -2
View File
@@ -3,6 +3,7 @@ import { create } from 'zustand';
interface PlayerState {
audioUrl: string | null;
audioId: string | null;
profileId: string | null;
title: string | null;
isPlaying: boolean;
currentTime: number;
@@ -10,8 +11,11 @@ interface PlayerState {
volume: number;
isLooping: boolean;
shouldRestart: boolean;
shouldAutoPlay: boolean;
onFinish: (() => void) | null;
setAudio: (url: string, id: string, title?: string) => void;
setAudio: (url: string, id: string, profileId: string | null, title?: string) => void;
setAudioWithAutoPlay: (url: string, id: string, profileId: string | null, title?: string) => void;
setIsPlaying: (playing: boolean) => void;
setCurrentTime: (time: number) => void;
setDuration: (duration: number) => void;
@@ -19,12 +23,15 @@ interface PlayerState {
toggleLoop: () => void;
restartCurrentAudio: () => void;
clearRestartFlag: () => void;
clearAutoPlayFlag: () => void;
setOnFinish: (callback: (() => void) | null) => void;
reset: () => void;
}
export const usePlayerStore = create<PlayerState>((set) => ({
audioUrl: null,
audioId: null,
profileId: null,
title: null,
isPlaying: false,
currentTime: 0,
@@ -32,15 +39,30 @@ export const usePlayerStore = create<PlayerState>((set) => ({
volume: 1,
isLooping: false,
shouldRestart: false,
shouldAutoPlay: false,
onFinish: null,
setAudio: (url, id, title) =>
setAudio: (url, id, profileId, title) =>
set({
audioUrl: url,
audioId: id,
profileId: profileId || null,
title: title || null,
currentTime: 0,
isPlaying: false,
shouldRestart: false,
shouldAutoPlay: false,
}),
setAudioWithAutoPlay: (url, id, profileId, title) =>
set({
audioUrl: url,
audioId: id,
profileId: profileId || null,
title: title || null,
currentTime: 0,
isPlaying: false,
shouldRestart: false,
shouldAutoPlay: true,
}),
setIsPlaying: (playing) => set({ isPlaying: playing }),
setCurrentTime: (time) => set({ currentTime: time }),
@@ -49,15 +71,20 @@ export const usePlayerStore = create<PlayerState>((set) => ({
toggleLoop: () => set((state) => ({ isLooping: !state.isLooping })),
restartCurrentAudio: () => set({ shouldRestart: true }),
clearRestartFlag: () => set({ shouldRestart: false }),
clearAutoPlayFlag: () => set({ shouldAutoPlay: false }),
setOnFinish: (callback) => set({ onFinish: callback }),
reset: () =>
set({
audioUrl: null,
audioId: null,
profileId: null,
title: null,
isPlaying: false,
currentTime: 0,
duration: 0,
isLooping: false,
shouldRestart: false,
shouldAutoPlay: false,
onFinish: null,
}),
}));
+145
View File
@@ -0,0 +1,145 @@
import { create } from 'zustand';
import type { StoryItemDetail } from '@/lib/api/types';
interface StoryPlaybackState {
// Selection
selectedStoryId: string | null;
setSelectedStoryId: (id: string | null) => void;
selectedClipId: string | null;
setSelectedClipId: (id: string | null) => void;
// Track editor UI state
trackEditorHeight: number;
setTrackEditorHeight: (height: number) => void;
// Playback state
isPlaying: boolean;
currentTimeMs: number;
totalDurationMs: number;
playbackStoryId: string | null;
playbackItems: StoryItemDetail[] | null;
// Web Audio API timing (null when not playing)
playbackStartContextTime: number | null; // AudioContext.currentTime when playback started
playbackStartStoryTime: number | null; // Story time (ms) when playback started
// Actions
play: (storyId: string, items: StoryItemDetail[]) => void;
pause: () => void;
stop: () => void;
seek: (timeMs: number) => void;
setPlaybackTiming: (contextTime: number, storyTime: number) => void; // Set timing anchors for Web Audio API
setActiveStory: (storyId: string, items: StoryItemDetail[], totalDurationMs: number) => void; // Activate story for seeking without playing
}
const DEFAULT_TRACK_EDITOR_HEIGHT = 250;
export const useStoryStore = create<StoryPlaybackState>((set, get) => ({
// Selection
selectedStoryId: null,
setSelectedStoryId: (id) => set({ selectedStoryId: id }),
selectedClipId: null,
setSelectedClipId: (id) => set({ selectedClipId: id }),
// Track editor UI state
trackEditorHeight: DEFAULT_TRACK_EDITOR_HEIGHT,
setTrackEditorHeight: (height) => set({ trackEditorHeight: height }),
// Playback state
isPlaying: false,
currentTimeMs: 0,
totalDurationMs: 0,
playbackStoryId: null,
playbackItems: null,
playbackStartContextTime: null,
playbackStartStoryTime: null,
// Actions
play: (storyId, items) => {
// Calculate total duration from items
const maxEndTimeMs = Math.max(
...items.map((item) => item.start_time_ms + item.duration * 1000),
0,
);
// Find the minimum start time (first item)
const minStartTimeMs = Math.min(...items.map((item) => item.start_time_ms), 0);
// If resuming the same story, keep position; otherwise start at first item
const currentState = get();
const shouldResume = currentState.playbackStoryId === storyId && currentState.currentTimeMs > 0;
const startTimeMs = shouldResume ? currentState.currentTimeMs : minStartTimeMs;
console.log('[StoryStore] Play called:', {
storyId,
itemCount: items.length,
items: items.map((i) => ({
id: i.generation_id,
start: i.start_time_ms,
duration: i.duration,
})),
maxEndTimeMs,
minStartTimeMs,
startTimeMs,
shouldResume,
});
set({
isPlaying: true,
playbackStoryId: storyId,
playbackItems: items,
totalDurationMs: maxEndTimeMs,
currentTimeMs: startTimeMs,
});
},
pause: () => {
set({
isPlaying: false,
// Keep timing anchors so we can resume from same position
});
},
stop: () => {
set({
isPlaying: false,
currentTimeMs: 0,
playbackStoryId: null,
playbackItems: null,
totalDurationMs: 0,
playbackStartContextTime: null,
playbackStartStoryTime: null,
});
},
seek: (timeMs) => {
const state = get();
const clampedTime = Math.max(0, Math.min(timeMs, state.totalDurationMs));
set({
currentTimeMs: clampedTime,
// Reset timing anchors - will be set by hook when playback resumes
playbackStartContextTime: null,
playbackStartStoryTime: null,
});
},
setPlaybackTiming: (contextTime, storyTime) => {
set({
playbackStartContextTime: contextTime,
playbackStartStoryTime: storyTime,
});
},
setActiveStory: (storyId, items, totalDurationMs) => {
const currentState = get();
// Only update if switching to a different story
if (currentState.playbackStoryId !== storyId) {
set({
playbackStoryId: storyId,
playbackItems: items,
totalDurationMs,
currentTimeMs: 0,
isPlaying: false,
});
}
},
}));
+2
View File
@@ -1 +1,3 @@
# Backend package
__version__ = "0.1.8"
+263
View File
@@ -0,0 +1,263 @@
"""
Audio channel management module.
"""
from typing import List, Optional
from datetime import datetime
import uuid
from sqlalchemy.orm import Session
from .models import (
AudioChannelCreate,
AudioChannelUpdate,
AudioChannelResponse,
ChannelVoiceAssignment,
ProfileChannelAssignment,
)
from .database import (
AudioChannel as DBAudioChannel,
ChannelDeviceMapping as DBChannelDeviceMapping,
ProfileChannelMapping as DBProfileChannelMapping,
VoiceProfile as DBVoiceProfile,
)
async def list_channels(db: Session) -> List[AudioChannelResponse]:
"""List all audio channels."""
channels = db.query(DBAudioChannel).all()
result = []
for channel in channels:
# Get device IDs for this channel
device_mappings = db.query(DBChannelDeviceMapping).filter_by(
channel_id=channel.id
).all()
device_ids = [m.device_id for m in device_mappings]
result.append(AudioChannelResponse(
id=channel.id,
name=channel.name,
is_default=channel.is_default,
device_ids=device_ids,
created_at=channel.created_at,
))
return result
async def get_channel(channel_id: str, db: Session) -> Optional[AudioChannelResponse]:
"""Get a channel by ID."""
channel = db.query(DBAudioChannel).filter_by(id=channel_id).first()
if not channel:
return None
# Get device IDs
device_mappings = db.query(DBChannelDeviceMapping).filter_by(
channel_id=channel.id
).all()
device_ids = [m.device_id for m in device_mappings]
return AudioChannelResponse(
id=channel.id,
name=channel.name,
is_default=channel.is_default,
device_ids=device_ids,
created_at=channel.created_at,
)
async def create_channel(
data: AudioChannelCreate,
db: Session,
) -> AudioChannelResponse:
"""Create a new audio channel."""
# Check if name already exists
existing = db.query(DBAudioChannel).filter_by(name=data.name).first()
if existing:
raise ValueError(f"Channel with name '{data.name}' already exists")
# Create channel
channel = DBAudioChannel(
id=str(uuid.uuid4()),
name=data.name,
is_default=False,
created_at=datetime.utcnow(),
)
db.add(channel)
db.flush()
# Add device mappings
for device_id in data.device_ids:
mapping = DBChannelDeviceMapping(
id=str(uuid.uuid4()),
channel_id=channel.id,
device_id=device_id,
)
db.add(mapping)
db.commit()
db.refresh(channel)
return AudioChannelResponse(
id=channel.id,
name=channel.name,
is_default=channel.is_default,
device_ids=data.device_ids,
created_at=channel.created_at,
)
async def update_channel(
channel_id: str,
data: AudioChannelUpdate,
db: Session,
) -> Optional[AudioChannelResponse]:
"""Update an audio channel."""
channel = db.query(DBAudioChannel).filter_by(id=channel_id).first()
if not channel:
return None
if channel.is_default:
raise ValueError("Cannot modify the default channel")
# Update name if provided
if data.name is not None:
# Check if name already exists (excluding current channel)
existing = db.query(DBAudioChannel).filter(
DBAudioChannel.name == data.name,
DBAudioChannel.id != channel_id
).first()
if existing:
raise ValueError(f"Channel with name '{data.name}' already exists")
channel.name = data.name
# Update device mappings if provided
if data.device_ids is not None:
# Delete existing mappings
db.query(DBChannelDeviceMapping).filter_by(channel_id=channel_id).delete()
# Add new mappings
for device_id in data.device_ids:
mapping = DBChannelDeviceMapping(
id=str(uuid.uuid4()),
channel_id=channel.id,
device_id=device_id,
)
db.add(mapping)
db.commit()
db.refresh(channel)
# Get updated device IDs
device_mappings = db.query(DBChannelDeviceMapping).filter_by(
channel_id=channel.id
).all()
device_ids = [m.device_id for m in device_mappings]
return AudioChannelResponse(
id=channel.id,
name=channel.name,
is_default=channel.is_default,
device_ids=device_ids,
created_at=channel.created_at,
)
async def delete_channel(channel_id: str, db: Session) -> bool:
"""Delete an audio channel."""
channel = db.query(DBAudioChannel).filter_by(id=channel_id).first()
if not channel:
return False
if channel.is_default:
raise ValueError("Cannot delete the default channel")
# Delete device mappings
db.query(DBChannelDeviceMapping).filter_by(channel_id=channel_id).delete()
# Delete profile-channel mappings
db.query(DBProfileChannelMapping).filter_by(channel_id=channel_id).delete()
# Delete channel
db.delete(channel)
db.commit()
return True
async def get_channel_voices(channel_id: str, db: Session) -> List[str]:
"""Get list of profile IDs assigned to a channel."""
mappings = db.query(DBProfileChannelMapping).filter_by(
channel_id=channel_id
).all()
return [m.profile_id for m in mappings]
async def set_channel_voices(
channel_id: str,
data: ChannelVoiceAssignment,
db: Session,
) -> None:
"""Set which voices are assigned to a channel."""
# Verify channel exists
channel = db.query(DBAudioChannel).filter_by(id=channel_id).first()
if not channel:
raise ValueError(f"Channel {channel_id} not found")
# Verify all profiles exist
for profile_id in data.profile_ids:
profile = db.query(DBVoiceProfile).filter_by(id=profile_id).first()
if not profile:
raise ValueError(f"Profile {profile_id} not found")
# Delete existing mappings for this channel
db.query(DBProfileChannelMapping).filter_by(channel_id=channel_id).delete()
# Add new mappings
for profile_id in data.profile_ids:
mapping = DBProfileChannelMapping(
profile_id=profile_id,
channel_id=channel_id,
)
db.add(mapping)
db.commit()
async def get_profile_channels(profile_id: str, db: Session) -> List[str]:
"""Get list of channel IDs assigned to a profile."""
mappings = db.query(DBProfileChannelMapping).filter_by(
profile_id=profile_id
).all()
return [m.channel_id for m in mappings]
async def set_profile_channels(
profile_id: str,
data: ProfileChannelAssignment,
db: Session,
) -> None:
"""Set which channels a profile is assigned to."""
# Verify profile exists
profile = db.query(DBVoiceProfile).filter_by(id=profile_id).first()
if not profile:
raise ValueError(f"Profile {profile_id} not found")
# Verify all channels exist
for channel_id in data.channel_ids:
channel = db.query(DBAudioChannel).filter_by(id=channel_id).first()
if not channel:
raise ValueError(f"Channel {channel_id} not found")
# Delete existing mappings for this profile
db.query(DBProfileChannelMapping).filter_by(profile_id=profile_id).delete()
# Add new mappings
for channel_id in data.channel_ids:
mapping = DBProfileChannelMapping(
profile_id=profile_id,
channel_id=channel_id,
)
db.add(mapping)
db.commit()
+195 -1
View File
@@ -2,7 +2,7 @@
SQLite database ORM using SQLAlchemy.
"""
from sqlalchemy import create_engine, Column, String, Integer, Float, DateTime, Text, ForeignKey
from sqlalchemy import create_engine, Column, String, Integer, Float, DateTime, Text, ForeignKey, Boolean
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import sessionmaker, Session
from datetime import datetime
@@ -51,6 +51,31 @@ class Generation(Base):
created_at = Column(DateTime, default=datetime.utcnow)
class Story(Base):
"""Story database model."""
__tablename__ = "stories"
id = Column(String, primary_key=True, default=lambda: str(uuid.uuid4()))
name = Column(String, nullable=False)
description = Column(Text)
created_at = Column(DateTime, default=datetime.utcnow)
updated_at = Column(DateTime, default=datetime.utcnow, onupdate=datetime.utcnow)
class StoryItem(Base):
"""Story item database model (links generations to stories)."""
__tablename__ = "story_items"
id = Column(String, primary_key=True, default=lambda: str(uuid.uuid4()))
story_id = Column(String, ForeignKey("stories.id"), nullable=False)
generation_id = Column(String, ForeignKey("generations.id"), nullable=False)
start_time_ms = Column(Integer, nullable=False, default=0) # Milliseconds from story start
track = Column(Integer, nullable=False, default=0) # Track number (0 = main track)
trim_start_ms = Column(Integer, nullable=False, default=0) # Milliseconds trimmed from start
trim_end_ms = Column(Integer, nullable=False, default=0) # Milliseconds trimmed from end
created_at = Column(DateTime, default=datetime.utcnow)
class Project(Base):
"""Audio studio project database model."""
__tablename__ = "projects"
@@ -62,6 +87,33 @@ class Project(Base):
updated_at = Column(DateTime, default=datetime.utcnow, onupdate=datetime.utcnow)
class AudioChannel(Base):
"""Audio channel (bus) database model."""
__tablename__ = "audio_channels"
id = Column(String, primary_key=True, default=lambda: str(uuid.uuid4()))
name = Column(String, nullable=False)
is_default = Column(Boolean, default=False)
created_at = Column(DateTime, default=datetime.utcnow)
class ChannelDeviceMapping(Base):
"""Mapping between channels and OS audio devices."""
__tablename__ = "channel_device_mappings"
id = Column(String, primary_key=True, default=lambda: str(uuid.uuid4()))
channel_id = Column(String, ForeignKey("audio_channels.id"), nullable=False)
device_id = Column(String, nullable=False) # OS device identifier
class ProfileChannelMapping(Base):
"""Mapping between voice profiles and audio channels (many-to-many)."""
__tablename__ = "profile_channel_mappings"
profile_id = Column(String, ForeignKey("profiles.id"), primary_key=True)
channel_id = Column(String, ForeignKey("audio_channels.id"), primary_key=True)
# Database setup will be initialized in init_db()
engine = None
SessionLocal = None
@@ -81,7 +133,149 @@ def init_db():
)
SessionLocal = sessionmaker(autocommit=False, autoflush=False, bind=engine)
# Run migrations before creating tables
_run_migrations(engine)
Base.metadata.create_all(bind=engine)
# Create default channel if it doesn't exist
db = SessionLocal()
try:
default_channel = db.query(AudioChannel).filter(AudioChannel.is_default == True).first()
if not default_channel:
default_channel = AudioChannel(
id=str(uuid.uuid4()),
name="Default",
is_default=True
)
db.add(default_channel)
# Assign all existing profiles to default channel
profiles = db.query(VoiceProfile).all()
for profile in profiles:
mapping = ProfileChannelMapping(
profile_id=profile.id,
channel_id=default_channel.id
)
db.add(mapping)
db.commit()
finally:
db.close()
def _run_migrations(engine):
"""Run database migrations."""
from sqlalchemy import inspect, text
inspector = inspect(engine)
# Check if story_items table exists
if 'story_items' not in inspector.get_table_names():
return # Table doesn't exist yet, will be created fresh
# Get columns in story_items table
columns = {col['name'] for col in inspector.get_columns('story_items')}
# Migration: Remove position column and ensure start_time_ms exists
# SQLite doesn't support DROP COLUMN easily, so we recreate the table
if 'position' in columns:
print("Migrating story_items: removing position column, using start_time_ms")
with engine.connect() as conn:
# Check if start_time_ms already exists
has_start_time = 'start_time_ms' in columns
if not has_start_time:
# First, add the new column temporarily
conn.execute(text("ALTER TABLE story_items ADD COLUMN start_time_ms INTEGER DEFAULT 0"))
# Calculate timecodes from position ordering
result = conn.execute(text("""
SELECT si.id, si.story_id, si.position, g.duration
FROM story_items si
JOIN generations g ON si.generation_id = g.id
ORDER BY si.story_id, si.position
"""))
rows = result.fetchall()
current_story_id = None
current_time_ms = 0
for row in rows:
item_id, story_id, position, duration = row
if story_id != current_story_id:
current_story_id = story_id
current_time_ms = 0
conn.execute(
text("UPDATE story_items SET start_time_ms = :time WHERE id = :id"),
{"time": current_time_ms, "id": item_id}
)
current_time_ms += int(duration * 1000) + 200
conn.commit()
# Now recreate the table without the position column
# 1. Create new table
conn.execute(text("""
CREATE TABLE story_items_new (
id VARCHAR PRIMARY KEY,
story_id VARCHAR NOT NULL,
generation_id VARCHAR NOT NULL,
start_time_ms INTEGER NOT NULL DEFAULT 0,
created_at DATETIME,
FOREIGN KEY (story_id) REFERENCES stories(id),
FOREIGN KEY (generation_id) REFERENCES generations(id)
)
"""))
# 2. Copy data
conn.execute(text("""
INSERT INTO story_items_new (id, story_id, generation_id, start_time_ms, created_at)
SELECT id, story_id, generation_id, start_time_ms, created_at FROM story_items
"""))
# 3. Drop old table
conn.execute(text("DROP TABLE story_items"))
# 4. Rename new table
conn.execute(text("ALTER TABLE story_items_new RENAME TO story_items"))
conn.commit()
print("Migrated story_items table to use start_time_ms (removed position column)")
# Migration: Add track column if it doesn't exist
# Re-check columns after potential position migration
columns = {col['name'] for col in inspector.get_columns('story_items')}
if 'track' not in columns:
print("Migrating story_items: adding track column")
with engine.connect() as conn:
conn.execute(text("ALTER TABLE story_items ADD COLUMN track INTEGER NOT NULL DEFAULT 0"))
conn.commit()
print("Added track column to story_items")
# Migration: Add trim columns if they don't exist
# Re-check columns after potential track migration
columns = {col['name'] for col in inspector.get_columns('story_items')}
if 'trim_start_ms' not in columns:
print("Migrating story_items: adding trim_start_ms column")
with engine.connect() as conn:
conn.execute(text("ALTER TABLE story_items ADD COLUMN trim_start_ms INTEGER NOT NULL DEFAULT 0"))
conn.commit()
print("Added trim_start_ms column to story_items")
columns = {col['name'] for col in inspector.get_columns('story_items')}
if 'trim_end_ms' not in columns:
print("Migrating story_items: adding trim_end_ms column")
with engine.connect() as conn:
conn.execute(text("ALTER TABLE story_items ADD COLUMN trim_end_ms INTEGER NOT NULL DEFAULT 0"))
conn.commit()
print("Added trim_end_ms column to story_items")
def get_db():
+461 -32
View File
@@ -11,6 +11,7 @@ from fastapi.staticfiles import StaticFiles
from sqlalchemy.orm import Session
from typing import List, Optional
from datetime import datetime
import asyncio
import uvicorn
import argparse
import torch
@@ -18,8 +19,11 @@ import tempfile
import io
from pathlib import Path
import uuid
import asyncio
import signal
import os
from . import database, models, profiles, history, tts, transcribe, config, export_import
from . import database, models, profiles, history, tts, transcribe, config, export_import, channels, stories, __version__
from .database import get_db, Generation as DBGeneration, VoiceProfile as DBVoiceProfile
from .utils.progress import get_progress_manager
from .utils.tasks import get_task_manager
@@ -27,7 +31,7 @@ from .utils.tasks import get_task_manager
app = FastAPI(
title="voicebox API",
description="Production-quality Qwen3-TTS voice cloning API",
version="0.1.0",
version=__version__,
)
# CORS middleware
@@ -47,21 +51,42 @@ app.add_middleware(
@app.get("/")
async def root():
"""Root endpoint."""
return {"message": "voicebox API", "version": "0.1.2"}
return {"message": "voicebox API", "version": __version__}
@app.post("/shutdown")
async def shutdown():
"""Gracefully shutdown the server."""
async def shutdown_async():
await asyncio.sleep(0.1) # Give response time to send
os.kill(os.getpid(), signal.SIGTERM)
asyncio.create_task(shutdown_async())
return {"message": "Shutting down..."}
@app.get("/health", response_model=models.HealthResponse)
async def health():
"""Health check endpoint."""
from huggingface_hub import hf_hub_download
from huggingface_hub import hf_hub_download, constants as hf_constants
from pathlib import Path
import os
tts_model = tts.get_tts_model()
gpu_available = torch.cuda.is_available()
# Check for GPU availability (CUDA or MPS)
has_cuda = torch.cuda.is_available()
has_mps = hasattr(torch.backends, 'mps') and torch.backends.mps.is_available()
gpu_available = has_cuda or has_mps
gpu_type = None
if has_cuda:
gpu_type = f"CUDA ({torch.cuda.get_device_name(0)})"
elif has_mps:
gpu_type = "MPS (Apple Silicon)"
vram_used = None
if gpu_available:
if has_cuda:
vram_used = torch.cuda.memory_allocated() / 1024 / 1024 # MB
# Check if model is loaded - use the same logic as model status endpoint
@@ -97,8 +122,8 @@ async def health():
model_downloaded = True
break
except (ImportError, Exception):
# Method 2: Check cache directory
cache_dir = os.path.expanduser("~/.cache/huggingface/hub")
# Method 2: Check cache directory (using HuggingFace's OS-specific cache location)
cache_dir = hf_constants.HF_HUB_CACHE
repo_cache = Path(cache_dir) / "models--" + default_model_id.replace("/", "--")
if repo_cache.exists():
has_model_files = (
@@ -117,6 +142,7 @@ async def health():
model_downloaded=model_downloaded,
model_size=model_size,
gpu_available=gpu_available,
gpu_type=gpu_type,
vram_used_mb=vram_used,
)
@@ -292,6 +318,125 @@ async def export_profile(
raise HTTPException(status_code=500, detail=str(e))
# ============================================
# AUDIO CHANNEL ENDPOINTS
# ============================================
@app.get("/channels", response_model=List[models.AudioChannelResponse])
async def list_channels(db: Session = Depends(get_db)):
"""List all audio channels."""
return await channels.list_channels(db)
@app.post("/channels", response_model=models.AudioChannelResponse)
async def create_channel(
data: models.AudioChannelCreate,
db: Session = Depends(get_db),
):
"""Create a new audio channel."""
try:
return await channels.create_channel(data, db)
except ValueError as e:
raise HTTPException(status_code=400, detail=str(e))
@app.get("/channels/{channel_id}", response_model=models.AudioChannelResponse)
async def get_channel(
channel_id: str,
db: Session = Depends(get_db),
):
"""Get an audio channel by ID."""
channel = await channels.get_channel(channel_id, db)
if not channel:
raise HTTPException(status_code=404, detail="Channel not found")
return channel
@app.put("/channels/{channel_id}", response_model=models.AudioChannelResponse)
async def update_channel(
channel_id: str,
data: models.AudioChannelUpdate,
db: Session = Depends(get_db),
):
"""Update an audio channel."""
try:
channel = await channels.update_channel(channel_id, data, db)
if not channel:
raise HTTPException(status_code=404, detail="Channel not found")
return channel
except ValueError as e:
raise HTTPException(status_code=400, detail=str(e))
@app.delete("/channels/{channel_id}")
async def delete_channel(
channel_id: str,
db: Session = Depends(get_db),
):
"""Delete an audio channel."""
try:
success = await channels.delete_channel(channel_id, db)
if not success:
raise HTTPException(status_code=404, detail="Channel not found")
return {"message": "Channel deleted successfully"}
except ValueError as e:
raise HTTPException(status_code=400, detail=str(e))
@app.get("/channels/{channel_id}/voices")
async def get_channel_voices(
channel_id: str,
db: Session = Depends(get_db),
):
"""Get list of profile IDs assigned to a channel."""
try:
profile_ids = await channels.get_channel_voices(channel_id, db)
return {"profile_ids": profile_ids}
except ValueError as e:
raise HTTPException(status_code=400, detail=str(e))
@app.put("/channels/{channel_id}/voices")
async def set_channel_voices(
channel_id: str,
data: models.ChannelVoiceAssignment,
db: Session = Depends(get_db),
):
"""Set which voices are assigned to a channel."""
try:
await channels.set_channel_voices(channel_id, data, db)
return {"message": "Channel voices updated successfully"}
except ValueError as e:
raise HTTPException(status_code=400, detail=str(e))
@app.get("/profiles/{profile_id}/channels")
async def get_profile_channels(
profile_id: str,
db: Session = Depends(get_db),
):
"""Get list of channel IDs assigned to a profile."""
try:
channel_ids = await channels.get_profile_channels(profile_id, db)
return {"channel_ids": channel_ids}
except ValueError as e:
raise HTTPException(status_code=400, detail=str(e))
@app.put("/profiles/{profile_id}/channels")
async def set_profile_channels(
profile_id: str,
data: models.ProfileChannelAssignment,
db: Session = Depends(get_db),
):
"""Set which channels a profile is assigned to."""
try:
await channels.set_profile_channels(profile_id, data, db)
return {"message": "Profile channels updated successfully"}
except ValueError as e:
raise HTTPException(status_code=400, detail=str(e))
# ============================================
# GENERATION ENDPOINTS
# ============================================
@@ -328,6 +473,36 @@ async def generate_speech(
tts_model = tts.get_tts_model()
# Load the requested model size if different from current (async to not block)
model_size = data.model_size or "1.7B"
# Check if model needs to be downloaded first
model_path = tts_model._get_model_path(model_size)
if model_path.startswith("Qwen/"):
# Model not cached - check if it exists remotely or needs download
from huggingface_hub import constants as hf_constants
repo_cache = Path(hf_constants.HF_HUB_CACHE) / ("models--" + model_path.replace("/", "--"))
if not repo_cache.exists():
# Start download in background
model_name = f"qwen-tts-{model_size}"
async def download_model_background():
try:
await tts_model.load_model_async(model_size)
except Exception as e:
task_manager.error_download(model_name, str(e))
task_manager.start_download(model_name)
asyncio.create_task(download_model_background())
# Return 202 Accepted with download info
raise HTTPException(
status_code=202,
detail={
"message": f"Model {model_size} is being downloaded. Please wait and try again.",
"model_name": model_name,
"downloading": True
}
)
await tts_model.load_model_async(model_size)
audio, sample_rate = await tts_model.generate(
data.text,
@@ -561,6 +736,37 @@ async def transcribe_audio(
# Transcribe
whisper_model = transcribe.get_whisper_model()
# Check if Whisper model is downloaded (uses default size "base")
model_size = whisper_model.model_size
model_name = f"openai/whisper-{model_size}"
# Check if model is cached
from huggingface_hub import constants as hf_constants
repo_cache = Path(hf_constants.HF_HUB_CACHE) / ("models--" + model_name.replace("/", "--"))
if not repo_cache.exists():
# Start download in background
progress_model_name = f"whisper-{model_size}"
async def download_whisper_background():
try:
await whisper_model.load_model_async(model_size)
except Exception as e:
get_task_manager().error_download(progress_model_name, str(e))
get_task_manager().start_download(progress_model_name)
asyncio.create_task(download_whisper_background())
# Return 202 Accepted
raise HTTPException(
status_code=202,
detail={
"message": f"Whisper model {model_size} is being downloaded. Please wait and try again.",
"model_name": progress_model_name,
"downloading": True
}
)
text = await whisper_model.transcribe(tmp_path, language)
return models.TranscriptionResponse(
@@ -575,6 +781,209 @@ async def transcribe_audio(
Path(tmp_path).unlink(missing_ok=True)
# ============================================
# STORY ENDPOINTS
# ============================================
@app.get("/stories", response_model=List[models.StoryResponse])
async def list_stories(db: Session = Depends(get_db)):
"""List all stories."""
return await stories.list_stories(db)
@app.post("/stories", response_model=models.StoryResponse)
async def create_story(
data: models.StoryCreate,
db: Session = Depends(get_db),
):
"""Create a new story."""
try:
return await stories.create_story(data, db)
except Exception as e:
raise HTTPException(status_code=400, detail=str(e))
@app.get("/stories/{story_id}", response_model=models.StoryDetailResponse)
async def get_story(
story_id: str,
db: Session = Depends(get_db),
):
"""Get a story with all its items."""
story = await stories.get_story(story_id, db)
if not story:
raise HTTPException(status_code=404, detail="Story not found")
return story
@app.put("/stories/{story_id}", response_model=models.StoryResponse)
async def update_story(
story_id: str,
data: models.StoryCreate,
db: Session = Depends(get_db),
):
"""Update a story."""
story = await stories.update_story(story_id, data, db)
if not story:
raise HTTPException(status_code=404, detail="Story not found")
return story
@app.delete("/stories/{story_id}")
async def delete_story(
story_id: str,
db: Session = Depends(get_db),
):
"""Delete a story."""
success = await stories.delete_story(story_id, db)
if not success:
raise HTTPException(status_code=404, detail="Story not found")
return {"message": "Story deleted successfully"}
@app.post("/stories/{story_id}/items", response_model=models.StoryItemDetail)
async def add_story_item(
story_id: str,
data: models.StoryItemCreate,
db: Session = Depends(get_db),
):
"""Add a generation to a story."""
item = await stories.add_item_to_story(story_id, data, db)
if not item:
raise HTTPException(status_code=404, detail="Story or generation not found")
return item
@app.delete("/stories/{story_id}/items/{item_id}")
async def remove_story_item(
story_id: str,
item_id: str,
db: Session = Depends(get_db),
):
"""Remove a story item from a story."""
success = await stories.remove_item_from_story(story_id, item_id, db)
if not success:
raise HTTPException(status_code=404, detail="Story item not found")
return {"message": "Item removed successfully"}
@app.put("/stories/{story_id}/items/times")
async def update_story_item_times(
story_id: str,
data: models.StoryItemBatchUpdate,
db: Session = Depends(get_db),
):
"""Update story item timecodes."""
success = await stories.update_story_item_times(story_id, data, db)
if not success:
raise HTTPException(status_code=400, detail="Invalid timecode update request")
return {"message": "Item timecodes updated successfully"}
@app.put("/stories/{story_id}/items/reorder", response_model=List[models.StoryItemDetail])
async def reorder_story_items(
story_id: str,
data: models.StoryItemReorder,
db: Session = Depends(get_db),
):
"""Reorder story items and recalculate timecodes."""
items = await stories.reorder_story_items(story_id, data.generation_ids, db)
if items is None:
raise HTTPException(status_code=400, detail="Invalid reorder request - ensure all generation IDs belong to this story")
return items
@app.put("/stories/{story_id}/items/{item_id}/move", response_model=models.StoryItemDetail)
async def move_story_item(
story_id: str,
item_id: str,
data: models.StoryItemMove,
db: Session = Depends(get_db),
):
"""Move a story item (update position and/or track)."""
item = await stories.move_story_item(story_id, item_id, data, db)
if item is None:
raise HTTPException(status_code=404, detail="Story item not found")
return item
@app.put("/stories/{story_id}/items/{item_id}/trim", response_model=models.StoryItemDetail)
async def trim_story_item(
story_id: str,
item_id: str,
data: models.StoryItemTrim,
db: Session = Depends(get_db),
):
"""Trim a story item (update trim_start_ms and trim_end_ms)."""
item = await stories.trim_story_item(story_id, item_id, data, db)
if item is None:
raise HTTPException(status_code=404, detail="Story item not found or invalid trim values")
return item
@app.post("/stories/{story_id}/items/{item_id}/split", response_model=List[models.StoryItemDetail])
async def split_story_item(
story_id: str,
item_id: str,
data: models.StoryItemSplit,
db: Session = Depends(get_db),
):
"""Split a story item at a given time, creating two clips."""
items = await stories.split_story_item(story_id, item_id, data, db)
if items is None:
raise HTTPException(status_code=404, detail="Story item not found or invalid split point")
return items
@app.post("/stories/{story_id}/items/{item_id}/duplicate", response_model=models.StoryItemDetail)
async def duplicate_story_item(
story_id: str,
item_id: str,
db: Session = Depends(get_db),
):
"""Duplicate a story item, creating a copy with all properties."""
item = await stories.duplicate_story_item(story_id, item_id, db)
if item is None:
raise HTTPException(status_code=404, detail="Story item not found")
return item
@app.get("/stories/{story_id}/export-audio")
async def export_story_audio(
story_id: str,
db: Session = Depends(get_db),
):
"""Export story as single mixed audio file with timecode-based mixing."""
try:
# Get story to create filename
story = db.query(database.Story).filter_by(id=story_id).first()
if not story:
raise HTTPException(status_code=404, detail="Story not found")
# Export audio
audio_bytes = await stories.export_story_audio(story_id, db)
if not audio_bytes:
raise HTTPException(status_code=400, detail="Story has no audio items")
# Create safe filename
safe_name = "".join(c for c in story.name if c.isalnum() or c in (' ', '-', '_')).strip()
if not safe_name:
safe_name = "story"
filename = f"{safe_name}.wav"
# Return as streaming response
return StreamingResponse(
io.BytesIO(audio_bytes),
media_type="audio/wav",
headers={
"Content-Disposition": f'attachment; filename="{filename}"'
}
)
except HTTPException:
raise
except Exception as e:
raise HTTPException(status_code=500, detail=str(e))
# ============================================
# FILE SERVING
# ============================================
@@ -668,7 +1077,7 @@ async def get_model_progress(model_name: str):
@app.get("/models/status", response_model=models.ModelStatusListResponse)
async def get_model_status():
"""Get status of all available models."""
from huggingface_hub import hf_hub_download
from huggingface_hub import hf_hub_download, constants as hf_constants
from pathlib import Path
import os
@@ -771,10 +1180,10 @@ async def get_model_status():
pass
break
# Method 2: Fallback to checking cache directory directly
# Method 2: Fallback to checking cache directory directly (using HuggingFace's OS-specific cache location)
if not downloaded:
try:
cache_dir = os.path.expanduser("~/.cache/huggingface/hub")
cache_dir = hf_constants.HF_HUB_CACHE
repo_cache = Path(cache_dir) / "models--" + config["hf_repo_id"].replace("/", "--")
if repo_cache.exists():
@@ -883,22 +1292,22 @@ async def trigger_model_download(request: models.ModelDownloadRequest):
config = model_configs[request.model_name]
try:
# Start tracking download
task_manager.start_download(request.model_name)
# Trigger download by loading the model (which will download if not cached)
# Run in background to avoid blocking
await asyncio.to_thread(config["load_func"])
# Mark download as complete
task_manager.complete_download(request.model_name)
return {"message": f"Model {request.model_name} download started"}
except Exception as e:
# Mark download as failed
task_manager.error_download(request.model_name, str(e))
raise HTTPException(status_code=500, detail=str(e))
async def download_in_background():
"""Download model in background without blocking the HTTP request."""
try:
await asyncio.to_thread(config["load_func"])
task_manager.complete_download(request.model_name)
except Exception as e:
task_manager.error_download(request.model_name, str(e))
# Start tracking download
task_manager.start_download(request.model_name)
# Start download in background task (don't await)
asyncio.create_task(download_in_background())
# Return immediately - frontend should poll progress endpoint
return {"message": f"Model {request.model_name} download started"}
@app.delete("/models/{model_name}")
@@ -906,6 +1315,7 @@ async def delete_model(model_name: str):
"""Delete a downloaded model from the HuggingFace cache."""
import shutil
import os
from huggingface_hub import constants as hf_constants
# Map model names to HuggingFace repo IDs
model_configs = {
@@ -958,8 +1368,8 @@ async def delete_model(model_name: str):
if whisper_model.is_loaded() and whisper_model.model_size == config["model_size"]:
transcribe.unload_whisper_model()
# Find and delete the cache directory
cache_dir = os.path.expanduser("~/.cache/huggingface/hub")
# Find and delete the cache directory (using HuggingFace's OS-specific cache location)
cache_dir = hf_constants.HF_HUB_CACHE
repo_cache_dir = Path(cache_dir) / ("models--" + hf_repo_id.replace("/", "--"))
# Check if the cache directory exists
@@ -1053,13 +1463,32 @@ async def get_active_tasks():
# STARTUP & SHUTDOWN
# ============================================
def _get_gpu_status() -> str:
"""Get GPU availability status."""
if torch.cuda.is_available():
return f"CUDA ({torch.cuda.get_device_name(0)})"
elif hasattr(torch.backends, 'mps') and torch.backends.mps.is_available():
return "MPS (Apple Silicon)"
return "None (CPU only)"
@app.on_event("startup")
async def startup_event():
"""Run on application startup."""
print("voicebox API starting up...")
database.init_db()
print(f"Database initialized at {database._db_path}")
print(f"GPU available: {torch.cuda.is_available()}")
print(f"GPU available: {_get_gpu_status()}")
# Ensure HuggingFace cache directory exists
try:
from huggingface_hub import constants as hf_constants
cache_dir = Path(hf_constants.HF_HUB_CACHE)
cache_dir.mkdir(parents=True, exist_ok=True)
print(f"HuggingFace cache directory: {cache_dir}")
except Exception as e:
print(f"Warning: Could not create HuggingFace cache directory: {e}")
print("Model downloads may fail. Please ensure the directory exists and has write permissions.")
@app.on_event("shutdown")
+132
View File
@@ -118,6 +118,7 @@ class HealthResponse(BaseModel):
model_downloaded: Optional[bool] = None # Whether model is cached/downloaded
model_size: Optional[str] = None # Current model size if loaded
gpu_available: bool
gpu_type: Optional[str] = None # GPU type (CUDA, MPS, or None)
vram_used_mb: Optional[float] = None
@@ -159,3 +160,134 @@ class ActiveTasksResponse(BaseModel):
"""Response model for active tasks."""
downloads: List[ActiveDownloadTask]
generations: List[ActiveGenerationTask]
class AudioChannelCreate(BaseModel):
"""Request model for creating an audio channel."""
name: str = Field(..., min_length=1, max_length=100)
device_ids: List[str] = Field(default_factory=list)
class AudioChannelUpdate(BaseModel):
"""Request model for updating an audio channel."""
name: Optional[str] = Field(None, min_length=1, max_length=100)
device_ids: Optional[List[str]] = None
class AudioChannelResponse(BaseModel):
"""Response model for audio channel."""
id: str
name: str
is_default: bool
device_ids: List[str]
created_at: datetime
class Config:
from_attributes = True
class ChannelVoiceAssignment(BaseModel):
"""Request model for assigning voices to a channel."""
profile_ids: List[str]
class ProfileChannelAssignment(BaseModel):
"""Request model for assigning channels to a profile."""
channel_ids: List[str]
class StoryCreate(BaseModel):
"""Request model for creating a story."""
name: str = Field(..., min_length=1, max_length=100)
description: Optional[str] = Field(None, max_length=500)
class StoryResponse(BaseModel):
"""Response model for story (list view)."""
id: str
name: str
description: Optional[str]
created_at: datetime
updated_at: datetime
item_count: int = 0
class Config:
from_attributes = True
class StoryItemDetail(BaseModel):
"""Detail model for story item with generation info."""
id: str
story_id: str
generation_id: str
start_time_ms: int
track: int = 0
trim_start_ms: int = 0
trim_end_ms: int = 0
created_at: datetime
# Generation details
profile_id: str
profile_name: str
text: str
language: str
audio_path: str
duration: float
seed: Optional[int]
instruct: Optional[str]
generation_created_at: datetime
class Config:
from_attributes = True
class StoryDetailResponse(BaseModel):
"""Response model for story with items."""
id: str
name: str
description: Optional[str]
created_at: datetime
updated_at: datetime
items: List[StoryItemDetail] = []
class Config:
from_attributes = True
class StoryItemCreate(BaseModel):
"""Request model for adding a generation to a story."""
generation_id: str
start_time_ms: Optional[int] = None # If not provided, will be calculated automatically
track: Optional[int] = 0 # Track number (0 = main track)
class StoryItemUpdateTime(BaseModel):
"""Request model for updating a story item's timecode."""
generation_id: str
start_time_ms: int = Field(..., ge=0)
class StoryItemBatchUpdate(BaseModel):
"""Request model for batch updating story item timecodes."""
updates: List[StoryItemUpdateTime]
class StoryItemReorder(BaseModel):
"""Request model for reordering story items."""
generation_ids: List[str] = Field(..., min_length=1)
class StoryItemMove(BaseModel):
"""Request model for moving a story item (position and/or track)."""
start_time_ms: int = Field(..., ge=0)
track: int = 0
class StoryItemTrim(BaseModel):
"""Request model for trimming a story item."""
trim_start_ms: int = Field(..., ge=0)
trim_end_ms: int = Field(..., ge=0)
class StoryItemSplit(BaseModel):
"""Request model for splitting a story item."""
split_time_ms: int = Field(..., ge=0) # Time within the clip to split at (relative to clip start)
+972
View File
@@ -0,0 +1,972 @@
"""
Story management module.
"""
from typing import List, Optional
from datetime import datetime
import uuid
import tempfile
from pathlib import Path
from sqlalchemy.orm import Session
from sqlalchemy import func
from .models import (
StoryCreate,
StoryResponse,
StoryDetailResponse,
StoryItemDetail,
StoryItemCreate,
StoryItemBatchUpdate,
StoryItemMove,
StoryItemTrim,
StoryItemSplit,
)
from .database import Story as DBStory, StoryItem as DBStoryItem, Generation as DBGeneration, VoiceProfile as DBVoiceProfile
from .utils.audio import load_audio, save_audio
import numpy as np
async def create_story(
data: StoryCreate,
db: Session,
) -> StoryResponse:
"""
Create a new story.
Args:
data: Story creation data
db: Database session
Returns:
Created story
"""
db_story = DBStory(
id=str(uuid.uuid4()),
name=data.name,
description=data.description,
created_at=datetime.utcnow(),
updated_at=datetime.utcnow(),
)
db.add(db_story)
db.commit()
db.refresh(db_story)
# Get item count
item_count = db.query(func.count(DBStoryItem.id)).filter(
DBStoryItem.story_id == db_story.id
).scalar()
response = StoryResponse.model_validate(db_story)
response.item_count = item_count
return response
async def list_stories(
db: Session,
) -> List[StoryResponse]:
"""
List all stories.
Args:
db: Database session
Returns:
List of stories with item counts
"""
stories = db.query(DBStory).order_by(DBStory.updated_at.desc()).all()
result = []
for story in stories:
item_count = db.query(func.count(DBStoryItem.id)).filter(
DBStoryItem.story_id == story.id
).scalar()
response = StoryResponse.model_validate(story)
response.item_count = item_count
result.append(response)
return result
async def get_story(
story_id: str,
db: Session,
) -> Optional[StoryDetailResponse]:
"""
Get a story with all its items.
Args:
story_id: Story ID
db: Database session
Returns:
Story with items or None if not found
"""
story = db.query(DBStory).filter_by(id=story_id).first()
if not story:
return None
# Get all items ordered by start_time_ms
items = db.query(
DBStoryItem,
DBGeneration,
DBVoiceProfile.name.label('profile_name')
).join(
DBGeneration,
DBStoryItem.generation_id == DBGeneration.id
).join(
DBVoiceProfile,
DBGeneration.profile_id == DBVoiceProfile.id
).filter(
DBStoryItem.story_id == story_id
).order_by(DBStoryItem.start_time_ms).all()
# Build item details
item_details = []
for item, generation, profile_name in items:
item_detail = StoryItemDetail(
id=item.id,
story_id=item.story_id,
generation_id=item.generation_id,
start_time_ms=item.start_time_ms,
track=item.track,
trim_start_ms=getattr(item, 'trim_start_ms', 0),
trim_end_ms=getattr(item, 'trim_end_ms', 0),
created_at=item.created_at,
profile_id=generation.profile_id,
profile_name=profile_name,
text=generation.text,
language=generation.language,
audio_path=generation.audio_path,
duration=generation.duration,
seed=generation.seed,
instruct=generation.instruct,
generation_created_at=generation.created_at,
)
item_details.append(item_detail)
response = StoryDetailResponse.model_validate(story)
response.items = item_details
return response
async def update_story(
story_id: str,
data: StoryCreate,
db: Session,
) -> Optional[StoryResponse]:
"""
Update a story.
Args:
story_id: Story ID
data: Update data
db: Database session
Returns:
Updated story or None if not found
"""
story = db.query(DBStory).filter_by(id=story_id).first()
if not story:
return None
story.name = data.name
story.description = data.description
story.updated_at = datetime.utcnow()
db.commit()
db.refresh(story)
# Get item count
item_count = db.query(func.count(DBStoryItem.id)).filter(
DBStoryItem.story_id == story.id
).scalar()
response = StoryResponse.model_validate(story)
response.item_count = item_count
return response
async def delete_story(
story_id: str,
db: Session,
) -> bool:
"""
Delete a story and all its items.
Args:
story_id: Story ID
db: Database session
Returns:
True if deleted, False if not found
"""
story = db.query(DBStory).filter_by(id=story_id).first()
if not story:
return False
# Delete all items
db.query(DBStoryItem).filter_by(story_id=story_id).delete()
# Delete story
db.delete(story)
db.commit()
return True
async def add_item_to_story(
story_id: str,
data: StoryItemCreate,
db: Session,
) -> Optional[StoryItemDetail]:
"""
Add a generation to a story.
Args:
story_id: Story ID
data: Item creation data
db: Database session
Returns:
Created item detail or None if story/generation not found
"""
# Verify story exists
story = db.query(DBStory).filter_by(id=story_id).first()
if not story:
return None
# Verify generation exists
generation = db.query(DBGeneration).filter_by(id=data.generation_id).first()
if not generation:
return None
# Check if generation is already in story
existing = db.query(DBStoryItem).filter_by(
story_id=story_id,
generation_id=data.generation_id
).first()
if existing:
# Return existing item
profile = db.query(DBVoiceProfile).filter_by(id=generation.profile_id).first()
return StoryItemDetail(
id=existing.id,
story_id=existing.story_id,
generation_id=existing.generation_id,
start_time_ms=existing.start_time_ms,
track=existing.track,
trim_start_ms=getattr(existing, 'trim_start_ms', 0),
trim_end_ms=getattr(existing, 'trim_end_ms', 0),
created_at=existing.created_at,
profile_id=generation.profile_id,
profile_name=profile.name if profile else "Unknown",
text=generation.text,
language=generation.language,
audio_path=generation.audio_path,
duration=generation.duration,
seed=generation.seed,
instruct=generation.instruct,
generation_created_at=generation.created_at,
)
# Calculate start_time_ms if not provided
if data.start_time_ms is not None:
start_time_ms = data.start_time_ms
else:
# Find the maximum end time (start_time_ms + duration_ms) of existing items
existing_items = db.query(
DBStoryItem,
DBGeneration
).join(
DBGeneration,
DBStoryItem.generation_id == DBGeneration.id
).filter(
DBStoryItem.story_id == story_id
).all()
if not existing_items:
# First item starts at 0
start_time_ms = 0
else:
max_end_time_ms = 0
for item, gen in existing_items:
item_end_ms = item.start_time_ms + int(gen.duration * 1000)
max_end_time_ms = max(max_end_time_ms, item_end_ms)
# Add 200ms gap after the last item
start_time_ms = max_end_time_ms + 200
# Get track from data or default to 0
track = data.track if data.track is not None else 0
# Create item
item = DBStoryItem(
id=str(uuid.uuid4()),
story_id=story_id,
generation_id=data.generation_id,
start_time_ms=start_time_ms,
track=track,
created_at=datetime.utcnow(),
)
db.add(item)
# Update story updated_at
story.updated_at = datetime.utcnow()
db.commit()
db.refresh(item)
# Get profile name
profile = db.query(DBVoiceProfile).filter_by(id=generation.profile_id).first()
return StoryItemDetail(
id=item.id,
story_id=item.story_id,
generation_id=item.generation_id,
start_time_ms=item.start_time_ms,
track=item.track,
trim_start_ms=getattr(item, 'trim_start_ms', 0),
trim_end_ms=getattr(item, 'trim_end_ms', 0),
created_at=item.created_at,
profile_id=generation.profile_id,
profile_name=profile.name if profile else "Unknown",
text=generation.text,
language=generation.language,
audio_path=generation.audio_path,
duration=generation.duration,
seed=generation.seed,
instruct=generation.instruct,
generation_created_at=generation.created_at,
)
async def move_story_item(
story_id: str,
item_id: str,
data: StoryItemMove,
db: Session,
) -> Optional[StoryItemDetail]:
"""
Move a story item (update position and/or track).
Args:
story_id: Story ID
item_id: Story item ID
data: New position and track data
db: Database session
Returns:
Updated item detail or None if not found
"""
# Get the item
item = db.query(DBStoryItem).filter_by(
id=item_id,
story_id=story_id,
).first()
if not item:
return None
# Get the generation
generation = db.query(DBGeneration).filter_by(id=item.generation_id).first()
if not generation:
return None
# Update position and track
item.start_time_ms = data.start_time_ms
item.track = data.track
# Update story updated_at
story = db.query(DBStory).filter_by(id=story_id).first()
if story:
story.updated_at = datetime.utcnow()
db.commit()
db.refresh(item)
# Get profile name
profile = db.query(DBVoiceProfile).filter_by(id=generation.profile_id).first()
return StoryItemDetail(
id=item.id,
story_id=item.story_id,
generation_id=item.generation_id,
start_time_ms=item.start_time_ms,
track=item.track,
trim_start_ms=getattr(item, 'trim_start_ms', 0),
trim_end_ms=getattr(item, 'trim_end_ms', 0),
created_at=item.created_at,
profile_id=generation.profile_id,
profile_name=profile.name if profile else "Unknown",
text=generation.text,
language=generation.language,
audio_path=generation.audio_path,
duration=generation.duration,
seed=generation.seed,
instruct=generation.instruct,
generation_created_at=generation.created_at,
)
async def remove_item_from_story(
story_id: str,
item_id: str,
db: Session,
) -> bool:
"""
Remove a story item from a story.
Args:
story_id: Story ID
item_id: Story item ID to remove
db: Database session
Returns:
True if removed, False if not found
"""
item = db.query(DBStoryItem).filter_by(
id=item_id,
story_id=story_id,
).first()
if not item:
return False
# Delete item
db.delete(item)
# Update story updated_at
story = db.query(DBStory).filter_by(id=story_id).first()
if story:
story.updated_at = datetime.utcnow()
db.commit()
return True
async def trim_story_item(
story_id: str,
item_id: str,
data: StoryItemTrim,
db: Session,
) -> Optional[StoryItemDetail]:
"""
Trim a story item (update trim_start_ms and trim_end_ms).
Args:
story_id: Story ID
item_id: Story item ID
data: Trim data (trim_start_ms, trim_end_ms)
db: Database session
Returns:
Updated item detail or None if not found
"""
# Get the item
item = db.query(DBStoryItem).filter_by(
id=item_id,
story_id=story_id,
).first()
if not item:
return None
# Get the generation
generation = db.query(DBGeneration).filter_by(id=item.generation_id).first()
if not generation:
return None
# Validate trim values don't exceed duration
max_duration_ms = int(generation.duration * 1000)
if data.trim_start_ms + data.trim_end_ms >= max_duration_ms:
return None # Invalid trim - would result in zero or negative duration
# Update trim values
item.trim_start_ms = data.trim_start_ms
item.trim_end_ms = data.trim_end_ms
# Update story updated_at
story = db.query(DBStory).filter_by(id=story_id).first()
if story:
story.updated_at = datetime.utcnow()
db.commit()
db.refresh(item)
# Get profile name
profile = db.query(DBVoiceProfile).filter_by(id=generation.profile_id).first()
return StoryItemDetail(
id=item.id,
story_id=item.story_id,
generation_id=item.generation_id,
start_time_ms=item.start_time_ms,
track=item.track,
trim_start_ms=item.trim_start_ms,
trim_end_ms=item.trim_end_ms,
created_at=item.created_at,
profile_id=generation.profile_id,
profile_name=profile.name if profile else "Unknown",
text=generation.text,
language=generation.language,
audio_path=generation.audio_path,
duration=generation.duration,
seed=generation.seed,
instruct=generation.instruct,
generation_created_at=generation.created_at,
)
async def split_story_item(
story_id: str,
item_id: str,
data: StoryItemSplit,
db: Session,
) -> Optional[List[StoryItemDetail]]:
"""
Split a story item at a given time, creating two clips.
Args:
story_id: Story ID
item_id: Story item ID to split
data: Split data (split_time_ms - time within clip to split at)
db: Database session
Returns:
List of two updated item details (original and new) or None if not found/invalid
"""
# Get the item
item = db.query(DBStoryItem).filter_by(
id=item_id,
story_id=story_id,
).first()
if not item:
return None
# Get the generation
generation = db.query(DBGeneration).filter_by(id=item.generation_id).first()
if not generation:
return None
# Calculate effective duration and validate split point
current_trim_start = getattr(item, 'trim_start_ms', 0)
current_trim_end = getattr(item, 'trim_end_ms', 0)
original_duration_ms = int(generation.duration * 1000)
effective_duration_ms = original_duration_ms - current_trim_start - current_trim_end
# Validate split_time_ms is within the effective duration
if data.split_time_ms <= 0 or data.split_time_ms >= effective_duration_ms:
return None # Invalid split point
# Calculate the absolute time in the original audio where we're splitting
absolute_split_ms = current_trim_start + data.split_time_ms
# Update original clip: trim from the end
item.trim_end_ms = original_duration_ms - absolute_split_ms
# Create new clip: starts after the split, trimmed from the start
new_item = DBStoryItem(
id=str(uuid.uuid4()),
story_id=story_id,
generation_id=item.generation_id, # Same generation, different trim
start_time_ms=item.start_time_ms + data.split_time_ms,
track=item.track,
trim_start_ms=absolute_split_ms,
trim_end_ms=current_trim_end,
created_at=datetime.utcnow(),
)
db.add(new_item)
# Update story updated_at
story = db.query(DBStory).filter_by(id=story_id).first()
if story:
story.updated_at = datetime.utcnow()
db.commit()
db.refresh(item)
db.refresh(new_item)
# Get profile name
profile = db.query(DBVoiceProfile).filter_by(id=generation.profile_id).first()
profile_name = profile.name if profile else "Unknown"
# Build response items
original_item_detail = StoryItemDetail(
id=item.id,
story_id=item.story_id,
generation_id=item.generation_id,
start_time_ms=item.start_time_ms,
track=item.track,
trim_start_ms=item.trim_start_ms,
trim_end_ms=item.trim_end_ms,
created_at=item.created_at,
profile_id=generation.profile_id,
profile_name=profile_name,
text=generation.text,
language=generation.language,
audio_path=generation.audio_path,
duration=generation.duration,
seed=generation.seed,
instruct=generation.instruct,
generation_created_at=generation.created_at,
)
new_item_detail = StoryItemDetail(
id=new_item.id,
story_id=new_item.story_id,
generation_id=new_item.generation_id,
start_time_ms=new_item.start_time_ms,
track=new_item.track,
trim_start_ms=new_item.trim_start_ms,
trim_end_ms=new_item.trim_end_ms,
created_at=new_item.created_at,
profile_id=generation.profile_id,
profile_name=profile_name,
text=generation.text,
language=generation.language,
audio_path=generation.audio_path,
duration=generation.duration,
seed=generation.seed,
instruct=generation.instruct,
generation_created_at=generation.created_at,
)
return [original_item_detail, new_item_detail]
async def duplicate_story_item(
story_id: str,
item_id: str,
db: Session,
) -> Optional[StoryItemDetail]:
"""
Duplicate a story item, creating a copy with all properties.
Args:
story_id: Story ID
item_id: Story item ID to duplicate
db: Database session
Returns:
New item detail or None if not found
"""
# Get the original item
original_item = db.query(DBStoryItem).filter_by(
id=item_id,
story_id=story_id,
).first()
if not original_item:
return None
# Get the generation
generation = db.query(DBGeneration).filter_by(id=original_item.generation_id).first()
if not generation:
return None
# Calculate effective duration
current_trim_start = getattr(original_item, 'trim_start_ms', 0)
current_trim_end = getattr(original_item, 'trim_end_ms', 0)
original_duration_ms = int(generation.duration * 1000)
effective_duration_ms = original_duration_ms - current_trim_start - current_trim_end
# Create duplicate item - place it right after the original
new_item = DBStoryItem(
id=str(uuid.uuid4()),
story_id=story_id,
generation_id=original_item.generation_id, # Same generation as original
start_time_ms=original_item.start_time_ms + effective_duration_ms + 200, # 200ms gap
track=original_item.track,
trim_start_ms=current_trim_start,
trim_end_ms=current_trim_end,
created_at=datetime.utcnow(),
)
db.add(new_item)
# Update story updated_at
story = db.query(DBStory).filter_by(id=story_id).first()
if story:
story.updated_at = datetime.utcnow()
db.commit()
db.refresh(new_item)
# Get profile name
profile = db.query(DBVoiceProfile).filter_by(id=generation.profile_id).first()
return StoryItemDetail(
id=new_item.id,
story_id=new_item.story_id,
generation_id=new_item.generation_id,
start_time_ms=new_item.start_time_ms,
track=new_item.track,
trim_start_ms=new_item.trim_start_ms,
trim_end_ms=new_item.trim_end_ms,
created_at=new_item.created_at,
profile_id=generation.profile_id,
profile_name=profile.name if profile else "Unknown",
text=generation.text,
language=generation.language,
audio_path=generation.audio_path,
duration=generation.duration,
seed=generation.seed,
instruct=generation.instruct,
generation_created_at=generation.created_at,
)
async def update_story_item_times(
story_id: str,
data: StoryItemBatchUpdate,
db: Session,
) -> bool:
"""
Update story item timecodes.
Args:
story_id: Story ID
data: Batch update data with timecodes
db: Database session
Returns:
True if updated, False if story not found or invalid
"""
story = db.query(DBStory).filter_by(id=story_id).first()
if not story:
return False
# Get all items for this story
items = db.query(DBStoryItem).filter_by(story_id=story_id).all()
item_map = {item.generation_id: item for item in items}
# Verify all generation IDs belong to this story and update timecodes
for update in data.updates:
if update.generation_id not in item_map:
return False
item_map[update.generation_id].start_time_ms = update.start_time_ms
# Update story updated_at
story.updated_at = datetime.utcnow()
db.commit()
return True
async def reorder_story_items(
story_id: str,
generation_ids: List[str],
db: Session,
gap_ms: int = 200,
) -> Optional[List[StoryItemDetail]]:
"""
Reorder story items and recalculate timecodes.
Args:
story_id: Story ID
generation_ids: List of generation IDs in the desired order
db: Database session
gap_ms: Gap in milliseconds between items (default 200ms)
Returns:
Updated list of story items with new timecodes, or None if invalid
"""
story = db.query(DBStory).filter_by(id=story_id).first()
if not story:
return None
# Get all items for this story with their generation data
items_with_gen = db.query(
DBStoryItem,
DBGeneration,
DBVoiceProfile.name.label('profile_name')
).join(
DBGeneration,
DBStoryItem.generation_id == DBGeneration.id
).join(
DBVoiceProfile,
DBGeneration.profile_id == DBVoiceProfile.id
).filter(
DBStoryItem.story_id == story_id
).all()
# Create maps for quick lookup
item_map = {item.generation_id: (item, gen, profile_name) for item, gen, profile_name in items_with_gen}
# Verify all generation IDs belong to this story
if set(generation_ids) != set(item_map.keys()):
return None
# Recalculate timecodes based on new order
current_time_ms = 0
updated_items = []
for gen_id in generation_ids:
item, generation, profile_name = item_map[gen_id]
# Update the item's start time
item.start_time_ms = current_time_ms
# Calculate the duration in ms
duration_ms = int(generation.duration * 1000)
# Move to next position (current end + gap)
current_time_ms += duration_ms + gap_ms
# Build the response item
updated_items.append(StoryItemDetail(
id=item.id,
story_id=item.story_id,
generation_id=item.generation_id,
start_time_ms=item.start_time_ms,
track=item.track,
trim_start_ms=getattr(item, 'trim_start_ms', 0),
trim_end_ms=getattr(item, 'trim_end_ms', 0),
created_at=item.created_at,
profile_id=generation.profile_id,
profile_name=profile_name,
text=generation.text,
language=generation.language,
audio_path=generation.audio_path,
duration=generation.duration,
seed=generation.seed,
instruct=generation.instruct,
generation_created_at=generation.created_at,
))
# Update story updated_at
story.updated_at = datetime.utcnow()
db.commit()
return updated_items
async def export_story_audio(
story_id: str,
db: Session,
) -> Optional[bytes]:
"""
Export story as single mixed audio file with timecode-based mixing.
Args:
story_id: Story ID
db: Database session
Returns:
Audio file bytes or None if story not found
"""
story = db.query(DBStory).filter_by(id=story_id).first()
if not story:
return None
# Get all items ordered by start_time_ms
items = db.query(
DBStoryItem,
DBGeneration
).join(
DBGeneration,
DBStoryItem.generation_id == DBGeneration.id
).filter(
DBStoryItem.story_id == story_id
).order_by(DBStoryItem.start_time_ms).all()
if not items:
return None
# Load all audio files and calculate total duration
audio_data = []
sample_rate = 24000 # Default sample rate
for item, generation in items:
audio_path = Path(generation.audio_path)
if not audio_path.exists():
continue
try:
audio, sr = load_audio(str(audio_path), sample_rate=sample_rate)
sample_rate = sr # Use actual sample rate from first file
# Get trim values
trim_start_ms = getattr(item, 'trim_start_ms', 0)
trim_end_ms = getattr(item, 'trim_end_ms', 0)
# Calculate effective duration
original_duration_ms = int(generation.duration * 1000)
effective_duration_ms = original_duration_ms - trim_start_ms - trim_end_ms
# Slice audio based on trim values
trim_start_sample = int((trim_start_ms / 1000.0) * sample_rate)
trim_end_sample = int((trim_end_ms / 1000.0) * sample_rate)
# Extract the trimmed portion
if trim_end_ms > 0:
trimmed_audio = audio[trim_start_sample:-trim_end_sample] if trim_end_sample > 0 else audio[trim_start_sample:]
else:
trimmed_audio = audio[trim_start_sample:]
# Store audio with its timecode info
start_time_ms = item.start_time_ms
audio_data.append({
'audio': trimmed_audio,
'start_time_ms': start_time_ms,
'duration_ms': effective_duration_ms,
})
except Exception:
# Skip files that can't be loaded
continue
if not audio_data:
return None
# Calculate total duration: max(start_time_ms + duration_ms)
max_end_time_ms = max(
(data['start_time_ms'] + data['duration_ms'] for data in audio_data),
default=0
)
# Convert to samples
total_samples = int((max_end_time_ms / 1000.0) * sample_rate)
# Create output buffer initialized to zeros
final_audio = np.zeros(total_samples, dtype=np.float32)
# Mix each audio segment at its timecode position
for data in audio_data:
audio = data['audio']
start_time_ms = data['start_time_ms']
# Calculate start sample index
start_sample = int((start_time_ms / 1000.0) * sample_rate)
# Ensure we don't exceed buffer bounds
audio_length = len(audio)
end_sample = min(start_sample + audio_length, total_samples)
if start_sample < total_samples:
# Trim audio if it extends beyond buffer
audio_to_mix = audio[:end_sample - start_sample]
# Mix: add audio to existing buffer (overlapping audio will sum)
# Normalize to prevent clipping (simple approach: divide by max)
final_audio[start_sample:end_sample] += audio_to_mix
# Normalize to prevent clipping
max_val = np.abs(final_audio).max()
if max_val > 1.0:
final_audio = final_audio / max_val
# Save to temporary file
with tempfile.NamedTemporaryFile(suffix='.wav', delete=False) as tmp:
tmp_path = tmp.name
try:
save_audio(final_audio, tmp_path, sample_rate)
# Read file bytes
with open(tmp_path, 'rb') as f:
audio_bytes = f.read()
return audio_bytes
finally:
# Clean up temp file
Path(tmp_path).unlink(missing_ok=True)
+38 -4
View File
@@ -13,8 +13,11 @@
},
"app": {
"name": "@voicebox/app",
"version": "0.1.0",
"version": "0.1.5",
"dependencies": {
"@dnd-kit/core": "^6.3.1",
"@dnd-kit/sortable": "^10.0.0",
"@dnd-kit/utilities": "^3.2.2",
"@hookform/resolvers": "^3.9.0",
"@radix-ui/react-alert-dialog": "^1.1.1",
"@radix-ui/react-avatar": "^1.1.0",
@@ -32,6 +35,7 @@
"@radix-ui/react-toast": "^1.2.1",
"@tanstack/react-query": "^5.0.0",
"@tanstack/react-query-devtools": "^5.0.0",
"@tanstack/react-router": "^1.157.16",
"@tauri-apps/api": "^2.0.0",
"@tauri-apps/plugin-dialog": "^2.0.0",
"@tauri-apps/plugin-fs": "^2.0.0",
@@ -63,7 +67,7 @@
},
"landing": {
"name": "@voicebox/landing",
"version": "0.1.0",
"version": "0.1.5",
"dependencies": {
"@radix-ui/react-separator": "^1.1.8",
"@radix-ui/react-slot": "^1.2.4",
@@ -88,7 +92,7 @@
},
"tauri": {
"name": "@voicebox/tauri",
"version": "0.1.0",
"version": "0.1.5",
"dependencies": {
"@tauri-apps/api": "^2.0.0",
"@tauri-apps/plugin-shell": "^2.0.0",
@@ -107,7 +111,7 @@
},
"web": {
"name": "@voicebox/web",
"version": "0.1.0",
"version": "0.1.5",
"dependencies": {
"@tanstack/react-query": "^5.0.0",
"react": "^18.3.0",
@@ -188,6 +192,14 @@
"@biomejs/cli-win32-x64": ["@biomejs/[email protected]", "", { "os": "win32", "cpu": "x64" }, "sha512-qqGVWqNNek0KikwPZlOIoxtXgsNGsX+rgdEzgw82Re8nF02W+E2WokaQhpF5TdBh/D/RQ3TLppH+otp6ztN0lw=="],
"@dnd-kit/accessibility": ["@dnd-kit/[email protected]", "", { "dependencies": { "tslib": "^2.0.0" }, "peerDependencies": { "react": ">=16.8.0" } }, "sha512-2P+YgaXF+gRsIihwwY1gCsQSYnu9Zyj2py8kY5fFvUM1qm2WA2u639R6YNVfU4GWr+ZM5mqEsfHZZLoRONbemw=="],
"@dnd-kit/core": ["@dnd-kit/[email protected]", "", { "dependencies": { "@dnd-kit/accessibility": "^3.1.1", "@dnd-kit/utilities": "^3.2.2", "tslib": "^2.0.0" }, "peerDependencies": { "react": ">=16.8.0", "react-dom": ">=16.8.0" } }, "sha512-xkGBRQQab4RLwgXxoqETICr6S5JlogafbhNsidmrkVv2YRs5MLwpjoF2qpiGjQt8S9AoxtIV603s0GIUpY5eYQ=="],
"@dnd-kit/sortable": ["@dnd-kit/[email protected]", "", { "dependencies": { "@dnd-kit/utilities": "^3.2.2", "tslib": "^2.0.0" }, "peerDependencies": { "@dnd-kit/core": "^6.3.0", "react": ">=16.8.0" } }, "sha512-+xqhmIIzvAYMGfBYYnbKuNicfSsk4RksY2XdmJhT+HAC01nix6fHCztU68jooFiMUB01Ky3F0FyOvhG/BZrWkg=="],
"@dnd-kit/utilities": ["@dnd-kit/[email protected]", "", { "dependencies": { "tslib": "^2.0.0" }, "peerDependencies": { "react": ">=16.8.0" } }, "sha512-+MKAJEOfaBe5SmV6t34p80MMKhjvUz0vRrvVJbPT0WElzaOJ/1xs+D+KDv+tD/NE5ujfrChEcshd4fLn0wpiqg=="],
"@emnapi/runtime": ["@emnapi/[email protected]", "", { "dependencies": { "tslib": "^2.4.0" } }, "sha512-mehfKSMWjjNol8659Z8KxEMrdSJDDot5SXMq00dM8BN4o+CLNXQ0xH2V7EchNHV4RmbZLmmPdEaXZc5H2FXmDg=="],
"@esbuild/aix-ppc64": ["@esbuild/[email protected]", "", { "os": "aix", "cpu": "ppc64" }, "sha512-1SDgH6ZSPTlggy1yI6+Dbkiz8xzpHJEVAlF/AM1tHPLsf5STom9rwtjE4hKAF20FfXXNTFqEYXyJNWh1GiZedQ=="],
@@ -512,6 +524,8 @@
"@tailwindcss/vite": ["@tailwindcss/[email protected]", "", { "dependencies": { "@tailwindcss/node": "4.1.18", "@tailwindcss/oxide": "4.1.18", "tailwindcss": "4.1.18" }, "peerDependencies": { "vite": "^5.2.0 || ^6 || ^7" } }, "sha512-jVA+/UpKL1vRLg6Hkao5jldawNmRo7mQYrZtNHMIVpLfLhDml5nMRUo/8MwoX2vNXvnaXNNMedrMfMugAVX1nA=="],
"@tanstack/history": ["@tanstack/[email protected]", "", {}, "sha512-xyIfof8eHBuub1CkBnbKNKQXeRZC4dClhmzePHVOEel4G7lk/dW+TQ16da7CFdeNLv6u6Owf5VoBQxoo6DFTSA=="],
"@tanstack/query-core": ["@tanstack/[email protected]", "", {}, "sha512-OMD2HLpNouXEfZJWcKeVKUgQ5n+n3A2JFmBaScpNDUqSrQSjiveC7dKMe53uJUg1nDG16ttFPz2xfilz6i2uVg=="],
"@tanstack/query-devtools": ["@tanstack/[email protected]", "", {}, "sha512-N8D27KH1vEpVacvZgJL27xC6yPFUy0Zkezn5gnB3L3gRCxlDeSuiya7fKge8Y91uMTnC8aSxBQhcK6ocY7alpQ=="],
@@ -520,6 +534,14 @@
"@tanstack/react-query-devtools": ["@tanstack/[email protected]", "", { "dependencies": { "@tanstack/query-devtools": "5.92.0" }, "peerDependencies": { "@tanstack/react-query": "^5.90.14", "react": "^18 || ^19" } }, "sha512-ZJ1503ay5fFeEYFUdo7LMNFzZryi6B0Cacrgr2h1JRkvikK1khgIq6Nq2EcblqEdIlgB/r7XDW8f8DQ89RuUgg=="],
"@tanstack/react-router": ["@tanstack/[email protected]", "", { "dependencies": { "@tanstack/history": "1.154.14", "@tanstack/react-store": "^0.8.0", "@tanstack/router-core": "1.157.16", "isbot": "^5.1.22", "tiny-invariant": "^1.3.3", "tiny-warning": "^1.0.3" }, "peerDependencies": { "react": ">=18.0.0 || >=19.0.0", "react-dom": ">=18.0.0 || >=19.0.0" } }, "sha512-xwFQa7S7dhBhm3aJYwU79cITEYgAKSrcL6wokaROIvl2JyIeazn8jueWqUPJzFjv+QF6Q8euKRlKUEyb5q2ymg=="],
"@tanstack/react-store": ["@tanstack/[email protected]", "", { "dependencies": { "@tanstack/store": "0.8.0", "use-sync-external-store": "^1.6.0" }, "peerDependencies": { "react": "^16.8.0 || ^17.0.0 || ^18.0.0 || ^19.0.0", "react-dom": "^16.8.0 || ^17.0.0 || ^18.0.0 || ^19.0.0" } }, "sha512-1vG9beLIuB7q69skxK9r5xiLN3ztzIPfSQSs0GfeqWGO2tGIyInZx0x1COhpx97RKaONSoAb8C3dxacWksm1ow=="],
"@tanstack/router-core": ["@tanstack/[email protected]", "", { "dependencies": { "@tanstack/history": "1.154.14", "@tanstack/store": "^0.8.0", "cookie-es": "^2.0.0", "seroval": "^1.4.2", "seroval-plugins": "^1.4.2", "tiny-invariant": "^1.3.3", "tiny-warning": "^1.0.3" } }, "sha512-eJuVgM7KZYTTr4uPorbUzUflmljMVcaX2g6VvhITLnHmg9SBx9RAgtQ1HmT+72mzyIbRSlQ1q0fY/m+of/fosA=="],
"@tanstack/store": ["@tanstack/[email protected]", "", {}, "sha512-Om+BO0YfMZe//X2z0uLF2j+75nQga6TpTJgLJQBiq85aOyZNIhkCgleNcud2KQg4k4v9Y9l+Uhru3qWMPGTOzQ=="],
"@tauri-apps/api": ["@tauri-apps/[email protected]", "", {}, "sha512-IGlhP6EivjXHepbBic618GOmiWe4URJiIeZFlB7x3czM0yDHHYviH1Xvoiv4FefdkQtn6v7TuwWCRfOGdnVUGw=="],
"@tauri-apps/cli": ["@tauri-apps/[email protected]", "", { "optionalDependencies": { "@tauri-apps/cli-darwin-arm64": "2.9.6", "@tauri-apps/cli-darwin-x64": "2.9.6", "@tauri-apps/cli-linux-arm-gnueabihf": "2.9.6", "@tauri-apps/cli-linux-arm64-gnu": "2.9.6", "@tauri-apps/cli-linux-arm64-musl": "2.9.6", "@tauri-apps/cli-linux-riscv64-gnu": "2.9.6", "@tauri-apps/cli-linux-x64-gnu": "2.9.6", "@tauri-apps/cli-linux-x64-musl": "2.9.6", "@tauri-apps/cli-win32-arm64-msvc": "2.9.6", "@tauri-apps/cli-win32-ia32-msvc": "2.9.6", "@tauri-apps/cli-win32-x64-msvc": "2.9.6" }, "bin": { "tauri": "tauri.js" } }, "sha512-3xDdXL5omQ3sPfBfdC8fCtDKcnyV7OqyzQgfyT5P3+zY6lcPqIYKQBvUasNvppi21RSdfhy44ttvJmftb0PCDw=="],
@@ -664,6 +686,8 @@
"convert-source-map": ["[email protected]", "", {}, "sha512-Kvp459HrV2FEJ1CAsi1Ku+MY3kasH19TFykTz2xWmMeq6bk2NU3XXvfJ+Q61m0xktWwt+1HSYf3JZsTms3aRJg=="],
"cookie-es": ["[email protected]", "", {}, "sha512-RAj4E421UYRgqokKUmotqAwuplYw15qtdXfY+hGzgCJ/MBjCVZcSoHK/kH9kocfjRjcDME7IiDWR/1WX1TM2Pg=="],
"cross-spawn": ["[email protected]", "", { "dependencies": { "path-key": "^3.1.0", "shebang-command": "^2.0.0", "which": "^2.0.1" } }, "sha512-uV2QOWP2nWzsy2aMp8aRibhi9dlzF5Hgh5SHaB9OiTGEyDTiJJyx0uy51QXdyWbtAHNua4XJzUKca3OzKUd3vA=="],
"cssesc": ["[email protected]", "", { "bin": { "cssesc": "bin/cssesc" } }, "sha512-/Tb/JcjK111nNScGob5MNtsntNM1aCNUDipB/TkwZFhyDrrE47SOx/18wF2bbjgc3ZzCSKW1T5nt5EbFoAz/Vg=="],
@@ -792,6 +816,8 @@
"is-path-inside": ["[email protected]", "", {}, "sha512-Fd4gABb+ycGAmKou8eMftCupSir5lRxqf4aD/vd0cD2qc4HL07OjCeuHMr8Ro4CoMaeCKDB0/ECBOVWjTwUvPQ=="],
"isbot": ["[email protected]", "", {}, "sha512-aCMIBSKd/XPRYdiCQTLC8QHH4YT8B3JUADu+7COgYIZPvkeoMcUHMRjZLM9/7V8fCj+l7FSREc1lOPNjzogo/A=="],
"isexe": ["[email protected]", "", {}, "sha512-RHxMLp9lnKHGHRng9QFhRCMbYAcVpn69smSGcq3f36xjgVVWThj4qqLbTLlq7Ssj8B+fIQ1EuCEGI2lKsyQeIw=="],
"jiti": ["[email protected]", "", { "bin": { "jiti": "bin/jiti.js" } }, "sha512-/imKNG4EbWNrVjoNC/1H5/9GFy+tqjGBHCaSsN+P2RnPqjsLmv6UD3Ej+Kj8nBWaRAwyk7kK5ZUc+OEatnTR3A=="],
@@ -966,6 +992,10 @@
"semver": ["[email protected]", "", { "bin": { "semver": "bin/semver.js" } }, "sha512-BR7VvDCVHO+q2xBEWskxS6DJE1qRnb7DxzUrogb71CWoSficBxYsiAGd+Kl0mmq/MprG9yArRkyrQxTO6XjMzA=="],
"seroval": ["[email protected]", "", {}, "sha512-OE4cvmJ1uSPrKorFIH9/w/Qwuvi/IMcGbv5RKgcJ/zjA/IohDLU6SVaxFN9FwajbP7nsX0dQqMDes1whk3y+yw=="],
"seroval-plugins": ["[email protected]", "", { "peerDependencies": { "seroval": "^1.0" } }, "sha512-EAHqADIQondwRZIdeW2I636zgsODzoBDwb3PT/+7TLDWyw1Dy/Xv7iGUIEXXav7usHDE9HVhOU61irI3EnyyHA=="],
"sharp": ["[email protected]", "", { "dependencies": { "@img/colour": "^1.0.0", "detect-libc": "^2.1.2", "semver": "^7.7.3" }, "optionalDependencies": { "@img/sharp-darwin-arm64": "0.34.5", "@img/sharp-darwin-x64": "0.34.5", "@img/sharp-libvips-darwin-arm64": "1.2.4", "@img/sharp-libvips-darwin-x64": "1.2.4", "@img/sharp-libvips-linux-arm": "1.2.4", "@img/sharp-libvips-linux-arm64": "1.2.4", "@img/sharp-libvips-linux-ppc64": "1.2.4", "@img/sharp-libvips-linux-riscv64": "1.2.4", "@img/sharp-libvips-linux-s390x": "1.2.4", "@img/sharp-libvips-linux-x64": "1.2.4", "@img/sharp-libvips-linuxmusl-arm64": "1.2.4", "@img/sharp-libvips-linuxmusl-x64": "1.2.4", "@img/sharp-linux-arm": "0.34.5", "@img/sharp-linux-arm64": "0.34.5", "@img/sharp-linux-ppc64": "0.34.5", "@img/sharp-linux-riscv64": "0.34.5", "@img/sharp-linux-s390x": "0.34.5", "@img/sharp-linux-x64": "0.34.5", "@img/sharp-linuxmusl-arm64": "0.34.5", "@img/sharp-linuxmusl-x64": "0.34.5", "@img/sharp-wasm32": "0.34.5", "@img/sharp-win32-arm64": "0.34.5", "@img/sharp-win32-ia32": "0.34.5", "@img/sharp-win32-x64": "0.34.5" } }, "sha512-Ou9I5Ft9WNcCbXrU9cMgPBcCK8LiwLqcbywW3t4oDV37n1pzpuNLsYiAV8eODnjbtQlSDwZ2cUEeQz4E54Hltg=="],
"shebang-command": ["[email protected]", "", { "dependencies": { "shebang-regex": "^3.0.0" } }, "sha512-kHxr2zZpYtdmrN1qDjrrX/Z1rR1kG8Dx+gkpK1G4eXmvXswmcE1hTWBWYUzlraYw1/yZp6YuDY77YtvbN0dmDA=="],
@@ -1002,6 +1032,10 @@
"thenify-all": ["[email protected]", "", { "dependencies": { "thenify": ">= 3.1.0 < 4" } }, "sha512-RNxQH/qI8/t3thXJDwcstUO4zeqo64+Uy/+sNVRBx4Xn2OX+OZ9oP+iJnNFqplFra2ZUVeKCSa2oVWi3T4uVmA=="],
"tiny-invariant": ["[email protected]", "", {}, "sha512-+FbBPE1o9QAYvviau/qC5SE3caw21q3xkvWKBtja5vgqOWIHHJ3ioaq1VPfn/Szqctz2bU/oYeKd9/z5BL+PVg=="],
"tiny-warning": ["[email protected]", "", {}, "sha512-lBN9zLN/oAf68o3zNXYrdCt1kP8WsiGW8Oo2ka41b2IM5JL/S1CTyX1rW0mb/zSuJun0ZUrDxx4sqvYS2FWzPA=="],
"tinyglobby": ["[email protected]", "", { "dependencies": { "fdir": "^6.5.0", "picomatch": "^4.0.3" } }, "sha512-j2Zq4NyQYG5XMST4cbs02Ak8iJUdxRM0XI5QyxXuZOzKOINmWurp3smXu3y5wDcJrptwpSjgXHzIQxR0omXljQ=="],
"to-regex-range": ["[email protected]", "", { "dependencies": { "is-number": "^7.0.0" } }, "sha512-65P7iz6X5yEr1cwcgvQxbbIw7Uk3gOy5dIdtZ4rDveLqhrdJP+Li/Hx6tyK0NEb+2GCyneCMJiGqrADCSNk8sQ=="],
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@voicebox/landing",
"version": "0.1.2",
"version": "0.1.8",
"description": "Landing page for voicebox.sh",
"scripts": {
"dev": "bun --bun next dev --turbo",
Binary file not shown.

After

Width:  |  Height:  |  Size: 134 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 129 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 108 KiB

+38 -3
View File
@@ -99,7 +99,7 @@ export default function Home() {
<div className="flex justify-center lg:hidden my-8">
<div className="w-full max-w-2xl">
<Image
src="/VoiceBoxAppScreenshot.webp"
src="/assets/app-screenshot-1.webp"
alt="Voicebox Application Screenshot"
width={1920}
height={1080}
@@ -180,7 +180,7 @@ export default function Home() {
<div className="hidden lg:block relative">
<div className="absolute right-0 top-0 -mt-10 w-[200%] -mr-[100%]">
<Image
src="/VoiceBoxAppScreenshot.webp"
src="/assets/app-screenshot-1.webp"
alt="Voicebox Application Screenshot"
width={1920}
height={1080}
@@ -193,6 +193,41 @@ export default function Home() {
</div>
</section>
{/* Screenshots Section */}
<section className="py-12 sm:py-16 md:py-20">
<div className="w-full md:w-[150%] md:-ml-[25%]">
<div className="grid grid-cols-1 md:grid-cols-3 gap-6 px-8">
<div className="w-full">
<Image
src="/assets/app-screenshot-2.webp"
alt="Voicebox Screenshot 2"
width={1920}
height={1080}
className="w-full h-auto rounded-lg shadow-lg"
/>
</div>
<div className="w-full">
<Image
src="/assets/app-screenshot-1.webp"
alt="Voicebox Screenshot 1"
width={1920}
height={1080}
className="w-full h-auto rounded-lg shadow-lg"
/>
</div>
<div className="w-full">
<Image
src="/assets/app-screenshot-3.webp"
alt="Voicebox Screenshot 3"
width={1920}
height={1080}
className="w-full h-auto rounded-lg shadow-lg"
/>
</div>
</div>
</div>
</section>
{/* Demo Video Section */}
<section className="py-12 sm:py-16 md:py-20">
<div className="container mx-auto px-4 sm:px-6 lg:px-8 max-w-7xl">
@@ -207,7 +242,7 @@ export default function Home() {
controls
playsInline
preload="metadata"
poster="/VoiceBoxAppScreenshot.webp"
poster="/assets/app-screenshot-1.webp"
>
<source
src="/voicebox-demo.webm"
+2 -2
View File
@@ -1,6 +1,6 @@
{
"name": "voicebox",
"version": "0.1.2",
"version": "0.1.8",
"private": true,
"workspaces": [
"app",
@@ -12,7 +12,7 @@
"dev": "cd tauri && bun run tauri dev",
"dev:web": "cd web && bun run dev",
"dev:landing": "cd landing && bun run dev",
"dev:server": "uvicorn backend.main:app --reload --port 17493",
"dev:server": "uvicorn backend.main:app --reload --port 8000",
"build": "cd tauri && bun run tauri build",
"build:web": "cd web && bun run build",
"build:landing": "cd landing && bun run build",
+1 -1
View File
@@ -1,7 +1,7 @@
{
"name": "@voicebox/tauri",
"private": true,
"version": "0.1.2",
"version": "0.1.8",
"type": "module",
"scripts": {
"dev": "vite",
+613 -11
View File
@@ -32,6 +32,28 @@ dependencies = [
"alloc-no-stdlib",
]
[[package]]
name = "alsa"
version = "0.9.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ed7572b7ba83a31e20d1b48970ee402d2e3e0537dcfe0a3ff4d6eb7508617d43"
dependencies = [
"alsa-sys",
"bitflags 2.10.0",
"cfg-if",
"libc",
]
[[package]]
name = "alsa-sys"
version = "0.3.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "db8fee663d06c4e303404ef5f40488a53e062f89ba8bfed81f42325aafad1527"
dependencies = [
"libc",
"pkg-config",
]
[[package]]
name = "android_system_properties"
version = "0.1.5"
@@ -56,6 +78,12 @@ dependencies = [
"derive_arbitrary",
]
[[package]]
name = "arrayvec"
version = "0.7.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7c02d123df017efcdfbd739ef81735b36c5ba83ec3c59c80a9d7ecc718f92e50"
[[package]]
name = "atk"
version = "0.18.2"
@@ -276,6 +304,8 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6354c81bbfd62d9cfa9cb3c773c2b7b2a3a482d569de977fd0e961f6e7c00583"
dependencies = [
"find-msvc-tools",
"jobserver",
"libc",
"shlex",
]
@@ -376,6 +406,16 @@ dependencies = [
"version_check",
]
[[package]]
name = "core-foundation"
version = "0.9.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "91e195e091a93c46f7102ec7818a2aa394e1e1771c3ab4825963fa03e45afb8f"
dependencies = [
"core-foundation-sys",
"libc",
]
[[package]]
name = "core-foundation"
version = "0.10.1"
@@ -399,9 +439,9 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "fa95a34622365fa5bbf40b20b75dba8dfa8c94c734aea8ac9a5ca38af14316f1"
dependencies = [
"bitflags 2.10.0",
"core-foundation",
"core-foundation 0.10.1",
"core-graphics-types",
"foreign-types",
"foreign-types 0.5.0",
"libc",
]
@@ -412,10 +452,21 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3d44a101f213f6c4cdc1853d4b78aef6db6bdfa3468798cc1d9912f4735013eb"
dependencies = [
"bitflags 2.10.0",
"core-foundation",
"core-foundation 0.10.1",
"libc",
]
[[package]]
name = "coreaudio-rs"
version = "0.11.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "321077172d79c662f64f5071a03120748d5bb652f5231570141be24cfcd2bace"
dependencies = [
"bitflags 1.3.2",
"core-foundation-sys",
"coreaudio-sys",
]
[[package]]
name = "coreaudio-sys"
version = "0.2.17"
@@ -425,6 +476,29 @@ dependencies = [
"bindgen",
]
[[package]]
name = "cpal"
version = "0.15.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "873dab07c8f743075e57f524c583985fbaf745602acbe916a01539364369a779"
dependencies = [
"alsa",
"core-foundation-sys",
"coreaudio-rs",
"dasp_sample",
"jni",
"js-sys",
"libc",
"mach2",
"ndk 0.8.0",
"ndk-context",
"oboe",
"wasm-bindgen",
"wasm-bindgen-futures",
"web-sys",
"windows 0.54.0",
]
[[package]]
name = "cpufeatures"
version = "0.2.17"
@@ -540,6 +614,12 @@ dependencies = [
"syn 2.0.114",
]
[[package]]
name = "dasp_sample"
version = "0.11.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0c87e182de0887fd5361989c677c4e8f5000cd9491d6d563161a8f3a5519fc7f"
[[package]]
name = "deranged"
version = "0.5.5"
@@ -755,6 +835,12 @@ dependencies = [
"windows-sys 0.61.2",
]
[[package]]
name = "extended"
version = "0.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "af9673d8203fcb076b19dfd17e38b3d4ae9f44959416ea532ce72415a6020365"
[[package]]
name = "fastrand"
version = "2.3.0"
@@ -813,6 +899,15 @@ version = "1.0.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3f9eec918d3f24069decb9af1554cad7c880e2da24a9afd88aca000531ab82c1"
[[package]]
name = "foreign-types"
version = "0.3.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f6f339eb8adc052cd2ca78910fda869aefa38d22d5cb648e6485e4d3fc06f3b1"
dependencies = [
"foreign-types-shared 0.1.1",
]
[[package]]
name = "foreign-types"
version = "0.5.0"
@@ -820,7 +915,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d737d9aa519fb7b749cbc3b962edcf310a8dd1f4b67c91c4f83975dbdd17d965"
dependencies = [
"foreign-types-macros",
"foreign-types-shared",
"foreign-types-shared 0.3.1",
]
[[package]]
@@ -834,6 +929,12 @@ dependencies = [
"syn 2.0.114",
]
[[package]]
name = "foreign-types-shared"
version = "0.1.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "00b0228411908ca8685dba7fc2cdd70ec9990a6e753e89b6ac91a84c40fbaf4b"
[[package]]
name = "foreign-types-shared"
version = "0.3.1"
@@ -866,6 +967,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2dff15bf788c671c1934e366d07e30c1814a8ef514e1af724a602e8a2fbe1b10"
dependencies = [
"futures-core",
"futures-sink",
]
[[package]]
@@ -1235,6 +1337,25 @@ dependencies = [
"syn 2.0.114",
]
[[package]]
name = "h2"
version = "0.4.12"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f3c0b69cfcb4e1b9f1bf2f53f95f766e4661169728ec61cd3fe5a0166f2d1386"
dependencies = [
"atomic-waker",
"bytes",
"fnv",
"futures-core",
"futures-sink",
"http",
"indexmap 2.13.0",
"slab",
"tokio",
"tokio-util",
"tracing",
]
[[package]]
name = "hashbrown"
version = "0.12.3"
@@ -1332,6 +1453,7 @@ dependencies = [
"bytes",
"futures-channel",
"futures-core",
"h2",
"http",
"http-body",
"httparse",
@@ -1360,6 +1482,22 @@ dependencies = [
"webpki-roots",
]
[[package]]
name = "hyper-tls"
version = "0.6.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "70206fc6890eaca9fde8a0bf71caa2ddfc9fe045ac9e5c70df101a7dbde866e0"
dependencies = [
"bytes",
"http-body-util",
"hyper",
"hyper-util",
"native-tls",
"tokio",
"tokio-native-tls",
"tower-service",
]
[[package]]
name = "hyper-util"
version = "0.1.19"
@@ -1379,9 +1517,11 @@ dependencies = [
"percent-encoding",
"pin-project-lite",
"socket2",
"system-configuration",
"tokio",
"tower-service",
"tracing",
"windows-registry",
]
[[package]]
@@ -1653,6 +1793,16 @@ version = "0.3.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8eaf4bc02d17cbdd7ff4c7438cafcdf7fb9a4613313ad11b4f8fefe7d3fa0130"
[[package]]
name = "jobserver"
version = "0.1.34"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9afb3de4395d6b3e67a780b6de64b51c978ecf11cb9a462c66be7d4ca9039d33"
dependencies = [
"getrandom 0.3.4",
"libc",
]
[[package]]
name = "js-sys"
version = "0.3.85"
@@ -1814,6 +1964,15 @@ version = "0.1.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c41e0c4fef86961ac6d6f8a82609f55f31b05e4fce149ac5710e439df7619ba4"
[[package]]
name = "mach2"
version = "0.4.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d640282b302c0bb0a2a8e0233ead9035e3bed871f0b7e81fe4a1ec829765db44"
dependencies = [
"libc",
]
[[package]]
name = "malloc_buf"
version = "0.0.6"
@@ -1929,6 +2088,37 @@ dependencies = [
"windows-sys 0.60.2",
]
[[package]]
name = "native-tls"
version = "0.2.14"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "87de3442987e9dbec73158d5c715e7ad9072fda936bb03d19d7fa10e00520f0e"
dependencies = [
"libc",
"log",
"openssl",
"openssl-probe",
"openssl-sys",
"schannel",
"security-framework",
"security-framework-sys",
"tempfile",
]
[[package]]
name = "ndk"
version = "0.8.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2076a31b7010b17a38c01907c45b945e8f11495ee4dd588309718901b1f7a5b7"
dependencies = [
"bitflags 2.10.0",
"jni-sys",
"log",
"ndk-sys 0.5.0+25.2.9519653",
"num_enum",
"thiserror 1.0.69",
]
[[package]]
name = "ndk"
version = "0.9.0"
@@ -1938,7 +2128,7 @@ dependencies = [
"bitflags 2.10.0",
"jni-sys",
"log",
"ndk-sys",
"ndk-sys 0.6.0+11769913",
"num_enum",
"raw-window-handle",
"thiserror 1.0.69",
@@ -1950,6 +2140,15 @@ version = "0.1.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "27b02d87554356db9e9a873add8782d4ea6e3e58ea071a9adb9a2e8ddb884a8b"
[[package]]
name = "ndk-sys"
version = "0.5.0+25.2.9519653"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8c196769dd60fd4f363e11d948139556a344e79d451aeb2fa2fd040738ef7691"
dependencies = [
"jni-sys",
]
[[package]]
name = "ndk-sys"
version = "0.6.0+11769913"
@@ -1987,6 +2186,17 @@ version = "0.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cf97ec579c3c42f953ef76dbf8d55ac91fb219dde70e49aa4a6b7d74e9919050"
[[package]]
name = "num-derive"
version = "0.4.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ed3955f1a9c7c0c15e092f9c887db08b1fc683305fdf6eb6684f22555355e202"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.114",
]
[[package]]
name = "num-integer"
version = "0.1.46"
@@ -2260,6 +2470,29 @@ dependencies = [
"objc2-security",
]
[[package]]
name = "oboe"
version = "0.6.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e8b61bebd49e5d43f5f8cc7ee2891c16e0f41ec7954d36bcb6c14c5e0de867fb"
dependencies = [
"jni",
"ndk 0.8.0",
"ndk-context",
"num-derive",
"num-traits",
"oboe-sys",
]
[[package]]
name = "oboe-sys"
version = "0.6.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6c8bb09a4a2b1d668170cfe0a7d5bc103f8999fb316c98099b6a9939c9f2e79d"
dependencies = [
"cc",
]
[[package]]
name = "once_cell"
version = "1.21.3"
@@ -2278,6 +2511,50 @@ dependencies = [
"pathdiff",
]
[[package]]
name = "openssl"
version = "0.10.75"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "08838db121398ad17ab8531ce9de97b244589089e290a384c900cb9ff7434328"
dependencies = [
"bitflags 2.10.0",
"cfg-if",
"foreign-types 0.3.2",
"libc",
"once_cell",
"openssl-macros",
"openssl-sys",
]
[[package]]
name = "openssl-macros"
version = "0.1.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a948666b637a0f465e8564c73e89d4dde00d72d4d473cc972f390fc3dcee7d9c"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.114",
]
[[package]]
name = "openssl-probe"
version = "0.1.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d05e27ee213611ffe7d6348b942e8f942b37114c00cc03cec254295a4a17852e"
[[package]]
name = "openssl-sys"
version = "0.9.111"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "82cab2d520aa75e3c58898289429321eb788c3106963d0dc886ec7a5f4adc321"
dependencies = [
"cc",
"libc",
"pkg-config",
"vcpkg",
]
[[package]]
name = "option-ext"
version = "0.2.0"
@@ -2925,16 +3202,22 @@ checksum = "eddd3ca559203180a307f12d114c268abf583f59b03cb906fd0b3ff8646c1147"
dependencies = [
"base64 0.22.1",
"bytes",
"encoding_rs",
"futures-channel",
"futures-core",
"futures-util",
"h2",
"http",
"http-body",
"http-body-util",
"hyper",
"hyper-rustls",
"hyper-tls",
"hyper-util",
"js-sys",
"log",
"mime",
"native-tls",
"percent-encoding",
"pin-project-lite",
"quinn",
@@ -2945,6 +3228,7 @@ dependencies = [
"serde_urlencoded",
"sync_wrapper",
"tokio",
"tokio-native-tls",
"tokio-rustls",
"tokio-util",
"tower",
@@ -3080,6 +3364,15 @@ dependencies = [
"winapi-util",
]
[[package]]
name = "schannel"
version = "0.1.28"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "891d81b926048e76efe18581bf793546b4c0eaf8448d72be8de2bbee5fd166e1"
dependencies = [
"windows-sys 0.61.2",
]
[[package]]
name = "schemars"
version = "0.8.22"
@@ -3143,6 +3436,29 @@ version = "1.5.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6ccf069cb109cf8e01ebdca0d55dfce45dbbf669e8c56ed5c62150b056d3ec9f"
[[package]]
name = "security-framework"
version = "2.11.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "897b2245f0b511c87893af39b033e5ca9cce68824c4d7e7630b5a1d339658d02"
dependencies = [
"bitflags 2.10.0",
"core-foundation 0.9.4",
"core-foundation-sys",
"libc",
"security-framework-sys",
]
[[package]]
name = "security-framework-sys"
version = "2.15.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cc1f0cbffaac4852523ce30d8bd3c5cdc873501d96ff467ca09b6767bb8cd5c0"
dependencies = [
"core-foundation-sys",
"libc",
]
[[package]]
name = "selectors"
version = "0.24.0"
@@ -3448,7 +3764,7 @@ checksum = "aac18da81ebbf05109ab275b157c22a653bb3c12cf884450179942f81bcbf6c3"
dependencies = [
"bytemuck",
"js-sys",
"ndk",
"ndk 0.9.0",
"objc2",
"objc2-core-foundation",
"objc2-core-graphics",
@@ -3542,6 +3858,201 @@ dependencies = [
"serde_json",
]
[[package]]
name = "symphonia"
version = "0.5.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5773a4c030a19d9bfaa090f49746ff35c75dfddfa700df7a5939d5e076a57039"
dependencies = [
"lazy_static",
"symphonia-bundle-flac",
"symphonia-bundle-mp3",
"symphonia-codec-aac",
"symphonia-codec-adpcm",
"symphonia-codec-alac",
"symphonia-codec-pcm",
"symphonia-codec-vorbis",
"symphonia-core",
"symphonia-format-caf",
"symphonia-format-isomp4",
"symphonia-format-mkv",
"symphonia-format-ogg",
"symphonia-format-riff",
"symphonia-metadata",
]
[[package]]
name = "symphonia-bundle-flac"
version = "0.5.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c91565e180aea25d9b80a910c546802526ffd0072d0b8974e3ebe59b686c9976"
dependencies = [
"log",
"symphonia-core",
"symphonia-metadata",
"symphonia-utils-xiph",
]
[[package]]
name = "symphonia-bundle-mp3"
version = "0.5.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4872dd6bb56bf5eac799e3e957aa1981086c3e613b27e0ac23b176054f7c57ed"
dependencies = [
"lazy_static",
"log",
"symphonia-core",
"symphonia-metadata",
]
[[package]]
name = "symphonia-codec-aac"
version = "0.5.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4c263845aa86881416849c1729a54c7f55164f8b96111dba59de46849e73a790"
dependencies = [
"lazy_static",
"log",
"symphonia-core",
]
[[package]]
name = "symphonia-codec-adpcm"
version = "0.5.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2dddc50e2bbea4cfe027441eece77c46b9f319748605ab8f3443350129ddd07f"
dependencies = [
"log",
"symphonia-core",
]
[[package]]
name = "symphonia-codec-alac"
version = "0.5.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8413fa754942ac16a73634c9dfd1500ed5c61430956b33728567f667fdd393ab"
dependencies = [
"log",
"symphonia-core",
]
[[package]]
name = "symphonia-codec-pcm"
version = "0.5.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4e89d716c01541ad3ebe7c91ce4c8d38a7cf266a3f7b2f090b108fb0cb031d95"
dependencies = [
"log",
"symphonia-core",
]
[[package]]
name = "symphonia-codec-vorbis"
version = "0.5.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f025837c309cd69ffef572750b4a2257b59552c5399a5e49707cc5b1b85d1c73"
dependencies = [
"log",
"symphonia-core",
"symphonia-utils-xiph",
]
[[package]]
name = "symphonia-core"
version = "0.5.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ea00cc4f79b7f6bb7ff87eddc065a1066f3a43fe1875979056672c9ef948c2af"
dependencies = [
"arrayvec",
"bitflags 1.3.2",
"bytemuck",
"lazy_static",
"log",
]
[[package]]
name = "symphonia-format-caf"
version = "0.5.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b8faf379316b6b6e6bbc274d00e7a592e0d63ff1a7e182ce8ba25e24edd3d096"
dependencies = [
"log",
"symphonia-core",
"symphonia-metadata",
]
[[package]]
name = "symphonia-format-isomp4"
version = "0.5.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "243739585d11f81daf8dac8d9f3d18cc7898f6c09a259675fc364b382c30e0a5"
dependencies = [
"encoding_rs",
"log",
"symphonia-core",
"symphonia-metadata",
"symphonia-utils-xiph",
]
[[package]]
name = "symphonia-format-mkv"
version = "0.5.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "122d786d2c43a49beb6f397551b4a050d8229eaa54c7ddf9ee4b98899b8742d0"
dependencies = [
"lazy_static",
"log",
"symphonia-core",
"symphonia-metadata",
"symphonia-utils-xiph",
]
[[package]]
name = "symphonia-format-ogg"
version = "0.5.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2b4955c67c1ed3aa8ae8428d04ca8397fbef6a19b2b051e73b5da8b1435639cb"
dependencies = [
"log",
"symphonia-core",
"symphonia-metadata",
"symphonia-utils-xiph",
]
[[package]]
name = "symphonia-format-riff"
version = "0.5.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c2d7c3df0e7d94efb68401d81906eae73c02b40d5ec1a141962c592d0f11a96f"
dependencies = [
"extended",
"log",
"symphonia-core",
"symphonia-metadata",
]
[[package]]
name = "symphonia-metadata"
version = "0.5.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "36306ff42b9ffe6e5afc99d49e121e0bd62fe79b9db7b9681d48e29fa19e6b16"
dependencies = [
"encoding_rs",
"lazy_static",
"log",
"symphonia-core",
]
[[package]]
name = "symphonia-utils-xiph"
version = "0.5.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ee27c85ab799a338446b68eec77abf42e1a6f1bb490656e121c6e27bfbab9f16"
dependencies = [
"symphonia-core",
"symphonia-metadata",
]
[[package]]
name = "syn"
version = "1.0.109"
@@ -3584,6 +4095,27 @@ dependencies = [
"syn 2.0.114",
]
[[package]]
name = "system-configuration"
version = "0.6.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3c879d448e9d986b661742763247d3693ed13609438cf3d006f51f5368a5ba6b"
dependencies = [
"bitflags 2.10.0",
"core-foundation 0.9.4",
"system-configuration-sys",
]
[[package]]
name = "system-configuration-sys"
version = "0.6.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8e1d1b10ced5ca923a1fcb8d03e96b8d3268065d724548c0211415ff6ac6bac4"
dependencies = [
"core-foundation-sys",
"libc",
]
[[package]]
name = "system-deps"
version = "6.2.2"
@@ -3605,7 +4137,7 @@ checksum = "f3a753bdc39c07b192151523a3f77cd0394aa75413802c883a0f6f6a0e5ee2e7"
dependencies = [
"bitflags 2.10.0",
"block2",
"core-foundation",
"core-foundation 0.10.1",
"core-graphics",
"crossbeam-channel",
"dispatch",
@@ -3618,9 +4150,9 @@ dependencies = [
"lazy_static",
"libc",
"log",
"ndk",
"ndk 0.9.0",
"ndk-context",
"ndk-sys",
"ndk-sys 0.6.0+11769913",
"objc2",
"objc2-app-kit",
"objc2-foundation",
@@ -3836,6 +4368,16 @@ dependencies = [
"url",
]
[[package]]
name = "tauri-plugin-process"
version = "2.3.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d55511a7bf6cd70c8767b02c97bf8134fa434daf3926cfc1be0a0f94132d165a"
dependencies = [
"tauri",
"tauri-plugin",
]
[[package]]
name = "tauri-plugin-shell"
version = "2.3.4"
@@ -4138,6 +4680,16 @@ dependencies = [
"syn 2.0.114",
]
[[package]]
name = "tokio-native-tls"
version = "0.3.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "bbae76ab933c85776efabc971569dd6119c580d8f5d448769dec1764bf796ef2"
dependencies = [
"native-tls",
"tokio",
]
[[package]]
name = "tokio-rustls"
version = "0.26.4"
@@ -4469,6 +5021,12 @@ dependencies = [
"wasm-bindgen",
]
[[package]]
name = "vcpkg"
version = "0.2.15"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "accd4ea62f7bb7a82fe23066fb0957d48ef677f6eeb8215f372f52e48bb32426"
[[package]]
name = "version-compare"
version = "0.2.1"
@@ -4483,21 +5041,25 @@ checksum = "0b928f33d975fc6ad9f86c8f283853ad26bdd5b10b7f1542aa2fa15e2289105a"
[[package]]
name = "voicebox"
version = "0.1.1"
version = "0.1.8"
dependencies = [
"base64 0.22.1",
"core-foundation-sys",
"coreaudio-sys",
"cpal",
"hound",
"objc",
"reqwest",
"scopeguard",
"screencapturekit",
"serde",
"serde_json",
"symphonia",
"tauri",
"tauri-build",
"tauri-plugin-dialog",
"tauri-plugin-fs",
"tauri-plugin-process",
"tauri-plugin-shell",
"tauri-plugin-updater",
"tokio",
@@ -4805,6 +5367,16 @@ dependencies = [
"windows-version",
]
[[package]]
name = "windows"
version = "0.54.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9252e5725dbed82865af151df558e754e4a3c2c30818359eb17465f1346a1b49"
dependencies = [
"windows-core 0.54.0",
"windows-targets 0.52.6",
]
[[package]]
name = "windows"
version = "0.61.3"
@@ -4848,6 +5420,16 @@ dependencies = [
"windows-core 0.62.2",
]
[[package]]
name = "windows-core"
version = "0.54.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "12661b9c89351d684a50a8a643ce5f608e20243b9fb84687800163429f161d65"
dependencies = [
"windows-result 0.1.2",
"windows-targets 0.52.6",
]
[[package]]
name = "windows-core"
version = "0.61.2"
@@ -4950,6 +5532,26 @@ dependencies = [
"windows-link 0.2.1",
]
[[package]]
name = "windows-registry"
version = "0.6.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "02752bf7fbdcce7f2a27a742f798510f3e5ad88dbe84871e5168e2120c3d5720"
dependencies = [
"windows-link 0.2.1",
"windows-result 0.4.1",
"windows-strings 0.5.1",
]
[[package]]
name = "windows-result"
version = "0.1.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5e383302e8ec8515204254685643de10811af0ed97ea37210dc26fb0032647f8"
dependencies = [
"windows-targets 0.52.6",
]
[[package]]
name = "windows-result"
version = "0.3.4"
@@ -5305,7 +5907,7 @@ dependencies = [
"jni",
"kuchikiki",
"libc",
"ndk",
"ndk 0.9.0",
"objc2",
"objc2-app-kit",
"objc2-core-foundation",
+5 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "voicebox"
version = "0.1.2"
version = "0.1.8"
description = "A production-quality desktop app for Qwen3-TTS voice cloning and generation"
authors = ["you"]
license = ""
@@ -20,8 +20,11 @@ tauri-plugin-shell = "2.0"
serde = { version = "1.0", features = ["derive"] }
serde_json = "1.0"
tokio = { version = "1", features = ["full"] }
reqwest = { version = "0.12", features = ["blocking", "json"] }
hound = "3.5"
base64 = "0.22"
cpal = "0.15"
symphonia = { version = "0.5", features = ["all"] }
scopeguard = "1.2.0"
[target.'cfg(target_os = "macos")'.dependencies]
@@ -36,6 +39,7 @@ windows = { version = "0.62", features = ["Win32_Foundation", "Win32_UI_WindowsA
[target.'cfg(not(any(target_os = "android", target_os = "ios")))'.dependencies]
tauri-plugin-updater = "2.0"
tauri-plugin-process = "2.0"
[features]
# This feature is used for production builds or when `devPath` points to the filesystem
@@ -17,6 +17,7 @@
"shell:allow-execute",
"shell:allow-spawn",
"updater:default",
"process:default",
"dialog:default",
"dialog:allow-save",
"dialog:allow-open",
Binary file not shown.
File diff suppressed because one or more lines are too long
@@ -1 +1 @@
{"default":{"identifier":"default","description":"Default permissions for voicebox","remote":{"urls":["http://localhost:*"]},"local":true,"windows":["main"],"permissions":["core:default","core:window:default","core:window:allow-start-dragging","core:webview:default","core:webview:allow-internal-toggle-devtools","shell:allow-open","shell:allow-execute","shell:allow-spawn","updater:default","dialog:default","dialog:allow-save","dialog:allow-open","fs:default","fs:read-all","fs:write-all"],"platforms":["linux","macOS","windows"]}}
{"default":{"identifier":"default","description":"Default permissions for voicebox","remote":{"urls":["http://localhost:*"]},"local":true,"windows":["main"],"permissions":["core:default","core:window:default","core:window:allow-start-dragging","core:webview:default","core:webview:allow-internal-toggle-devtools","shell:allow-open","shell:allow-execute","shell:allow-spawn","updater:default","process:default","dialog:default","dialog:allow-save","dialog:allow-open","fs:default","fs:read-all","fs:write-all"],"platforms":["linux","macOS","windows"]}}
@@ -5948,6 +5948,36 @@
"const": "fs:write-files",
"markdownDescription": "This enables all file write related commands without any pre-configured accessible paths."
},
{
"description": "This permission set configures which\nprocess features are by default exposed.\n\n#### Granted Permissions\n\nThis enables to quit via `allow-exit` and restart via `allow-restart`\nthe application.\n\n#### This default permission set includes:\n\n- `allow-exit`\n- `allow-restart`",
"type": "string",
"const": "process:default",
"markdownDescription": "This permission set configures which\nprocess features are by default exposed.\n\n#### Granted Permissions\n\nThis enables to quit via `allow-exit` and restart via `allow-restart`\nthe application.\n\n#### This default permission set includes:\n\n- `allow-exit`\n- `allow-restart`"
},
{
"description": "Enables the exit command without any pre-configured scope.",
"type": "string",
"const": "process:allow-exit",
"markdownDescription": "Enables the exit command without any pre-configured scope."
},
{
"description": "Enables the restart command without any pre-configured scope.",
"type": "string",
"const": "process:allow-restart",
"markdownDescription": "Enables the restart command without any pre-configured scope."
},
{
"description": "Denies the exit command without any pre-configured scope.",
"type": "string",
"const": "process:deny-exit",
"markdownDescription": "Denies the exit command without any pre-configured scope."
},
{
"description": "Denies the restart command without any pre-configured scope.",
"type": "string",
"const": "process:deny-restart",
"markdownDescription": "Denies the restart command without any pre-configured scope."
},
{
"description": "This permission set configures which\nshell functionality is exposed by default.\n\n#### Granted Permissions\n\nIt allows to use the `open` functionality with a reasonable\nscope pre-configured. It will allow opening `http(s)://`,\n`tel:` and `mailto:` links.\n\n#### This default permission set includes:\n\n- `allow-open`",
"type": "string",
@@ -5948,6 +5948,36 @@
"const": "fs:write-files",
"markdownDescription": "This enables all file write related commands without any pre-configured accessible paths."
},
{
"description": "This permission set configures which\nprocess features are by default exposed.\n\n#### Granted Permissions\n\nThis enables to quit via `allow-exit` and restart via `allow-restart`\nthe application.\n\n#### This default permission set includes:\n\n- `allow-exit`\n- `allow-restart`",
"type": "string",
"const": "process:default",
"markdownDescription": "This permission set configures which\nprocess features are by default exposed.\n\n#### Granted Permissions\n\nThis enables to quit via `allow-exit` and restart via `allow-restart`\nthe application.\n\n#### This default permission set includes:\n\n- `allow-exit`\n- `allow-restart`"
},
{
"description": "Enables the exit command without any pre-configured scope.",
"type": "string",
"const": "process:allow-exit",
"markdownDescription": "Enables the exit command without any pre-configured scope."
},
{
"description": "Enables the restart command without any pre-configured scope.",
"type": "string",
"const": "process:allow-restart",
"markdownDescription": "Enables the restart command without any pre-configured scope."
},
{
"description": "Denies the exit command without any pre-configured scope.",
"type": "string",
"const": "process:deny-exit",
"markdownDescription": "Denies the exit command without any pre-configured scope."
},
{
"description": "Denies the restart command without any pre-configured scope.",
"type": "string",
"const": "process:deny-restart",
"markdownDescription": "Denies the restart command without any pre-configured scope."
},
{
"description": "This permission set configures which\nshell functionality is exposed by default.\n\n#### Granted Permissions\n\nIt allows to use the `open` functionality with a reasonable\nscope pre-configured. It will allow opening `http(s)://`,\n`tel:` and `mailto:` links.\n\n#### This default permission set includes:\n\n- `allow-open`",
"type": "string",
@@ -5948,6 +5948,36 @@
"const": "fs:write-files",
"markdownDescription": "This enables all file write related commands without any pre-configured accessible paths."
},
{
"description": "This permission set configures which\nprocess features are by default exposed.\n\n#### Granted Permissions\n\nThis enables to quit via `allow-exit` and restart via `allow-restart`\nthe application.\n\n#### This default permission set includes:\n\n- `allow-exit`\n- `allow-restart`",
"type": "string",
"const": "process:default",
"markdownDescription": "This permission set configures which\nprocess features are by default exposed.\n\n#### Granted Permissions\n\nThis enables to quit via `allow-exit` and restart via `allow-restart`\nthe application.\n\n#### This default permission set includes:\n\n- `allow-exit`\n- `allow-restart`"
},
{
"description": "Enables the exit command without any pre-configured scope.",
"type": "string",
"const": "process:allow-exit",
"markdownDescription": "Enables the exit command without any pre-configured scope."
},
{
"description": "Enables the restart command without any pre-configured scope.",
"type": "string",
"const": "process:allow-restart",
"markdownDescription": "Enables the restart command without any pre-configured scope."
},
{
"description": "Denies the exit command without any pre-configured scope.",
"type": "string",
"const": "process:deny-exit",
"markdownDescription": "Denies the exit command without any pre-configured scope."
},
{
"description": "Denies the restart command without any pre-configured scope.",
"type": "string",
"const": "process:deny-restart",
"markdownDescription": "Denies the restart command without any pre-configured scope."
},
{
"description": "This permission set configures which\nshell functionality is exposed by default.\n\n#### Granted Permissions\n\nIt allows to use the `open` functionality with a reasonable\nscope pre-configured. It will allow opening `http(s)://`,\n`tel:` and `mailto:` links.\n\n#### This default permission set includes:\n\n- `allow-open`",
"type": "string",
+468
View File
@@ -0,0 +1,468 @@
use cpal::traits::{DeviceTrait, HostTrait, StreamTrait};
use cpal::{Device, Host, SampleFormat, Stream, StreamConfig};
use std::sync::{Arc, Mutex};
use std::sync::atomic::{AtomicBool, AtomicUsize, Ordering};
#[derive(Debug, Clone, serde::Serialize)]
pub struct AudioOutputDevice {
pub id: String,
pub name: String,
pub is_default: bool,
}
pub struct AudioOutputState {
host: Host,
stop_flag: Arc<AtomicBool>,
}
impl AudioOutputState {
pub fn new() -> Self {
Self {
host: cpal::default_host(),
stop_flag: Arc::new(AtomicBool::new(false)),
}
}
pub fn stop_all_playback(&self) -> Result<(), String> {
eprintln!("stop_all_playback: Setting stop flag");
self.stop_flag.store(true, Ordering::Relaxed);
eprintln!("stop_all_playback: Stop flag set - active streams will output silence");
Ok(())
}
pub fn list_output_devices(&self) -> Result<Vec<AudioOutputDevice>, String> {
let devices = self
.host
.output_devices()
.map_err(|e| format!("Failed to enumerate output devices: {}", e))?;
let default_device = self.host.default_output_device();
let mut result = Vec::new();
for device in devices {
let name = device
.name()
.map_err(|e| format!("Failed to get device name: {}", e))?;
// Generate a stable ID from the device name (cpal doesn't provide stable IDs)
let id = format!("device_{}", name.replace(' ', "_").to_lowercase());
let is_default = default_device
.as_ref()
.map(|d| d.name().unwrap_or_default() == name)
.unwrap_or(false);
result.push(AudioOutputDevice {
id,
name,
is_default,
});
}
Ok(result)
}
pub async fn play_audio_to_devices(
&self,
audio_data: Vec<u8>,
device_ids: Vec<String>,
) -> Result<(), String> {
eprintln!("play_audio_to_devices called with {} bytes, {} device IDs", audio_data.len(), device_ids.len());
eprintln!("Requested device IDs: {:?}", device_ids);
// Decode audio file (assuming WAV format)
eprintln!("Decoding audio data...");
let (samples, sample_rate, channels) = self.decode_wav(&audio_data)?;
eprintln!("Audio decoded: {} samples, {}Hz, {} channels", samples.len(), sample_rate, channels);
// Find devices by ID
eprintln!("Enumerating output devices...");
let devices: Vec<Device> = self
.host
.output_devices()
.map_err(|e| format!("Failed to enumerate devices: {}", e))?
.filter_map(|device| {
let name = device.name().ok()?;
let id = format!("device_{}", name.replace(' ', "_").to_lowercase());
eprintln!("Found device: {} (id: {})", name, id);
if device_ids.contains(&id) {
eprintln!(" -> Matched! Will play to this device");
Some(device)
} else {
None
}
})
.collect();
if devices.is_empty() {
eprintln!("ERROR: No matching devices found");
return Err("No matching devices found".to_string());
}
eprintln!("Playing to {} device(s)", devices.len());
// Stop any existing playback first
self.stop_all_playback().ok();
// Reset stop flag for new playback
self.stop_flag.store(false, Ordering::Relaxed);
// Play to each device
for (i, device) in devices.iter().enumerate() {
let device_name = device.name().unwrap_or_else(|_| "unknown".to_string());
eprintln!("Playing to device {}/{}: {}", i + 1, devices.len(), device_name);
self.play_to_device(device, samples.clone(), sample_rate, channels, self.stop_flag.clone())
.map_err(|e| format!("Failed to play to device {}: {}", device_name, e))?;
eprintln!("Successfully started playback on device: {}", device_name);
}
eprintln!("play_audio_to_devices completed successfully");
Ok(())
}
fn decode_wav(&self, data: &[u8]) -> Result<(Vec<f32>, u32, u16), String> {
use symphonia::core::formats::FormatOptions;
use symphonia::core::io::MediaSourceStream;
use symphonia::core::meta::MetadataOptions;
eprintln!("decode_wav: Creating MediaSourceStream from {} bytes", data.len());
let mss = MediaSourceStream::new(
Box::new(std::io::Cursor::new(data.to_vec())),
Default::default(),
);
eprintln!("decode_wav: Probing audio format...");
let mut format = symphonia::default::get_probe()
.format(
&Default::default(),
mss,
&FormatOptions::default(),
&MetadataOptions::default(),
)
.map_err(|e| {
eprintln!("decode_wav: Failed to probe audio: {}", e);
format!("Failed to probe audio: {}", e)
})?
.format;
eprintln!("decode_wav: Audio format probed successfully");
eprintln!("decode_wav: Finding audio track...");
let track = format
.tracks()
.iter()
.find(|t| t.codec_params.codec != symphonia::core::codecs::CODEC_TYPE_NULL)
.ok_or_else(|| {
eprintln!("decode_wav: No audio track found");
"No audio track found".to_string()
})?;
let sample_rate = track
.codec_params
.sample_rate
.ok_or_else(|| {
eprintln!("decode_wav: No sample rate found in track");
"No sample rate found".to_string()
})?;
let channels = track
.codec_params
.channels
.ok_or_else(|| {
eprintln!("decode_wav: No channels found in track");
"No channels found".to_string()
})?
.count() as u16;
eprintln!("decode_wav: Track info - sample_rate: {}, channels: {}", sample_rate, channels);
eprintln!("decode_wav: Creating decoder...");
let mut decoder = symphonia::default::get_codecs()
.make(&track.codec_params, &Default::default())
.map_err(|e| {
eprintln!("decode_wav: Failed to create decoder: {}", e);
format!("Failed to create decoder: {}", e)
})?;
eprintln!("decode_wav: Decoder created successfully");
let mut samples = Vec::new();
let mut packet_count = 0;
eprintln!("decode_wav: Starting packet decoding loop...");
loop {
let packet = match format.next_packet() {
Ok(packet) => packet,
Err(e) => {
eprintln!("decode_wav: End of stream or error: {:?}", e);
break;
}
};
packet_count += 1;
let decoded = decoder
.decode(&packet)
.map_err(|e| {
eprintln!("decode_wav: Decode error on packet {}: {}", packet_count, e);
format!("Decode error: {}", e)
})?;
// Convert to f32 samples by matching on the buffer type
use symphonia::core::audio::{AudioBufferRef, Signal};
use symphonia::core::conv::FromSample;
let spec = *decoded.spec();
let num_channels = spec.channels.count();
let num_frames = decoded.frames();
eprintln!("decode_wav: Packet {} - {} frames, {} channels", packet_count, num_frames, num_channels);
// Interleave samples from all channels
for frame_idx in 0..num_frames {
for ch in 0..num_channels {
let sample_f32 = match &decoded {
AudioBufferRef::U8(buf) => f32::from_sample(buf.chan(ch)[frame_idx]),
AudioBufferRef::U16(buf) => f32::from_sample(buf.chan(ch)[frame_idx]),
AudioBufferRef::U24(buf) => f32::from_sample(buf.chan(ch)[frame_idx]),
AudioBufferRef::U32(buf) => f32::from_sample(buf.chan(ch)[frame_idx]),
AudioBufferRef::S8(buf) => f32::from_sample(buf.chan(ch)[frame_idx]),
AudioBufferRef::S16(buf) => f32::from_sample(buf.chan(ch)[frame_idx]),
AudioBufferRef::S24(buf) => f32::from_sample(buf.chan(ch)[frame_idx]),
AudioBufferRef::S32(buf) => f32::from_sample(buf.chan(ch)[frame_idx]),
AudioBufferRef::F32(buf) => buf.chan(ch)[frame_idx],
AudioBufferRef::F64(buf) => buf.chan(ch)[frame_idx] as f32,
};
samples.push(sample_f32);
}
}
}
eprintln!("decode_wav: Decoded {} packets, total {} samples", packet_count, samples.len());
eprintln!("decode_wav: Returning sample_rate={}, channels={}", sample_rate, channels);
Ok((samples, sample_rate, channels))
}
fn play_to_device(
&self,
device: &Device,
samples: Vec<f32>,
sample_rate: u32,
channels: u16,
stop_flag: Arc<AtomicBool>,
) -> Result<(), String> {
let device_name = device.name().unwrap_or_else(|_| "unknown".to_string());
eprintln!("play_to_device: Starting playback to device: {}", device_name);
eprintln!("play_to_device: Input - {} samples, {}Hz, {} channels", samples.len(), sample_rate, channels);
let config = device
.default_output_config()
.map_err(|e| format!("Failed to get default config: {}", e))?;
// Prepare samples for the device's format
let device_sample_rate = config.sample_rate().0;
let device_channels = config.channels();
let device_sample_format = config.sample_format();
eprintln!("play_to_device: Device config - {}Hz, {} channels, format: {:?}",
device_sample_rate, device_channels, device_sample_format);
// Resample if needed (simple linear interpolation for now)
let resampled = if device_sample_rate != sample_rate {
eprintln!("play_to_device: Resampling from {}Hz to {}Hz", sample_rate, device_sample_rate);
let result = self.resample(&samples, sample_rate, device_sample_rate);
eprintln!("play_to_device: Resampled {} samples to {} samples", samples.len(), result.len());
result
} else {
eprintln!("play_to_device: No resampling needed");
samples
};
// Interleave/convert channels if needed
eprintln!("play_to_device: Interleaving channels from {} to {} channels", channels, device_channels);
let interleaved = self.interleave_channels(&resampled, channels, device_channels);
eprintln!("play_to_device: Interleaved to {} samples", interleaved.len());
// Calculate duration before moving interleaved
let duration_secs = (interleaved.len() as f64 / (device_sample_rate as f64 * device_channels as f64)).ceil() as u64 + 1;
// Create shared buffer for playback
let buffer: Arc<Mutex<Vec<f32>>> = Arc::new(Mutex::new(interleaved));
let position = Arc::new(AtomicUsize::new(0));
let buffer_clone = buffer.clone();
let position_clone = position.clone();
let err_fn = |err| eprintln!("Playback error: {}", err);
let stream_config = StreamConfig {
channels: device_channels,
sample_rate: cpal::SampleRate(device_sample_rate),
buffer_size: cpal::BufferSize::Default,
};
let stop_flag_clone = stop_flag.clone();
let stream = match config.sample_format() {
SampleFormat::F32 => {
let buffer = buffer_clone.clone();
let pos = position_clone.clone();
device
.build_output_stream(
&stream_config,
move |data: &mut [f32], _: &cpal::OutputCallbackInfo| {
// Check stop flag - if set, output silence
if stop_flag_clone.load(Ordering::Relaxed) {
for sample in data.iter_mut() {
*sample = 0.0;
}
return;
}
let mut idx = pos.load(Ordering::Relaxed);
let buf = buffer.lock().unwrap();
for sample in data.iter_mut() {
if idx < buf.len() {
*sample = buf[idx];
idx += 1;
} else {
*sample = 0.0;
}
}
pos.store(idx, Ordering::Relaxed);
},
err_fn,
None,
)
.map_err(|e| format!("Failed to build stream: {}", e))?
}
SampleFormat::I16 => {
let buffer = buffer_clone.clone();
let pos = position_clone.clone();
device
.build_output_stream(
&stream_config,
move |data: &mut [i16], _: &cpal::OutputCallbackInfo| {
// Check stop flag - if set, output silence
if stop_flag_clone.load(Ordering::Relaxed) {
for sample in data.iter_mut() {
*sample = 0;
}
return;
}
let mut idx = pos.load(Ordering::Relaxed);
let buf = buffer.lock().unwrap();
for sample in data.iter_mut() {
if idx < buf.len() {
*sample = (buf[idx] * 32767.0) as i16;
idx += 1;
} else {
*sample = 0;
}
}
pos.store(idx, Ordering::Relaxed);
},
err_fn,
None,
)
.map_err(|e| format!("Failed to build stream: {}", e))?
}
SampleFormat::U16 => {
let buffer = buffer_clone.clone();
let pos = position_clone.clone();
device
.build_output_stream(
&stream_config,
move |data: &mut [u16], _: &cpal::OutputCallbackInfo| {
// Check stop flag - if set, output silence
if stop_flag_clone.load(Ordering::Relaxed) {
for sample in data.iter_mut() {
*sample = 32768;
}
return;
}
let mut idx = pos.load(Ordering::Relaxed);
let buf = buffer.lock().unwrap();
for sample in data.iter_mut() {
if idx < buf.len() {
*sample = ((buf[idx] + 1.0) * 32767.5) as u16;
idx += 1;
} else {
*sample = 32768;
}
}
pos.store(idx, Ordering::Relaxed);
},
err_fn,
None,
)
.map_err(|e| format!("Failed to build stream: {}", e))?
}
_ => return Err("Unsupported sample format".to_string()),
};
eprintln!("play_to_device: Starting stream playback...");
stream.play().map_err(|e| {
eprintln!("play_to_device: Failed to play stream: {}", e);
format!("Failed to play stream: {}", e)
})?;
eprintln!("play_to_device: Stream started successfully");
eprintln!("play_to_device: Function completed successfully");
Ok(())
}
fn resample(&self, samples: &[f32], from_rate: u32, to_rate: u32) -> Vec<f32> {
if from_rate == to_rate {
return samples.to_vec();
}
let ratio = to_rate as f64 / from_rate as f64;
let new_len = (samples.len() as f64 * ratio) as usize;
let mut resampled = Vec::with_capacity(new_len);
for i in 0..new_len {
let src_idx = (i as f64 / ratio) as usize;
if src_idx < samples.len() {
resampled.push(samples[src_idx]);
} else {
resampled.push(0.0);
}
}
resampled
}
fn interleave_channels(
&self,
samples: &[f32],
src_channels: u16,
dst_channels: u16,
) -> Vec<f32> {
if src_channels == dst_channels {
return samples.to_vec();
}
let mut interleaved = Vec::new();
let samples_per_channel = samples.len() / src_channels as usize;
for i in 0..samples_per_channel {
for ch in 0..dst_channels {
let src_ch = if ch < src_channels { ch } else { src_channels - 1 };
let idx = (i * src_channels as usize) + src_ch as usize;
if idx < samples.len() {
interleaved.push(samples[idx]);
} else {
interleaved.push(0.0);
}
}
}
interleaved
}
}
impl Default for AudioOutputState {
fn default() -> Self {
Self::new()
}
}
+187 -13
View File
@@ -2,6 +2,7 @@
#![cfg_attr(not(debug_assertions), windows_subsystem = "windows")]
mod audio_capture;
mod audio_output;
use std::sync::Mutex;
use tauri::{command, State, Manager, WindowEvent, Emitter, Listener, RunEvent};
@@ -302,6 +303,60 @@ async fn start_server(
Ok(format!("http://127.0.0.1:{}", SERVER_PORT))
}
/// Check if a Windows process is still running
#[cfg(windows)]
fn is_process_running(pid: u32) -> bool {
use std::process::Command;
if let Ok(output) = Command::new("tasklist")
.args(["/FI", &format!("PID eq {}", pid), "/FO", "CSV", "/NH"])
.output()
{
// If process exists, tasklist returns it in output
let output_str = String::from_utf8_lossy(&output.stdout);
return !output_str.trim().is_empty() && output_str.contains(&pid.to_string());
}
false
}
/// Kill entire Windows process tree by enumerating children
#[cfg(windows)]
fn kill_windows_process_tree(parent_pid: u32) -> Result<(), String> {
use std::process::Command;
// Find all child processes using WMIC
let output = Command::new("wmic")
.args([
"process",
"where",
&format!("ParentProcessId={}", parent_pid),
"get",
"ProcessId"
])
.output();
if let Ok(output) = output {
let output_str = String::from_utf8_lossy(&output.stdout);
for line in output_str.lines().skip(1) { // Skip header
if let Ok(child_pid) = line.trim().parse::<u32>() {
println!("Found child process: {}", child_pid);
// Recursively kill child's children
let _ = kill_windows_process_tree(child_pid);
// Kill the child
let _ = Command::new("taskkill")
.args(["/PID", &child_pid.to_string(), "/F"])
.output();
}
}
}
// Kill the parent process
let _ = Command::new("taskkill")
.args(["/PID", &parent_pid.to_string(), "/F"])
.output();
Ok(())
}
#[command]
async fn stop_server(state: State<'_, ServerState>) -> Result<(), String> {
let pid = state.server_pid.lock().unwrap().take();
@@ -331,13 +386,59 @@ async fn stop_server(state: State<'_, ServerState>) -> Result<(), String> {
#[cfg(windows)]
{
use std::process::Command;
let _ = Command::new("taskkill")
.args(["/PID", &pid.to_string(), "/T", "/F"])
.output();
// Layer 1: Try graceful HTTP shutdown first
println!("Attempting graceful shutdown via HTTP...");
let client = reqwest::blocking::Client::builder()
.timeout(std::time::Duration::from_secs(2))
.build()
.unwrap();
let shutdown_result = client
.post(&format!("http://127.0.0.1:{}/shutdown", SERVER_PORT))
.send();
if shutdown_result.is_ok() {
println!("HTTP shutdown sent, waiting for graceful exit...");
// Wait up to 3 seconds for graceful shutdown
for i in 0..30 {
std::thread::sleep(std::time::Duration::from_millis(100));
if !is_process_running(pid) {
println!("Process exited gracefully after {}ms", i * 100);
return Ok(());
}
}
println!("Graceful shutdown timed out, forcing kill...");
} else {
println!("HTTP shutdown failed, forcing kill...");
}
// Layer 2: Kill process tree with enumeration
println!("Killing process tree for wrapper PID {}...", pid);
kill_windows_process_tree(pid)?;
// Layer 3: Verify and kill by name if still running
std::thread::sleep(std::time::Duration::from_millis(200));
if is_process_running(pid) {
println!("Process tree kill failed, killing by name...");
use std::process::Command;
let _ = Command::new("taskkill")
.args(["/IM", "voicebox-server.exe", "/T", "/F"])
.output();
}
// Layer 4: Final verification
std::thread::sleep(std::time::Duration::from_millis(200));
if is_process_running(pid) {
eprintln!("WARNING: Failed to kill server after all attempts");
} else {
println!("Server killed successfully");
}
}
#[cfg(unix)]
{
println!("stop_server: Process group kill completed");
}
println!("stop_server: Process group kill completed");
}
Ok(())
@@ -368,6 +469,29 @@ fn is_system_audio_supported() -> bool {
audio_capture::is_supported()
}
#[command]
fn list_audio_output_devices(
state: State<'_, audio_output::AudioOutputState>,
) -> Result<Vec<audio_output::AudioOutputDevice>, String> {
state.list_output_devices()
}
#[command]
async fn play_audio_to_devices(
state: State<'_, audio_output::AudioOutputState>,
audio_data: Vec<u8>,
device_ids: Vec<String>,
) -> Result<(), String> {
state.play_audio_to_devices(audio_data, device_ids).await
}
#[command]
fn stop_audio_playback(
state: State<'_, audio_output::AudioOutputState>,
) -> Result<(), String> {
state.stop_all_playback()
}
#[cfg_attr(mobile, tauri::mobile_entry_point)]
pub fn run() {
tauri::Builder::default()
@@ -380,9 +504,13 @@ pub fn run() {
keep_running_on_close: Mutex::new(false),
})
.manage(audio_capture::AudioCaptureState::new())
.manage(audio_output::AudioOutputState::new())
.setup(|app| {
#[cfg(desktop)]
app.handle().plugin(tauri_plugin_updater::Builder::new().build())?;
{
app.handle().plugin(tauri_plugin_updater::Builder::new().build())?;
app.handle().plugin(tauri_plugin_process::init())?;
}
// Hide title bar icon on Windows
#[cfg(windows)]
@@ -420,7 +548,10 @@ pub fn run() {
set_keep_server_running,
start_system_audio_capture,
stop_system_audio_capture,
is_system_audio_supported
is_system_audio_supported,
list_audio_output_devices,
play_audio_to_devices,
stop_audio_playback
])
.on_window_event(|window, event| {
if let WindowEvent::CloseRequested { api, .. } = event {
@@ -529,11 +660,54 @@ pub fn run() {
#[cfg(windows)]
{
// On Windows, use taskkill with /T to kill child processes
use std::process::Command;
let _ = Command::new("taskkill")
.args(["/PID", &pid.to_string(), "/T", "/F"])
.output();
// Layer 1: Try graceful HTTP shutdown first
println!("Attempting graceful shutdown via HTTP...");
let client = reqwest::blocking::Client::builder()
.timeout(std::time::Duration::from_secs(2))
.build()
.unwrap();
let shutdown_result = client
.post(&format!("http://127.0.0.1:{}/shutdown", SERVER_PORT))
.send();
if shutdown_result.is_ok() {
println!("HTTP shutdown sent, waiting for graceful exit...");
// Wait up to 3 seconds for graceful shutdown
for i in 0..30 {
std::thread::sleep(std::time::Duration::from_millis(100));
if !is_process_running(pid) {
println!("Process exited gracefully after {}ms", i * 100);
println!("Server process tree kill completed");
return;
}
}
println!("Graceful shutdown timed out, forcing kill...");
} else {
println!("HTTP shutdown failed, forcing kill...");
}
// Layer 2: Kill process tree with enumeration
println!("Killing process tree for wrapper PID {}...", pid);
let _ = kill_windows_process_tree(pid);
// Layer 3: Verify and kill by name if still running
std::thread::sleep(std::time::Duration::from_millis(200));
if is_process_running(pid) {
println!("Process tree kill failed, killing by name...");
use std::process::Command;
let _ = Command::new("taskkill")
.args(["/IM", "voicebox-server.exe", "/T", "/F"])
.output();
}
// Layer 4: Final verification
std::thread::sleep(std::time::Duration::from_millis(200));
if is_process_running(pid) {
eprintln!("WARNING: Failed to kill server after all attempts");
} else {
println!("Server killed successfully");
}
println!("Server process tree kill completed");
}
} else {
+1 -1
View File
@@ -1,7 +1,7 @@
{
"$schema": "https://schema.tauri.app/config/2",
"productName": "Voicebox",
"version": "0.1.2",
"version": "0.1.8",
"identifier": "sh.voicebox.app",
"build": {
"beforeDevCommand": "bun run dev",
+1 -1
View File
@@ -1,7 +1,7 @@
{
"name": "@voicebox/web",
"private": true,
"version": "0.1.2",
"version": "0.1.8",
"type": "module",
"scripts": {
"dev": "vite",