diff --git a/.github/assets/icon-dark.webp b/.github/assets/icon-dark.webp new file mode 100644 index 00000000..0e0c5830 Binary files /dev/null and b/.github/assets/icon-dark.webp differ diff --git a/.github/assets/screenshot.webp b/.github/assets/screenshot.webp new file mode 100644 index 00000000..4fe03fce Binary files /dev/null and b/.github/assets/screenshot.webp differ diff --git a/README.md b/README.md index eae6b2f0..81468f92 100644 --- a/README.md +++ b/README.md @@ -1,409 +1,234 @@ -# voicebox +

+ Voicebox +

-A production-quality desktop app for Qwen3-TTS voice cloning and generation. +

Voicebox

-**Domain:** voicebox.sh +

+ The open-source voice synthesis studio.
+ Clone voices. Generate speech. Build voice-powered apps.
+ All running locally on your machine. +

+ +

+ Website • + Download • + Features • + API • + Roadmap +

+ +
+ +

+ Voicebox App Screenshot +

+ +
+ +## Why Voicebox? + +Voice AI is exploding, but most tools are either cloud-locked, expensive, or a nightmare to set up. Voicebox is different: + +- **100% Local** — Your voice data never leaves your machine +- **Lightweight** — ~50MB app, runs on modest hardware +- **Fast** — Sub-2-second generation with caching +- **Flexible** — Use the app, integrate the API, or both +- **Open Source** — No subscriptions, no limits, no lock-in + +Built with **Tauri** (Rust), **TypeScript**, **React**, and **Python**. Native performance meets modern DX. --- -## Vision +## Download -Qwen3-TTS is a breakthrough model from Alibaba that achieves near-perfect voice cloning. The existing implementations (Voice-Clone-Studio, mimic, etc.) are either feature-rich but architecturally messy, or well-structured but limited in scope. +Voicebox is available now for macOS, Windows, and Linux. -voicebox aims to build the definitive Qwen3-TTS application by combining the best patterns from existing projects while avoiding their architectural mistakes. +| Platform | Download | +|----------|----------| +| macOS (Apple Silicon) | [voicebox-macos-arm64.dmg](https://github.com/voicebox-sh/voicebox/releases/latest) | +| macOS (Intel) | [voicebox-macos-x64.dmg](https://github.com/voicebox-sh/voicebox/releases/latest) | +| Windows | [voicebox-windows-x64.exe](https://github.com/voicebox-sh/voicebox/releases/latest) | +| Linux | [voicebox-linux-x64.AppImage](https://github.com/voicebox-sh/voicebox/releases/latest) | -## Design Principles +--- -1. **Clean architecture from day one** - No monolithic files, proper separation of concerns -2. **Desktop-first experience** - Native feel via Tauri, not a web app in disguise -3. **Production code quality** - Type safety, modularity, maintainability -4. **Performance and UX** - Smart caching, async operations, responsive UI -5. **Extensible design** - Easy to add new models, effects, and features -6. **Flexible deployment** - Run backend locally or connect to remote GPU machine with one click +## Features -## Technology Stack +### Voice Cloning with Qwen3-TTS -### Backend (Python) -- **FastAPI** - Async REST API -- **SQLAlchemy** - Database ORM with migrations -- **Pydantic** - Request/response validation -- **Qwen3-TTS** - Voice cloning model -- **Whisper** - Speech-to-text transcription -- **librosa + soundfile** - Audio processing +Powered by Alibaba's **Qwen3-TTS** — a breakthrough model that achieves near-perfect voice cloning from just a few seconds of audio. -### Frontend (Tauri + TypeScript) -- **Tauri** - Native desktop framework -- **React** - UI framework -- **TypeScript** - Type safety throughout -- **Bun** - Fast package manager and JavaScript runtime -- **React Query** - Server state management and API calls -- **OpenAPI (generated)** - Type-safe API client from FastAPI schema -- **Tailwind CSS** - Styling -- **Zustand** - Client-side state management -- **WaveSurfer.js** - Audio visualization +- **Instant cloning** — Upload a sample, get a voice profile +- **High fidelity** — Natural prosody, emotion, and cadence +- **Multi-language** — English, Chinese, and more coming -### Database -- **SQLite** - Local storage -- **Alembic** - Schema migrations +### Voice Profile Management -## Server/Client Mode +- **Create profiles** from audio files or record directly in-app +- **Import/Export** profiles to share or backup +- **Organize** with descriptions and language tags -voicebox supports flexible deployment for users with multiple machines: +### Speech Generation -### Local Mode (Default) -- Backend runs locally alongside the Tauri app -- Best for users with GPU on their primary machine +- **Text-to-speech** with any cloned voice +- **Batch generation** for long-form content +- **Smart caching** — regenerate instantly with voice prompt caching -### Remote Mode (One-Click Setup) -- **Use case:** Your laptop doesn't have a GPU, but your desktop does -- **Server:** Run voicebox on GPU machine, click "Start Server" - - Starts FastAPI backend on local network - - Shows connection URL (e.g., `http://192.168.1.100:8000`) -- **Client:** Run voicebox on laptop, enter server URL - - Connects to remote backend - - Full UI functionality, inference happens on GPU machine -- **Security:** Local network only for now (no internet exposure) +### Recording & Transcription -### How It Works -``` -┌─────────────────┐ ┌─────────────────┐ -│ Laptop │ │ Desktop │ -│ (Client) │ │ (Server) │ -│ │ │ │ -│ Tauri App ────────────────▶ FastAPI │ -│ React UI │ HTTP │ Qwen3-TTS │ -│ │ │ SQLite │ -│ │ │ CUDA/GPU │ -└─────────────────┘ └─────────────────┘ +- **In-app recording** with waveform visualization +- **Automatic transcription** powered by Whisper +- **Export recordings** in multiple formats + +### Generation History + +- **Full history** of all generated audio +- **Search & filter** by voice, text, or date +- **Re-generate** any past generation with one click + +### Flexible Deployment + +- **Local mode** — Everything runs on your machine +- **Remote mode** — Connect to a GPU server on your network +- **One-click server** — Turn any machine into a Voicebox server + +--- + +## API + +Voicebox exposes a full REST API, so you can integrate voice synthesis into your own apps. + +```bash +# Generate speech +curl -X POST http://localhost:8000/api/generate \ + -H "Content-Type: application/json" \ + -d '{"text": "Hello world", "profile_id": "abc123"}' + +# List voice profiles +curl http://localhost:8000/api/profiles + +# Create a profile from audio +curl -X POST http://localhost:8000/api/profiles \ + -F "audio=@voice-sample.wav" \ + -F "name=My Voice" ``` -**Benefits:** -- Use powerful GPU machine from lightweight laptop -- No complex setup - just click "Start Server" -- All data (history, profiles) lives on server -- Client is just a UI - no local storage needed in remote mode +**Use cases:** -## Core Features +- Game dialogue systems +- Podcast/video production pipelines +- Accessibility tools +- Voice assistants +- Content creation automation -### Phase 1 (MVP) -- Voice profile management -- Single-reference voice cloning -- Generation history with search -- Basic audio playback and preview -- Server/client mode (local network) -- One-click server startup +Full API documentation available at `http://localhost:8000/docs` when running. -### Phase 2 -- Multi-reference voice combination -- Batch variation generation -- Advanced audio normalization -- Export options and formats +--- -### Phase 3 -- Audio studio with timeline editing -- Word-level timestamps -- Project system (save/load sessions) -- Export options +## Tech Stack -### Phase 4 -- Voice design (text-to-voice) -- Preset voices with style control -- Conversation mode (multi-speaker) -- Custom audio effects +| Layer | Technology | +|-------|------------| +| Desktop App | Tauri (Rust) | +| Frontend | React, TypeScript, Tailwind CSS | +| State | Zustand, React Query | +| Backend | FastAPI (Python) | +| Voice Model | Qwen3-TTS | +| Transcription | Whisper | +| Database | SQLite | +| Audio | WaveSurfer.js, librosa | -## Key Differentiators +**Why this stack?** -What makes voicebox better than existing implementations: +- **Tauri over Electron** — 10x smaller bundle, native performance, lower memory +- **FastAPI** — Async Python with automatic OpenAPI schema generation +- **Type-safe end-to-end** — Generated TypeScript client from OpenAPI spec -1. **Clean codebase** - Modular architecture, no 2,000+ line files -2. **Type safety end-to-end** - OpenAPI-generated TypeScript client, Pydantic backend, React Query -3. **Smart caching** - Voice prompt caching for instant re-generation -4. **Desktop UX** - Native performance, keyboard shortcuts, native dialogs -5. **Server/client mode** - One-click remote GPU access from any device -6. **Multi-reference** - Combine voice samples for higher quality -7. **Audio studio** - Timeline-based editing with word-level precision -8. **Production patterns** - Cross-platform, graceful degradation, error recovery -9. **Database-backed** - Searchable history, project persistence -10. **Extensible** - Clean plugin system for models and features +--- -## Architecture Overview +## Roadmap + +Voicebox is the beginning of something bigger. Here's what's coming: + +### Coming Soon + +| Feature | Description | +|---------|-------------| +| **Real-time Synthesis** | Stream audio as it generates, word by word | +| **Conversation Mode** | Multi-speaker dialogues with automatic turn-taking | +| **Voice Effects** | Pitch shift, reverb, M3GAN-style effects | +| **Timeline Editor** | Audio studio with word-level precision editing | +| **More Models** | XTTS, Bark, and other open-source voice models | + +### Future Vision + +- **Voice Design** — Create new voices from text descriptions +- **Project System** — Save and load complex multi-voice sessions +- **Plugin Architecture** — Extend with custom models and effects +- **Mobile Companion** — Control Voicebox from your phone + +Voicebox aims to be the **one-stop shop for everything voice** — cloning, synthesis, editing, effects, and beyond. + +--- + +## Development + +### Prerequisites + +- [Bun](https://bun.sh) (package manager) +- [Rust](https://rustup.rs) (for Tauri) +- [Python 3.11+](https://python.org) (for backend) +- CUDA-capable GPU recommended (CPU inference supported but slower) + +### Setup + +```bash +# Clone the repo +git clone https://github.com/voicebox-sh/voicebox.git +cd voicebox + +# Install dependencies +bun install + +# Install Python dependencies +cd backend && pip install -r requirements.txt && cd .. + +# Start development +bun run dev +``` + +### Project Structure ``` voicebox/ -├── app/ # Shared React frontend (used by web & desktop) -│ ├── src/ -│ │ ├── components/ # React components -│ │ │ ├── VoiceProfiles/ -│ │ │ ├── Generation/ -│ │ │ ├── AudioStudio/ -│ │ │ ├── History/ -│ │ │ └── ServerSettings/ -│ │ ├── lib/ -│ │ │ ├── api/ # Generated OpenAPI client -│ │ │ ├── hooks/ # React Query hooks -│ │ │ └── utils/ -│ │ ├── types/ -│ │ └── App.tsx -│ ├── package.json -│ └── vite.config.ts -│ -├── tauri/ # Tauri desktop app (thin wrapper) -│ ├── src/ -│ │ └── main.tsx # Entry point, imports from ../app -│ ├── src-tauri/ # Rust backend -│ │ ├── src/ -│ │ │ └── main.rs # Sidecar management, IPC -│ │ ├── binaries/ # Bundled Python server -│ │ │ └── voicebox-server-{platform} -│ │ ├── Cargo.toml -│ │ └── tauri.conf.json -│ └── package.json -│ -├── web/ # Web deployment (thin wrapper) -│ ├── src/ -│ │ └── main.tsx # Entry point, imports from ../app -│ ├── package.json -│ └── vite.config.ts -│ -├── backend/ # Python FastAPI server -│ ├── main.py # FastAPI app + server mode -│ ├── models.py # Pydantic models -│ ├── tts.py # TTS inference -│ ├── transcribe.py # Whisper ASR -│ ├── profiles.py # Voice profiles -│ ├── history.py # Generation history -│ ├── studio.py # Audio editing -│ ├── database.py # SQLite ORM -│ ├── utils/ -│ │ ├── audio.py # Audio processing -│ │ ├── cache.py # Prompt caching -│ │ └── validation.py -│ ├── requirements.txt -│ └── build_binary.py # PyInstaller build script -│ -├── scripts/ -│ ├── build-server.sh # Build Python binary for all platforms -│ └── generate-api.sh # Generate OpenAPI client -│ -├── data/ # User data -│ ├── profiles/ -│ ├── generations/ -│ ├── projects/ -│ └── voicebox.db -│ -├── package.json # Root workspace config -└── docs/ - ├── ANALYSIS.md # Analysis of existing projects - ├── TAURI_PLAN.md # Tauri app structure and bundling strategy - └── ARCHITECTURE.md # Detailed architecture docs +├── app/ # Shared React frontend +├── tauri/ # Desktop app (Tauri + Rust) +├── web/ # Web deployment +├── backend/ # Python FastAPI server +├── landing/ # Marketing website +└── scripts/ # Build & release scripts ``` -**Key architectural decisions:** -- **Shared frontend** - `app/` contains all React code, used by both desktop and web -- **Thin wrappers** - `tauri/` and `web/` just configure build tools and entry points -- **Bundled backend** - Python server packaged as sidecar binary with PyInstaller -- **Type-safe API** - OpenAPI schema generated from FastAPI, TypeScript client auto-generated +--- -See [TAURI_PLAN.md](./docs/TAURI_PLAN.md) for detailed bundling strategy. +## Contributing -## Lessons from Existing Projects +Contributions welcome! Whether it's bug fixes, new features, or documentation improvements. -voicebox learns from five existing Qwen3-TTS implementations: +1. Fork the repo +2. Create a feature branch +3. Make your changes +4. Submit a PR -### voice (Rust CLI) -- ✅ Clean Rust/Python IPC pattern -- ✅ M3GAN voice effect -- ✅ Voice profile abstraction -- ❌ No concurrent requests -- ❌ No generation history - -### Voice-Clone-Studio -- ✅ Brilliant voice prompt caching -- ✅ Feature-rich (voice design, presets, conversations) -- ✅ VRAM-efficient model management -- ❌ 2,815-line single file -- ❌ Global state everywhere - -### Qwen3-TTS_server -- ✅ Clean modular structure -- ✅ FastAPI REST API design -- ✅ Health endpoint for monitoring -- ❌ No authentication or rate limiting -- ❌ No caching or streaming -- ❌ No OpenAPI client generation - -### mimic -- ✅ Excellent backend architecture (async, modular) -- ✅ Audio studio with timeline -- ✅ Database-backed history -- ✅ Multi-sample voice profiles -- ❌ 2,794-line app.js frontend -- ❌ Global state in UI - -### qwen3-tts-enhanced -- ✅ Multi-reference combination -- ✅ Cross-platform graceful degradation -- ✅ Audio validation -- ✅ Production error handling -- ❌ Still monolithic (1,892 lines) -- ❌ No API layer - -See [ANALYSIS.md](./docs/ANALYSIS.md) for detailed breakdown of each project. - -## Development Roadmap - -### Week 1: Foundation -- Project structure setup -- Backend skeleton (FastAPI + SQLite) -- OpenAPI schema generation -- Frontend skeleton (Tauri + React) -- TypeScript client generation from OpenAPI -- React Query setup -- Basic voice profile CRUD -- Server mode implementation -- Client connection UI - -### Week 2: Core Features -- TTS integration -- Voice cloning pipeline -- Voice prompt caching -- Generation history - -### Week 3: UX Polish -- Audio playback and preview -- Profile management UI -- History search and filters -- Error handling and validation - -### Week 4: Advanced Features -- Multi-reference combination -- Batch generation -- Audio normalization -- M3GAN effect - -### Week 5+: Studio Features -- Timeline editor -- Word-level timestamps -- Project system -- Export pipeline - -## Technical Decisions - -### Why Tauri over Electron? -- Smaller bundle size (Rust vs. Node.js) -- Better performance (native vs. V8) -- Lower memory usage -- Rust for system-level operations - -### Why FastAPI over Flask? -- Native async/await support -- Automatic OpenAPI schema generation -- Pydantic validation built-in -- Better performance - -### Why OpenAPI + React Query? -- **Type safety end-to-end** - FastAPI generates OpenAPI schema, we generate TypeScript client -- **No manual API code** - Client generated from `openapi.json` using openapi-typescript-codegen -- **Automatic caching** - React Query handles request deduplication and background refetching -- **Optimistic updates** - Update UI immediately, rollback on error -- **DevX** - Full autocomplete and type checking for all API calls - -**Example workflow:** -```bash -# Backend generates OpenAPI schema -python backend/main.py --openapi > openapi.json - -# Frontend generates TypeScript client -bun run generate-client - -# Use type-safe hooks in React -import { useQuery } from '@tanstack/react-query'; -import { ProfilesService } from '@/lib/api'; - -const { data: profiles } = useQuery({ - queryKey: ['profiles'], - queryFn: () => ProfilesService.listProfiles() -}); -``` - -### Why Bun over npm/yarn/pnpm? -- **Speed** - 20-30x faster than npm for install operations -- **Drop-in replacement** - Compatible with npm ecosystem, no migration needed -- **Built-in tooling** - Bundler, test runner, and package manager in one -- **Performance** - Faster script execution than Node.js -- **Developer experience** - Better error messages, workspaces support - -### Why SQLite over file-based storage? -- Full-text search -- Transactions and integrity -- Migrations via Alembic -- Easy to backup/restore - -### Why React over Vue/Svelte? -- Larger ecosystem -- Better TypeScript support -- Familiar to most developers -- Mature tooling - -### Why bundle Python server with PyInstaller? -- **No Python installation required** - Users don't need Python on their system -- **Consistent environment** - Exact dependencies bundled, no version conflicts -- **Single-click install** - One installer includes everything -- **Tauri sidecar pattern** - Rust spawns/manages Python process lifecycle -- **Platform-specific binaries** - PyInstaller creates native executables for each platform - -**Tradeoffs:** -- Larger bundle size (~500MB with models vs ~50MB without backend) -- Need separate build for each platform (macOS Intel/ARM, Windows, Linux) -- First launch slower (model loading time) - -**Alternative considered:** Require users to install Python and run `pip install` - rejected for poor UX - -### Why no Docker initially? -- Desktop app, not server deployment -- Users install locally -- Can add later for server mode - -## Performance Targets - -- **First generation:** < 10 seconds (cold start) -- **Cached generation:** < 2 seconds (warm start) -- **UI responsiveness:** 60 FPS at all times -- **Memory usage:** < 4GB VRAM for small models -- **Startup time:** < 3 seconds to UI -- **Database queries:** < 100ms for history search - -## Quality Standards - -- **No files over 500 lines** (except auto-generated) -- **Type hints on all Python functions** -- **TypeScript strict mode enabled** -- **OpenAPI client auto-generated from schema** -- **ESLint + Prettier for frontend** -- **Black + isort for backend** -- **All user-facing errors have context** -- **No global mutable state** -- **React Query for all server state** - -## Project Status - -**Current phase:** Planning and analysis - -**Documentation:** -- [ANALYSIS.md](./docs/ANALYSIS.md) - Comprehensive analysis of existing implementations -- [TAURI_PLAN.md](./docs/TAURI_PLAN.md) - Tauri app architecture and Python server bundling strategy +--- ## License -TBD +MIT License — use it however you want. -## Credits +--- -Built by analyzing and learning from: -- voice (Rust CLI) -- Voice-Clone-Studio -- Qwen3-TTS_server -- mimic -- qwen3-tts-enhanced - -Powered by Alibaba's Qwen3-TTS model. +

+ voicebox.sh +