Labyricorn/voicebox
1
0
Fork 0
mirror of https://github.com/jamiepine/voicebox.git synced 2026-09-15 04:40:40 -07:00
Code Issues Packages Projects Releases Wiki Activity

Revamp README.md for improved clarity and presentation

Browse Source 
- Updated the README to feature a new layout with centered headings and images for better visual appeal.
- Enhanced the introduction to clearly define Voicebox as an open-source voice synthesis studio.
- Added sections for API usage, tech stack, and roadmap to provide comprehensive information about the project.
- Removed outdated content and streamlined the structure for easier navigation and understanding.
This commit is contained in:
Jamie Pine
2026-01-25 19:32:30 -08:00
parent 7d43938b49
commit c14fb937ef
3 changed files with 191 additions and 366 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.github/assets/icon-dark.webp
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 10 KiB

.github/assets/screenshot.webp
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 182 KiB

README.md
+191 -366
View File
@@ -1,409 +1,234 @@
# voicebox
<p align="center">
<img src=".github/assets/icon-dark.webp" alt="Voicebox" width="120" height="120" />
</p>
A production-quality desktop app for Qwen3-TTS voice cloning and generation.
<h1 align="center">Voicebox</h1>
**Domain:** voicebox.sh
<p align="center">
<strong>The open-source voice synthesis studio.</strong><br/>
Clone voices. Generate speech. Build voice-powered apps.<br/>
All running locally on your machine.
</p>
<p align="center">
<a href="https://voicebox.sh">Website</a> •
<a href="#download">Download</a> •
<a href="#features">Features</a> •
<a href="#api">API</a> •
<a href="#roadmap">Roadmap</a>
</p>
<br/>
<p align="center">
<img src=".github/assets/screenshot.webp" alt="Voicebox App Screenshot" width="800" />
</p>
<br/>
## Why Voicebox?
Voice AI is exploding, but most tools are either cloud-locked, expensive, or a nightmare to set up. Voicebox is different:
- **100% Local** — Your voice data never leaves your machine
- **Lightweight** — ~50MB app, runs on modest hardware
- **Fast** — Sub-2-second generation with caching
- **Flexible** — Use the app, integrate the API, or both
- **Open Source** — No subscriptions, no limits, no lock-in
Built with **Tauri** (Rust), **TypeScript**, **React**, and **Python**. Native performance meets modern DX.
---
## Vision
## Download
Qwen3-TTS is a breakthrough model from Alibaba that achieves near-perfect voice cloning. The existing implementations (Voice-Clone-Studio, mimic, etc.) are either feature-rich but architecturally messy, or well-structured but limited in scope.
Voicebox is available now for macOS, Windows, and Linux.
voicebox aims to build the definitive Qwen3-TTS application by combining the best patterns from existing projects while avoiding their architectural mistakes.
| Platform | Download |
|----------|----------|
| macOS (Apple Silicon) | [voicebox-macos-arm64.dmg](https://github.com/voicebox-sh/voicebox/releases/latest) |
| macOS (Intel) | [voicebox-macos-x64.dmg](https://github.com/voicebox-sh/voicebox/releases/latest) |
| Windows | [voicebox-windows-x64.exe](https://github.com/voicebox-sh/voicebox/releases/latest) |
| Linux | [voicebox-linux-x64.AppImage](https://github.com/voicebox-sh/voicebox/releases/latest) |
## Design Principles
---
1. **Clean architecture from day one** - No monolithic files, proper separation of concerns
2. **Desktop-first experience** - Native feel via Tauri, not a web app in disguise
3. **Production code quality** - Type safety, modularity, maintainability
4. **Performance and UX** - Smart caching, async operations, responsive UI
5. **Extensible design** - Easy to add new models, effects, and features
6. **Flexible deployment** - Run backend locally or connect to remote GPU machine with one click
## Features
## Technology Stack
### Voice Cloning with Qwen3-TTS
### Backend (Python)
- **FastAPI** - Async REST API
- **SQLAlchemy** - Database ORM with migrations
- **Pydantic** - Request/response validation
- **Qwen3-TTS** - Voice cloning model
- **Whisper** - Speech-to-text transcription
- **librosa + soundfile** - Audio processing
Powered by Alibaba's **Qwen3-TTS** — a breakthrough model that achieves near-perfect voice cloning from just a few seconds of audio.
### Frontend (Tauri + TypeScript)
- **Tauri** - Native desktop framework
- **React** - UI framework
- **TypeScript** - Type safety throughout
- **Bun** - Fast package manager and JavaScript runtime
- **React Query** - Server state management and API calls
- **OpenAPI (generated)** - Type-safe API client from FastAPI schema
- **Tailwind CSS** - Styling
- **Zustand** - Client-side state management
- **WaveSurfer.js** - Audio visualization
- **Instant cloning** — Upload a sample, get a voice profile
- **High fidelity** — Natural prosody, emotion, and cadence
- **Multi-language** — English, Chinese, and more coming
### Database
- **SQLite** - Local storage
- **Alembic** - Schema migrations
### Voice Profile Management
## Server/Client Mode
- **Create profiles** from audio files or record directly in-app
- **Import/Export** profiles to share or backup
- **Organize** with descriptions and language tags
voicebox supports flexible deployment for users with multiple machines:
### Speech Generation
### Local Mode (Default)
- Backend runs locally alongside the Tauri app
- Best for users with GPU on their primary machine
- **Text-to-speech** with any cloned voice
- **Batch generation** for long-form content
- **Smart caching** — regenerate instantly with voice prompt caching
### Remote Mode (One-Click Setup)
- **Use case:** Your laptop doesn't have a GPU, but your desktop does
- **Server:** Run voicebox on GPU machine, click "Start Server"
- Starts FastAPI backend on local network
- Shows connection URL (e.g., `http://192.168.1.100:8000`)
- **Client:** Run voicebox on laptop, enter server URL
- Connects to remote backend
- Full UI functionality, inference happens on GPU machine
- **Security:** Local network only for now (no internet exposure)
### Recording & Transcription
### How It Works
```
┌─────────────────┐ ┌─────────────────┐
│ Laptop │ │ Desktop │
│ (Client) │ │ (Server) │
│ │ │ │
│ Tauri App ────────────────▶ FastAPI │
│ React UI │ HTTP │ Qwen3-TTS │
│ │ │ SQLite │
│ │ │ CUDA/GPU │
└─────────────────┘ └─────────────────┘
- **In-app recording** with waveform visualization
- **Automatic transcription** powered by Whisper
- **Export recordings** in multiple formats
### Generation History
- **Full history** of all generated audio
- **Search & filter** by voice, text, or date
- **Re-generate** any past generation with one click
### Flexible Deployment
- **Local mode** — Everything runs on your machine
- **Remote mode** — Connect to a GPU server on your network
- **One-click server** — Turn any machine into a Voicebox server
---
## API
Voicebox exposes a full REST API, so you can integrate voice synthesis into your own apps.
```bash
# Generate speech
curl -X POST http://localhost:8000/api/generate \
-H "Content-Type: application/json" \
-d '{"text": "Hello world", "profile_id": "abc123"}'
# List voice profiles
curl http://localhost:8000/api/profiles
# Create a profile from audio
curl -X POST http://localhost:8000/api/profiles \
-F "[email protected]" \
-F "name=My Voice"
```
**Benefits:**
- Use powerful GPU machine from lightweight laptop
- No complex setup - just click "Start Server"
- All data (history, profiles) lives on server
- Client is just a UI - no local storage needed in remote mode
**Use cases:**
## Core Features
- Game dialogue systems
- Podcast/video production pipelines
- Accessibility tools
- Voice assistants
- Content creation automation
### Phase 1 (MVP)
- Voice profile management
- Single-reference voice cloning
- Generation history with search
- Basic audio playback and preview
- Server/client mode (local network)
- One-click server startup
Full API documentation available at `http://localhost:8000/docs` when running.
### Phase 2
- Multi-reference voice combination
- Batch variation generation
- Advanced audio normalization
- Export options and formats
---
### Phase 3
- Audio studio with timeline editing
- Word-level timestamps
- Project system (save/load sessions)
- Export options
## Tech Stack
### Phase 4
- Voice design (text-to-voice)
- Preset voices with style control
- Conversation mode (multi-speaker)
- Custom audio effects
| Layer | Technology |
|-------|------------|
| Desktop App | Tauri (Rust) |
| Frontend | React, TypeScript, Tailwind CSS |
| State | Zustand, React Query |
| Backend | FastAPI (Python) |
| Voice Model | Qwen3-TTS |
| Transcription | Whisper |
| Database | SQLite |
| Audio | WaveSurfer.js, librosa |
## Key Differentiators
**Why this stack?**
What makes voicebox better than existing implementations:
- **Tauri over Electron** — 10x smaller bundle, native performance, lower memory
- **FastAPI** — Async Python with automatic OpenAPI schema generation
- **Type-safe end-to-end** — Generated TypeScript client from OpenAPI spec
1. **Clean codebase** - Modular architecture, no 2,000+ line files
2. **Type safety end-to-end** - OpenAPI-generated TypeScript client, Pydantic backend, React Query
3. **Smart caching** - Voice prompt caching for instant re-generation
4. **Desktop UX** - Native performance, keyboard shortcuts, native dialogs
5. **Server/client mode** - One-click remote GPU access from any device
6. **Multi-reference** - Combine voice samples for higher quality
7. **Audio studio** - Timeline-based editing with word-level precision
8. **Production patterns** - Cross-platform, graceful degradation, error recovery
9. **Database-backed** - Searchable history, project persistence
10. **Extensible** - Clean plugin system for models and features
---
## Architecture Overview
## Roadmap
Voicebox is the beginning of something bigger. Here's what's coming:
### Coming Soon
| Feature | Description |
|---------|-------------|
| **Real-time Synthesis** | Stream audio as it generates, word by word |
| **Conversation Mode** | Multi-speaker dialogues with automatic turn-taking |
| **Voice Effects** | Pitch shift, reverb, M3GAN-style effects |
| **Timeline Editor** | Audio studio with word-level precision editing |
| **More Models** | XTTS, Bark, and other open-source voice models |
### Future Vision
- **Voice Design** — Create new voices from text descriptions
- **Project System** — Save and load complex multi-voice sessions
- **Plugin Architecture** — Extend with custom models and effects
- **Mobile Companion** — Control Voicebox from your phone
Voicebox aims to be the **one-stop shop for everything voice** — cloning, synthesis, editing, effects, and beyond.
---
## Development
### Prerequisites
- [Bun](https://bun.sh) (package manager)
- [Rust](https://rustup.rs) (for Tauri)
- [Python 3.11+](https://python.org) (for backend)
- CUDA-capable GPU recommended (CPU inference supported but slower)
### Setup
```bash
# Clone the repo
git clone https://github.com/voicebox-sh/voicebox.git
cd voicebox
# Install dependencies
bun install
# Install Python dependencies
cd backend && pip install -r requirements.txt && cd ..
# Start development
bun run dev
```
### Project Structure
```
voicebox/
├── app/ # Shared React frontend (used by web & desktop)
│ ├── src/
│ │ ├── components/ # React components
│ │ │ ├── VoiceProfiles/
│ │ │ ├── Generation/
│ │ │ ├── AudioStudio/
│ │ │ ├── History/
│ │ │ └── ServerSettings/
│ │ ├── lib/
│ │ │ ├── api/ # Generated OpenAPI client
│ │ │ ├── hooks/ # React Query hooks
│ │ │ └── utils/
│ │ ├── types/
│ │ └── App.tsx
│ ├── package.json
│ └── vite.config.ts
│
├── tauri/ # Tauri desktop app (thin wrapper)
│ ├── src/
│ │ └── main.tsx # Entry point, imports from ../app
│ ├── src-tauri/ # Rust backend
│ │ ├── src/
│ │ │ └── main.rs # Sidecar management, IPC
│ │ ├── binaries/ # Bundled Python server
│ │ │ └── voicebox-server-{platform}
│ │ ├── Cargo.toml
│ │ └── tauri.conf.json
│ └── package.json
│
├── web/ # Web deployment (thin wrapper)
│ ├── src/
│ │ └── main.tsx # Entry point, imports from ../app
│ ├── package.json
│ └── vite.config.ts
│
├── backend/ # Python FastAPI server
│ ├── main.py # FastAPI app + server mode
│ ├── models.py # Pydantic models
│ ├── tts.py # TTS inference
│ ├── transcribe.py # Whisper ASR
│ ├── profiles.py # Voice profiles
│ ├── history.py # Generation history
│ ├── studio.py # Audio editing
│ ├── database.py # SQLite ORM
│ ├── utils/
│ │ ├── audio.py # Audio processing
│ │ ├── cache.py # Prompt caching
│ │ └── validation.py
│ ├── requirements.txt
│ └── build_binary.py # PyInstaller build script
│
├── scripts/
│ ├── build-server.sh # Build Python binary for all platforms
│ └── generate-api.sh # Generate OpenAPI client
│
├── data/ # User data
│ ├── profiles/
│ ├── generations/
│ ├── projects/
│ └── voicebox.db
│
├── package.json # Root workspace config
└── docs/
├── ANALYSIS.md # Analysis of existing projects
├── TAURI_PLAN.md # Tauri app structure and bundling strategy
└── ARCHITECTURE.md # Detailed architecture docs
├── app/ # Shared React frontend
├── tauri/ # Desktop app (Tauri + Rust)
├── web/ # Web deployment
├── backend/ # Python FastAPI server
├── landing/ # Marketing website
└── scripts/ # Build & release scripts
```
**Key architectural decisions:**
- **Shared frontend** - `app/` contains all React code, used by both desktop and web
- **Thin wrappers** - `tauri/` and `web/` just configure build tools and entry points
- **Bundled backend** - Python server packaged as sidecar binary with PyInstaller
- **Type-safe API** - OpenAPI schema generated from FastAPI, TypeScript client auto-generated
---
See [TAURI_PLAN.md](./docs/TAURI_PLAN.md) for detailed bundling strategy.
## Contributing
## Lessons from Existing Projects
Contributions welcome! Whether it's bug fixes, new features, or documentation improvements.
voicebox learns from five existing Qwen3-TTS implementations:
1. Fork the repo
2. Create a feature branch
3. Make your changes
4. Submit a PR
### voice (Rust CLI)
- ✅ Clean Rust/Python IPC pattern
- ✅ M3GAN voice effect
- ✅ Voice profile abstraction
- ❌ No concurrent requests
- ❌ No generation history
### Voice-Clone-Studio
- ✅ Brilliant voice prompt caching
- ✅ Feature-rich (voice design, presets, conversations)
- ✅ VRAM-efficient model management
- ❌ 2,815-line single file
- ❌ Global state everywhere
### Qwen3-TTS_server
- ✅ Clean modular structure
- ✅ FastAPI REST API design
- ✅ Health endpoint for monitoring
- ❌ No authentication or rate limiting
- ❌ No caching or streaming
- ❌ No OpenAPI client generation
### mimic
- ✅ Excellent backend architecture (async, modular)
- ✅ Audio studio with timeline
- ✅ Database-backed history
- ✅ Multi-sample voice profiles
- ❌ 2,794-line app.js frontend
- ❌ Global state in UI
### qwen3-tts-enhanced
- ✅ Multi-reference combination
- ✅ Cross-platform graceful degradation
- ✅ Audio validation
- ✅ Production error handling
- ❌ Still monolithic (1,892 lines)
- ❌ No API layer
See [ANALYSIS.md](./docs/ANALYSIS.md) for detailed breakdown of each project.
## Development Roadmap
### Week 1: Foundation
- Project structure setup
- Backend skeleton (FastAPI + SQLite)
- OpenAPI schema generation
- Frontend skeleton (Tauri + React)
- TypeScript client generation from OpenAPI
- React Query setup
- Basic voice profile CRUD
- Server mode implementation
- Client connection UI
### Week 2: Core Features
- TTS integration
- Voice cloning pipeline
- Voice prompt caching
- Generation history
### Week 3: UX Polish
- Audio playback and preview
- Profile management UI
- History search and filters
- Error handling and validation
### Week 4: Advanced Features
- Multi-reference combination
- Batch generation
- Audio normalization
- M3GAN effect
### Week 5+: Studio Features
- Timeline editor
- Word-level timestamps
- Project system
- Export pipeline
## Technical Decisions
### Why Tauri over Electron?
- Smaller bundle size (Rust vs. Node.js)
- Better performance (native vs. V8)
- Lower memory usage
- Rust for system-level operations
### Why FastAPI over Flask?
- Native async/await support
- Automatic OpenAPI schema generation
- Pydantic validation built-in
- Better performance
### Why OpenAPI + React Query?
- **Type safety end-to-end** - FastAPI generates OpenAPI schema, we generate TypeScript client
- **No manual API code** - Client generated from `openapi.json` using openapi-typescript-codegen
- **Automatic caching** - React Query handles request deduplication and background refetching
- **Optimistic updates** - Update UI immediately, rollback on error
- **DevX** - Full autocomplete and type checking for all API calls
**Example workflow:**
```bash
# Backend generates OpenAPI schema
python backend/main.py --openapi > openapi.json
# Frontend generates TypeScript client
bun run generate-client
# Use type-safe hooks in React
import { useQuery } from '@tanstack/react-query';
import { ProfilesService } from '@/lib/api';
const { data: profiles } = useQuery({
queryKey: ['profiles'],
queryFn: () => ProfilesService.listProfiles()
});
```
### Why Bun over npm/yarn/pnpm?
- **Speed** - 20-30x faster than npm for install operations
- **Drop-in replacement** - Compatible with npm ecosystem, no migration needed
- **Built-in tooling** - Bundler, test runner, and package manager in one
- **Performance** - Faster script execution than Node.js
- **Developer experience** - Better error messages, workspaces support
### Why SQLite over file-based storage?
- Full-text search
- Transactions and integrity
- Migrations via Alembic
- Easy to backup/restore
### Why React over Vue/Svelte?
- Larger ecosystem
- Better TypeScript support
- Familiar to most developers
- Mature tooling
### Why bundle Python server with PyInstaller?
- **No Python installation required** - Users don't need Python on their system
- **Consistent environment** - Exact dependencies bundled, no version conflicts
- **Single-click install** - One installer includes everything
- **Tauri sidecar pattern** - Rust spawns/manages Python process lifecycle
- **Platform-specific binaries** - PyInstaller creates native executables for each platform
**Tradeoffs:**
- Larger bundle size (~500MB with models vs ~50MB without backend)
- Need separate build for each platform (macOS Intel/ARM, Windows, Linux)
- First launch slower (model loading time)
**Alternative considered:** Require users to install Python and run `pip install` - rejected for poor UX
### Why no Docker initially?
- Desktop app, not server deployment
- Users install locally
- Can add later for server mode
## Performance Targets
- **First generation:** < 10 seconds (cold start)
- **Cached generation:** < 2 seconds (warm start)
- **UI responsiveness:** 60 FPS at all times
- **Memory usage:** < 4GB VRAM for small models
- **Startup time:** < 3 seconds to UI
- **Database queries:** < 100ms for history search
## Quality Standards
- **No files over 500 lines** (except auto-generated)
- **Type hints on all Python functions**
- **TypeScript strict mode enabled**
- **OpenAPI client auto-generated from schema**
- **ESLint + Prettier for frontend**
- **Black + isort for backend**
- **All user-facing errors have context**
- **No global mutable state**
- **React Query for all server state**
## Project Status
**Current phase:** Planning and analysis
**Documentation:**
- [ANALYSIS.md](./docs/ANALYSIS.md) - Comprehensive analysis of existing implementations
- [TAURI_PLAN.md](./docs/TAURI_PLAN.md) - Tauri app architecture and Python server bundling strategy
---
## License
TBD
MIT License — use it however you want.
## Credits
---
Built by analyzing and learning from:
- voice (Rust CLI)
- Voice-Clone-Studio
- Qwen3-TTS_server
- mimic
- qwen3-tts-enhanced
Powered by Alibaba's Qwen3-TTS model.
<p align="center">
<a href="https://voicebox.sh">voicebox.sh</a>
</p>