Compare commits

...
Author SHA1 Message Date
James Pine 192979a762 docs 2026-03-16 05:11:21 -07:00
James Pine e16cc42d53 enable Edit on GitHub and last updated on all doc pages 2026-03-16 04:45:40 -07:00
James Pine f10e965003 rewrite docs root page, add screenshot 2026-03-16 04:12:11 -07:00
James Pine a8968d4081 rewrite docs introduction based on README content 2026-03-16 04:09:50 -07:00
James Pine 7c4afbe4df expand sidebar groups by default, remove stale plans reference 2026-03-16 04:08:46 -07:00
James Pine a180fcc56f redirect root to /docs 2026-03-16 04:06:38 -07:00
James Pine 1860b8dc92 remove plans/ from docs site 2026-03-16 04:05:14 -07:00
James Pine 1597937535 Merge branch 'main' into better-docs
# Conflicts:
#	backend/main.py
#	docs/content/docs/plans/ADDING_TTS_ENGINES.md
#	docs/content/docs/plans/CUDA_BACKEND_SWAP.md
#	docs/content/docs/plans/CUDA_BACKEND_SWAP_FINAL.md
#	docs/content/docs/plans/EXTERNAL_PROVIDERS.md
#	docs/content/docs/plans/MLX_AUDIO.md
#	docs/content/docs/plans/PROJECT_STATUS.md
2026-03-16 04:01:08 -07:00
Jamie PineandGitHub ac41a89359 Merge pull request #285 from jamiepine/backend-refactor
Backend refactor: modular architecture, style guide, tooling
2026-03-16 03:51:10 -07:00
Jamie Pine 2e6efa00a2 Refactor documentation structure and dependencies for migration to Fumadocs
- Updated `.gitignore` to include new build and generated content directories.
- Removed outdated Mintlify configuration files and documentation.
- Introduced new `MIGRATION.md` to outline the transition from Mintlify to Fumadocs.
- Added `mdx-components.tsx` for MDX component configuration and compatibility.
- Updated `package.json` and `next.config.mjs` for new dependencies and Next.js configuration.
- Created `source.config.ts` for content source configuration.
- Added OpenAPI specification in `openapi.json` for API documentation.
- Removed legacy files and adjusted project structure to align with Fumadocs conventions.
2026-02-02 23:29:35 -08:00
Jamie Pine 788a04f265 Merge branch 'main' into better-docs 2026-02-02 23:18:06 -08:00
Jamie Pine 5cb54ee03c Update API documentation and enhance server configuration
- Added server configurations for local and production environments in `main.py`.
- Removed outdated authentication and generation API documentation files.
- Updated documentation structure to reflect the removal of deprecated API endpoints.
- Adjusted links in the quick start and developer setup documentation to point to the new API reference.
- Enhanced global CSS styles for improved theming support.
2026-01-31 01:45:42 -08:00
Jamie Pine 0922845101 disable cuda for 0.1.12 2026-01-31 01:44:34 -08:00
Jamie Pine 64dd29d35a Add initial setup for Fumadocs documentation migration
- Created new directory structure for documentation under `/docs2`.
- Added `.gitignore` to exclude build artifacts and dependencies.
- Introduced `package.json`, `next.config.mjs`, and `postcss.config.mjs` for project configuration.
- Implemented MDX components in `mdx-components.tsx` for rendering documentation.
- Migrated existing documentation content and created new files for auto-updater and other features.
- Established compatibility layer for Mintlify components in `mintlify-compat.tsx`.
- Set up OpenAPI documentation in `openapi.json`.
- Updated README and migration guide to reflect new structure and usage instructions.
- Ensured all components and pages are ready for development and deployment with Fumadocs.
2026-01-30 23:32:45 -08:00
113 changed files with 3218 additions and 4325 deletions
+25 -2
View File
@@ -1,3 +1,26 @@
node_modules
.mintlify
# deps
/node_modules
# generated content
.source
# test & build
/coverage
/.next/
/out/
/build
*.tsbuildinfo
# misc
.DS_Store
*.pem
/.pnp
.pnp.js
npm-debug.log*
yarn-debug.log*
yarn-error.log*
# others
.env*.local
.vercel
next-env.d.ts
-192
View File
@@ -1,192 +0,0 @@
# Auto-Updater Documentation
Voicebox includes automatic updates powered by Tauri's updater plugin. This document explains how it works for both users and developers.
## 1. Generate Signing Keys
Run this command to generate your signing keypair:
```bash
cd tauri && bun tauri signer generate -w ~/.tauri/voicebox.key
```
This creates:
- **Private key**: `~/.tauri/voicebox.key` (keep this secret!)
- **Public key**: `~/.tauri/voicebox.key.pub`
## 2. Update Configuration
Copy the content from `~/.tauri/voicebox.key.pub` and replace the placeholder in `tauri/src-tauri/tauri.conf.json`:
```json
{
"plugins": {
"updater": {
"pubkey": "PASTE_PUBLIC_KEY_CONTENT_HERE",
"endpoints": [
"https://github.com/YOUR_USERNAME/voicebox/releases/latest/download/latest.json"
]
}
}
}
```
Update the endpoint URL with your actual GitHub username/organization.
## 3. Building with Signatures
When building releases, set these environment variables:
**macOS/Linux:**
```bash
export TAURI_SIGNING_PRIVATE_KEY="$(cat ~/.tauri/voicebox.key)"
export TAURI_SIGNING_PRIVATE_KEY_PASSWORD=""
bun run build
```
**Windows PowerShell:**
```powershell
$env:TAURI_SIGNING_PRIVATE_KEY = Get-Content ~/.tauri/voicebox.key -Raw
$env:TAURI_SIGNING_PRIVATE_KEY_PASSWORD = ""
bun run build
```
## 4. GitHub Release Setup
When you create a GitHub release, the build process will generate:
- Installers for each platform
- `.sig` signature files
- `latest.json` update manifest
### Manual Release Process
1. Build the app with signing keys set
2. Create a new GitHub release
3. Upload all files from `tauri/src-tauri/target/release/bundle/`
4. Create `latest.json` in your release assets:
```json
{
"version": "0.2.0",
"notes": "Bug fixes and improvements",
"pub_date": "2026-01-25T12:00:00Z",
"platforms": {
"darwin-aarch64": {
"signature": "CONTENT_FROM_.app.tar.gz.sig",
"url": "https://github.com/YOUR_USERNAME/voicebox/releases/download/v0.2.0/voicebox_0.2.0_aarch64.dmg"
},
"darwin-x86_64": {
"signature": "CONTENT_FROM_.app.tar.gz.sig",
"url": "https://github.com/YOUR_USERNAME/voicebox/releases/download/v0.2.0/voicebox_0.2.0_x64.dmg"
},
"linux-x86_64": {
"signature": "CONTENT_FROM_.AppImage.sig",
"url": "https://github.com/YOUR_USERNAME/voicebox/releases/download/v0.2.0/voicebox_0.2.0_amd64.AppImage"
},
"windows-x86_64": {
"signature": "CONTENT_FROM_.msi.sig",
"url": "https://github.com/YOUR_USERNAME/voicebox/releases/download/v0.2.0/voicebox_0.2.0_x64_en-US.msi"
}
}
}
```
### Automated GitHub Actions (Recommended)
Create `.github/workflows/release.yml`:
```yaml
name: Release
on:
push:
tags:
- 'v*'
jobs:
release:
strategy:
matrix:
platform: [macos-latest, ubuntu-22.04, windows-latest]
runs-on: ${{ matrix.platform }}
steps:
- uses: actions/checkout@v4
- name: Setup Bun
uses: oven-sh/setup-bun@v1
- name: Install Rust
uses: dtolnay/rust-toolchain@stable
- name: Install dependencies (Ubuntu)
if: matrix.platform == 'ubuntu-22.04'
run: |
sudo apt-get update
sudo apt-get install -y libwebkit2gtk-4.1-dev libappindicator3-dev librsvg2-dev patchelf
- name: Install dependencies
run: bun install
- name: Build
env:
TAURI_SIGNING_PRIVATE_KEY: ${{ secrets.TAURI_SIGNING_PRIVATE_KEY }}
TAURI_SIGNING_PRIVATE_KEY_PASSWORD: ${{ secrets.TAURI_SIGNING_PRIVATE_KEY_PASSWORD }}
run: bun run build
- name: Upload Release
uses: softprops/action-gh-release@v1
with:
files: tauri/src-tauri/target/release/bundle/**/*
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
```
Add your private key to GitHub secrets:
- Go to Settings → Secrets and variables → Actions
- Add `TAURI_SIGNING_PRIVATE_KEY` with the content of `~/.tauri/voicebox.key`
- Add `TAURI_SIGNING_PRIVATE_KEY_PASSWORD` (empty string if no password)
## Frontend Integration
The frontend integration is complete with automatic update notifications and manual update checks:
- **Update Notification Banner** - Appears automatically when updates are available
- **Settings Panel** - Manual "Check for Updates" button in Settings tab
- **Update Hook** - React hook handles all update operations
See `docs/AUTOUPDATER_QUICKSTART.md` for a quick setup guide.
## Security Notes
- Never commit your private key to version control
- Store private keys securely (use GitHub secrets for CI/CD)
- The public key in `tauri.conf.json` is safe to commit
- Updates are cryptographically verified before installation
- HTTP endpoints are blocked by default (HTTPS only)
## Testing Updates
1. Build version 0.1.0 and install it
2. Update version in `tauri.conf.json` to 0.2.0
3. Build version 0.2.0 with signatures
4. Create a local server or GitHub release with `latest.json`
5. Run version 0.1.0 and trigger update check
6. Verify update downloads and installs correctly
## Troubleshooting
**"Invalid signature" error:**
- Verify public key matches the private key used to sign
- Ensure signature files (.sig) are uploaded correctly
**"No update available" when one exists:**
- Check endpoint URL is correct
- Verify `latest.json` format matches specification
- Ensure version in latest.json is higher than current version
**Build fails with signing:**
- Confirm environment variables are set correctly
- Check private key file exists and is readable
- Verify private key format (should start with `dW50cnVzdGVkIGNvbW1lbnQ6`)
-116
View File
@@ -1,116 +0,0 @@
# Autoupdater Quick Start
The Tauri v2 autoupdater has been fully configured and integrated. Follow these steps to activate it.
## What's Already Done
✅ Rust plugin installed and initialized
✅ Tauri configuration set up with updater settings
✅ Permissions granted for update operations
✅ GitHub Actions workflow updated with signing support
✅ Frontend components created and integrated
✅ Update notifications on app startup
✅ Manual update check in Settings tab
## Required Steps (5 minutes)
### 1. Generate Signing Keys
```bash
bun run generate:keys
```
This creates:
- Private key: `~/.tauri/voicebox.key` (keep secret!)
- Public key: `~/.tauri/voicebox.key.pub` (safe to share)
### 2. Update Tauri Config
Open `tauri/src-tauri/tauri.conf.json` and:
1. Replace `"REPLACE_WITH_YOUR_PUBLIC_KEY"` with the content from `~/.tauri/voicebox.key.pub`
2. Update the endpoint URL with your GitHub username:
```json
"endpoints": [
"https://github.com/YOUR_USERNAME/voicebox/releases/latest/download/latest.json"
]
```
### 3. Add GitHub Secrets
Go to your repo Settings → Secrets and variables → Actions:
1. Add `TAURI_SIGNING_PRIVATE_KEY`:
```bash
cat ~/.tauri/voicebox.key
```
Copy the entire output and paste as the secret value
2. Add `TAURI_SIGNING_PRIVATE_KEY_PASSWORD`:
Leave empty (or add your password if you set one)
### 4. Test the Setup
To test locally before creating a release:
```bash
bun run build:release
```
This will verify your keys are set up correctly.
## How It Works
### For Users
1. App checks for updates on startup (only in Tauri builds)
2. If an update is available, a banner appears at the top
3. Users can click "Install Now" to download and install
4. App restarts automatically after installation
### For Developers
1. Create a new git tag: `git tag v0.2.0 && git push --tags`
2. GitHub Actions builds signed releases for all platforms
3. Uploads installers and generates `latest.json` manifest
4. Users running older versions will be notified automatically
## UI Components
### Update Notification Banner
- Shows at top of app when update is available
- Appears automatically on startup
- Displays download/install progress
### Settings Panel
- Located in Settings tab
- Shows current version
- Manual "Check for Updates" button
- Update status and progress
## Troubleshooting
**"Public key not configured"**
- Make sure you copied the entire content from `voicebox.key.pub`
- The key should start with `dW50cnVzdGVkIGNvbW1lbnQ6`
**"Failed to check for updates"**
- Endpoint URL might be incorrect
- No releases published yet (expected for first setup)
**Build fails with signing error**
- Check that GitHub secrets are set correctly
- Verify private key file exists at `~/.tauri/voicebox.key`
## Next Release Workflow
1. Update version in `tauri/src-tauri/tauri.conf.json`
2. Commit changes
3. Create and push tag: `git tag v0.2.0 && git push --tags`
4. GitHub Actions will automatically build and create a draft release
5. Review the release and publish it
6. Users will be notified of the update
## See Also
- Full documentation: `docs/AUTOUPDATER.md`
- Build script: `scripts/prepare-release.sh`
- GitHub workflow: `.github/workflows/release.yml`
+87
View File
@@ -0,0 +1,87 @@
# Documentation Migration: Mintlify → Fumadocs
This document summarizes the migration of documentation from `/docs` (Mintlify) to `/docs2` (Fumadocs).
## What Was Done
### 1. Files Copied
- ✅ All 29 MDX files from `/docs` folders (overview, api, developer, plans)
- ✅ All 4 root-level markdown files (AUTOUPDATER.md, AUTOUPDATER_QUICKSTART.md, TROUBLESHOOTING.md, README.md)
- ✅ All images (3 webp files) → `public/images/`
- ✅ All logo files (2 png files) → `public/logo/`
### 2. Component Migration
Created compatibility layer in `components/mintlify-compat.tsx` that maps Mintlify components to Fumadocs equivalents:
- `<Frame>` → Simple div wrapper (images are zoomable by default in Fumadocs)
- `<CardGroup>` → `<Cards>` (Fumadocs component)
- `<Card>` → `<Card>` (with icon string → Lucide icon mapping)
- `<Steps>` / `<Step>` → Direct mapping to Fumadocs components
- `<Tip>`, `<Note>`, `<Info>` → `<Callout type="info">`
- `<Warning>` → `<Callout type="warn">`
- `<Danger>` → `<Callout type="error">`
- `<AccordionGroup>` / `<Accordion>` → HTML `<details>` / `<summary>` elements
### 3. Navigation Structure
Created `meta.json` files for each folder:
- `content/docs/meta.json` - Root documentation
- `content/docs/overview/meta.json` - Overview pages
- `content/docs/api/meta.json` - API reference
- `content/docs/developer/meta.json` - Developer docs
- `content/docs/plans/meta.json` - Plans/roadmap
### 4. Link Fixes
- Fixed incorrect `/guides/...` paths → `/overview/...`
- All internal links now use correct paths
### 5. Branding
- Updated `lib/layout.shared.tsx` to use "Voicebox" as the nav title
## File Structure
```
docs2/
├── components/
│ └── mintlify-compat.tsx # Mintlify → Fumadocs component mappings
├── content/docs/
│ ├── meta.json # Root navigation
│ ├── overview/ # 12 MDX files
│ ├── api/ # 5 MDX files
│ ├── developer/ # 12 MDX files
│ ├── plans/ # 4 MD files
│ └── *.md # 4 root markdown files
├── public/
│ ├── images/ # 3 webp files
│ └── logo/ # 2 png files
└── mdx-components.tsx # MDX component configuration
```
## Icon Mapping
The following icon strings are mapped to Lucide icons:
- `microphone` → Mic
- `film` → Film
- `code` → Code
- `shield` → Shield
- `download` → Download
- `rocket` → Rocket
- `apple` → Apple
- `windows` → Windows
- `server` → Server
- `user` → User
- `waveform` → Waveform
## Next Steps
1. **Test the build**: Run `npm run build` (requires Node.js >= 20.9.0)
2. **Start dev server**: Run `npm run dev` to preview
3. **Customize styling**: Update `app/global.css` if needed
4. **Add more icons**: Extend `iconMap` in `mintlify-compat.tsx` as needed
5. **Review navigation**: Adjust `meta.json` files to customize page order
## Notes
- Image paths (`/images/...`) work as-is since Next.js serves from `public/`
- All Mintlify components are now compatible with Fumadocs
- Navigation structure follows Fumadocs conventions
- No breaking changes to content - all MDX files work with compatibility layer
+29 -48
View File
@@ -1,64 +1,45 @@
# Voicebox Documentation
# fumadocs-ui-template
This directory contains the documentation for Voicebox, built with [Mintlify](https://mintlify.com).
This is a Next.js application generated with
[Create Fumadocs](https://github.com/fuma-nama/fumadocs).
## Development
### Prerequisites
Install Mintlify globally using bun:
Run development server:
```bash
bun add -g mintlify
npm run dev
# or
pnpm dev
# or
yarn dev
```
Or use the helper script:
Open http://localhost:3000 with your browser to see the result.
```bash
bun run install:mintlify
```
## Explore
### Running Locally
In the project, you can see:
```bash
bun run dev
```
- `lib/source.ts`: Code for content source adapter, [`loader()`](https://fumadocs.dev/docs/headless/source-api) provides the interface to access your content.
- `lib/layout.shared.tsx`: Shared options for layouts, optional but preferred to keep.
This will start the Mintlify dev server.
| Route | Description |
| ------------------------- | ------------------------------------------------------ |
| `app/(home)` | The route group for your landing page and other pages. |
| `app/docs` | The documentation layout and pages. |
| `app/api/search/route.ts` | The Route Handler for search. |
The docs will be available at `http://localhost:3000`
### Fumadocs MDX
### Structure
A `source.config.ts` config file has been included, you can customise different options like frontmatter schema.
```
docs/
├── mint.json # Mintlify configuration
├── custom.css # Custom styles
├── overview/ # Getting started & feature docs
├── guides/ # User guides
├── api/ # API reference
├── development/ # Developer documentation
├── logo/ # Logo assets
└── public/ # Static assets
```
Read the [Introduction](https://fumadocs.dev/docs/mdx) for further details.
### Writing Docs
## Learn More
- Use `.mdx` files for all documentation pages
- Follow the existing structure in `mint.json` for navigation
- Use Mintlify components for enhanced formatting (Card, CardGroup, Accordion, etc.)
- Reference the [Mintlify documentation](https://mintlify.com/docs) for available components
To learn more about Next.js and Fumadocs, take a look at the following
resources:
## Deployment
Docs are automatically deployed when changes are pushed to the main branch.
To manually deploy:
```bash
mintlify deploy
```
## Contributing
See [CONTRIBUTING.md](../CONTRIBUTING.md) for contribution guidelines.
- [Next.js Documentation](https://nextjs.org/docs) - learn about Next.js
features and API.
- [Learn Next.js](https://nextjs.org/learn) - an interactive Next.js tutorial.
- [Fumadocs](https://fumadocs.dev) - learn about Fumadocs
-55
View File
@@ -1,55 +0,0 @@
---
title: "Authentication"
description: "API authentication and security"
---
## Current Status
<Warning>
Authentication is not currently implemented in Voicebox. The API is intended for local use only.
</Warning>
## Local Usage
For local development and usage:
- API runs on `localhost:17493`
- No authentication required
- Access restricted to local machine
## Future Implementation
Authentication will be added in a future release for:
- Remote deployments
- Multi-user access
- Production environments
Planned authentication methods:
- API keys
- OAuth 2.0
- JWT tokens
## Security Best Practices
Until authentication is implemented:
<CardGroup cols={2}>
<Card title="Use VPN" icon="shield">
Use WireGuard or Tailscale for remote access
</Card>
<Card title="Reverse Proxy" icon="server">
Run behind nginx with basic auth
</Card>
<Card title="Firewall" icon="fire">
Restrict access to trusted IPs only
</Card>
<Card title="Local Only" icon="laptop">
Don't expose to public internet
</Card>
</CardGroup>
## Coming Soon
- API key management
- User accounts
- Rate limiting
- Access control
-119
View File
@@ -1,119 +0,0 @@
---
title: "Generation API"
description: "Generate speech from text"
---
## Generate Speech
```http
POST /generate
```
**Request:**
```json
{
"text": "Hello world",
"profile_id": "abc123",
"language": "en"
}
```
**Response:**
```json
{
"id": "gen123",
"text": "Hello world",
"profile_id": "abc123",
"language": "en",
"audio_url": "/audio/gen123.wav",
"duration": 2.3,
"created_at": "2024-01-29T12:00:00Z"
}
```
## List History
```http
GET /history
```
**Query Parameters:**
- `profile_id` (optional) - Filter by voice profile
- `limit` (optional) - Number of results (default: 50)
- `offset` (optional) - Pagination offset
**Response:**
```json
{
"generations": [
{
"id": "gen123",
"text": "Hello world",
"profile_id": "abc123",
"duration": 2.3,
"created_at": "2024-01-29T12:00:00Z"
}
],
"total": 100
}
```
## Get Generation
```http
GET /history/{id}
```
**Response:**
```json
{
"id": "gen123",
"text": "Hello world",
"profile_id": "abc123",
"language": "en",
"audio_url": "/audio/gen123.wav",
"duration": 2.3,
"created_at": "2024-01-29T12:00:00Z"
}
```
## Delete Generation
```http
DELETE /history/{id}
```
**Response:**
```json
{
"success": true
}
```
## TypeScript Example
```typescript
import { VoiceboxClient } from '@/lib/api'
const client = new VoiceboxClient({
baseUrl: 'http://localhost:17493'
})
// Generate speech
const generation = await client.generate({
text: 'Hello world',
profile_id: 'abc123',
language: 'en'
})
// Get audio URL
const audioUrl = generation.audio_url
// List history
const history = await client.listHistory({
profile_id: 'abc123',
limit: 20
})
```
For full API documentation, visit `http://localhost:17493/docs` when the server is running.
-219
View File
@@ -1,219 +0,0 @@
---
title: "API Overview"
description: "Integrate voice synthesis into your applications with the Voicebox REST API"
---
## Introduction
Voicebox exposes a full REST API that allows you to integrate voice synthesis into your own applications. The API runs on `http://localhost:17493` by default.
<Card title="Interactive API Docs" icon="book" href="http://localhost:17493/docs">
When Voicebox is running, visit the auto-generated API documentation at `http://localhost:17493/docs`
</Card>
## Base URL
```
http://localhost:17493
```
For remote deployments, replace `localhost` with your server's IP or hostname.
## Authentication
<Note>
Currently, the API does not require authentication for local development. Authentication will be added in a future release for production deployments.
</Note>
## Quick Example
Here's a simple example of generating speech:
```bash
# Generate speech
curl -X POST http://localhost:17493/generate \
-H "Content-Type: application/json" \
-d '{
"text": "Hello world",
"profile_id": "abc123",
"language": "en"
}'
```
## API Endpoints
The Voicebox API is organized into several categories:
<CardGroup cols={2}>
<Card title="Voice Profiles" icon="user" href="/api/voice-profiles">
Create, list, update, and delete voice profiles
</Card>
<Card title="Generation" icon="waveform" href="/api/generation">
Generate speech from text using voice profiles
</Card>
<Card title="Recordings" icon="microphone" href="/api/recordings">
Record and transcribe audio
</Card>
<Card title="Stories" icon="film">
Create and manage multi-voice stories (coming soon)
</Card>
</CardGroup>
## Core Endpoints
### Voice Profiles
```http
GET /profiles # List all profiles
POST /profiles # Create a new profile
GET /profiles/{id} # Get profile details
PUT /profiles/{id} # Update a profile
DELETE /profiles/{id} # Delete a profile
POST /profiles/{id}/samples # Add voice sample
```
### Generation
```http
POST /generate # Generate speech
GET /history # List generation history
GET /history/{id} # Get generation details
DELETE /history/{id} # Delete from history
```
### Recordings
```http
POST /recordings # Start recording
POST /recordings/stop # Stop recording
POST /transcribe # Transcribe audio
```
## Response Format
All API responses follow a consistent JSON format:
```json
{
"success": true,
"data": {
// Response data
},
"error": null
}
```
Error responses:
```json
{
"success": false,
"data": null,
"error": {
"message": "Error description",
"code": "ERROR_CODE"
}
}
```
## Data Models
### Voice Profile
```json
{
"id": "abc123",
"name": "John Smith",
"language": "en",
"description": "Professional narrator voice",
"created_at": "2024-01-29T12:00:00Z",
"samples": [
{
"id": "sample123",
"audio_path": "/path/to/sample.wav",
"duration": 15.5
}
]
}
```
### Generation
```json
{
"id": "gen123",
"text": "Hello world",
"profile_id": "abc123",
"language": "en",
"audio_path": "/path/to/output.wav",
"duration": 2.3,
"created_at": "2024-01-29T12:00:00Z"
}
```
## TypeScript Client
Voicebox provides an auto-generated TypeScript client with full type safety:
```typescript
import { VoiceboxClient } from '@/lib/api'
const client = new VoiceboxClient({
baseUrl: 'http://localhost:17493'
})
// Create a profile
const profile = await client.createProfile({
name: 'John Smith',
language: 'en'
})
// Generate speech
const generation = await client.generate({
text: 'Hello world',
profile_id: profile.id,
language: 'en'
})
```
The client is automatically generated from the OpenAPI schema. See [Development Setup](/development/setup#generate-openapi-client) for details.
## Rate Limiting
<Info>
Currently, there are no rate limits for local usage. Rate limiting will be added in a future release for production deployments.
</Info>
## WebSocket Support
<Note>
Real-time streaming generation via WebSockets is planned for a future release.
</Note>
## Use Cases
<CardGroup cols={2}>
<Card title="Game Development" icon="gamepad">
Generate dynamic dialogue for NPCs and characters
</Card>
<Card title="Content Creation" icon="video">
Automate voiceovers for videos and podcasts
</Card>
<Card title="Accessibility" icon="universal-access">
Build text-to-speech tools for visually impaired users
</Card>
<Card title="Voice Assistants" icon="robot">
Create custom voice interfaces
</Card>
</CardGroup>
## Next Steps
<CardGroup cols={2}>
<Card title="Voice Profiles API" icon="user" href="/api/voice-profiles">
Learn how to manage voice profiles
</Card>
<Card title="Generation API" icon="waveform" href="/api/generation">
Generate speech from text
</Card>
</CardGroup>
-95
View File
@@ -1,95 +0,0 @@
---
title: "Recordings API"
description: "Record and transcribe audio"
---
## Start Recording
```http
POST /recordings/start
```
**Request:**
```json
{
"source": "microphone"
}
```
**Response:**
```json
{
"recording_id": "rec123",
"status": "recording"
}
```
## Stop Recording
```http
POST /recordings/stop
```
**Request:**
```json
{
"recording_id": "rec123"
}
```
**Response:**
```json
{
"recording_id": "rec123",
"audio_url": "/audio/rec123.wav",
"duration": 15.5
}
```
## Transcribe Audio
```http
POST /transcribe
```
**Request:** (multipart/form-data)
```
audio: <file>
language: "en" (optional)
```
**Response:**
```json
{
"text": "Transcribed speech text here",
"language": "en",
"duration": 15.5,
"confidence": 0.95
}
```
## TypeScript Example
```typescript
import { VoiceboxClient } from '@/lib/api'
const client = new VoiceboxClient({
baseUrl: 'http://localhost:17493'
})
// Start recording
const recording = await client.startRecording({
source: 'microphone'
})
// ... record audio ...
// Stop recording
const result = await client.stopRecording(recording.id)
// Transcribe
const transcription = await client.transcribe(audioFile, 'en')
console.log(transcription.text)
```
For full API documentation, visit `http://localhost:17493/docs` when the server is running.
-149
View File
@@ -1,149 +0,0 @@
---
title: "Voice Profiles API"
description: "Manage voice profiles programmatically"
---
## Endpoints
### List Profiles
```http
GET /profiles
```
**Response:**
```json
{
"profiles": [
{
"id": "abc123",
"name": "John Smith",
"language": "en",
"description": "Professional narrator",
"created_at": "2024-01-29T12:00:00Z",
"sample_count": 2
}
]
}
```
### Get Profile
```http
GET /profiles/{id}
```
**Response:**
```json
{
"id": "abc123",
"name": "John Smith",
"language": "en",
"description": "Professional narrator",
"created_at": "2024-01-29T12:00:00Z",
"samples": [
{
"id": "sample123",
"duration": 15.5,
"created_at": "2024-01-29T12:00:00Z"
}
]
}
```
### Create Profile
```http
POST /profiles
```
**Request:**
```json
{
"name": "John Smith",
"language": "en",
"description": "Professional narrator"
}
```
**Response:**
```json
{
"id": "abc123",
"name": "John Smith",
"language": "en",
"description": "Professional narrator",
"created_at": "2024-01-29T12:00:00Z"
}
```
### Update Profile
```http
PUT /profiles/{id}
```
**Request:**
```json
{
"name": "Updated Name",
"description": "Updated description"
}
```
### Delete Profile
```http
DELETE /profiles/{id}
```
**Response:**
```json
{
"success": true
}
```
### Add Voice Sample
```http
POST /profiles/{id}/samples
```
**Request:** (multipart/form-data)
```
audio: <file>
```
**Response:**
```json
{
"sample_id": "sample123",
"duration": 15.5
}
```
## TypeScript Example
```typescript
import { VoiceboxClient } from '@/lib/api'
const client = new VoiceboxClient({
baseUrl: 'http://localhost:17493'
})
// Create profile
const profile = await client.createProfile({
name: 'John Smith',
language: 'en',
description: 'Professional narrator'
})
// Add sample
await client.addSample(profile.id, audioFile)
// List all profiles
const profiles = await client.listProfiles()
```
For full API documentation, visit `http://localhost:17493/docs` when the server is running.
+6
View File
@@ -0,0 +1,6 @@
import { HomeLayout } from 'fumadocs-ui/layouts/home';
import { baseOptions } from '@/lib/layout.shared';
export default function Layout({ children }: LayoutProps<'/'>) {
return <HomeLayout {...baseOptions()}>{children}</HomeLayout>;
}
+5
View File
@@ -0,0 +1,5 @@
import { redirect } from 'next/navigation';
export default function HomePage() {
redirect('/docs');
}
+7
View File
@@ -0,0 +1,7 @@
import { source } from '@/lib/source';
import { createFromSource } from 'fumadocs-core/search/server';
export const { GET } = createFromSource(source, {
// https://docs.orama.com/docs/orama-js/supported-languages
language: 'english',
});
+74
View File
@@ -0,0 +1,74 @@
import { createRelativeLink } from 'fumadocs-ui/mdx';
import { DocsBody, DocsDescription, DocsPage, DocsTitle } from 'fumadocs-ui/page';
import type { Metadata } from 'next';
import { notFound } from 'next/navigation';
import { MarkdownCopyButton, ViewOptionsPopover } from '@/components/ai/page-actions';
import { APIPage } from '@/components/api-page';
import { getPageImage, source } from '@/lib/source';
import { getMDXComponents } from '@/mdx-components';
export default async function Page(props: PageProps<'/docs/[[...slug]]'>) {
const params = await props.params;
const page = source.getPage(params.slug);
if (!page) notFound();
const MDX = page.data.body;
const markdownUrl = `${page.url}.mdx`;
const githubUrl = `https://github.com/jamiepine/voicebox/blob/main/docs/content/docs/${page.path}`;
return (
<DocsPage
toc={page.data.toc}
full={page.data.full}
editOnGithub={{
owner: 'jamiepine',
repo: 'voicebox',
sha: 'main',
path: `docs/content/docs/${page.path}`,
}}
lastUpdate={page.data.lastModified}
>
<DocsTitle>{page.data.title}</DocsTitle>
<DocsDescription className="mb-0">{page.data.description}</DocsDescription>
<div className="flex flex-row gap-2 items-center">
<MarkdownCopyButton markdownUrl={markdownUrl} />
<ViewOptionsPopover markdownUrl={markdownUrl} githubUrl={githubUrl} />
</div>
<div
role="separator"
style={{
height: '1px',
background: 'currentColor',
opacity: 0.15,
marginTop: '8px',
marginBottom: '24px',
}}
/>
<DocsBody>
<MDX
components={getMDXComponents({
a: createRelativeLink(source, page),
})}
/>
</DocsBody>
</DocsPage>
);
}
export async function generateStaticParams() {
return source.generateParams();
}
export async function generateMetadata(props: PageProps<'/docs/[[...slug]]'>): Promise<Metadata> {
const params = await props.params;
const page = source.getPage(params.slug);
if (!page) notFound();
return {
title: page.data.title,
description: page.data.description,
openGraph: {
images: getPageImage(page).url,
},
};
}
+11
View File
@@ -0,0 +1,11 @@
import { source } from '@/lib/source';
import { DocsLayout } from 'fumadocs-ui/layouts/docs';
import { baseOptions } from '@/lib/layout.shared';
export default function Layout({ children }: LayoutProps<'/docs'>) {
return (
<DocsLayout tree={source.pageTree} {...baseOptions()}>
{children}
</DocsLayout>
);
}
+14
View File
@@ -0,0 +1,14 @@
@import 'tailwindcss';
@import 'fumadocs-ui/css/neutral.css';
@import 'fumadocs-ui/css/preset.css';
@import 'fumadocs-openapi/css/preset.css';
:root {
--color-fd-primary: hsl(43, 50%, 50%);
--color-fd-primary-foreground: hsl(222.2, 47.4%, 11.2%);
}
.dark {
--color-fd-primary: hsl(43, 50%, 45%);
--color-fd-primary-foreground: hsl(0, 0%, 95%);
}
+17
View File
@@ -0,0 +1,17 @@
import { RootProvider } from 'fumadocs-ui/provider/next';
import './global.css';
import { Inter } from 'next/font/google';
const inter = Inter({
subsets: ['latin'],
});
export default function Layout({ children }: LayoutProps<'/'>) {
return (
<html lang="en" className={inter.className} suppressHydrationWarning>
<body className="flex flex-col min-h-screen">
<RootProvider>{children}</RootProvider>
</body>
</html>
);
}
+10
View File
@@ -0,0 +1,10 @@
import { getLLMText, source } from '@/lib/source';
export const revalidate = false;
export async function GET() {
const scan = source.getPages().map(getLLMText);
const scanned = await Promise.all(scan);
return new Response(scanned.join('\n\n'));
}
@@ -0,0 +1,20 @@
import { notFound } from 'next/navigation';
import { getLLMText, source } from '@/lib/source';
export const revalidate = false;
export async function GET(_req: Request, { params }: RouteContext<'/llms.mdx/docs/[[...slug]]'>) {
const { slug } = await params;
const page = source.getPage(slug);
if (!page) notFound();
return new Response(await getLLMText(page), {
headers: {
'Content-Type': 'text/markdown',
},
});
}
export function generateStaticParams() {
return source.generateParams();
}
+36
View File
@@ -0,0 +1,36 @@
import { getPageImage, source } from '@/lib/source';
import { notFound } from 'next/navigation';
import { ImageResponse } from 'next/og';
import { generate as DefaultImage } from 'fumadocs-ui/og';
export const revalidate = false;
export async function GET(
_req: Request,
{ params }: RouteContext<'/og/docs/[...slug]'>,
) {
const { slug } = await params;
const page = source.getPage(slug.slice(0, -1));
if (!page) notFound();
return new ImageResponse(
(
<DefaultImage
title={page.data.title}
description={page.data.description}
site="My App"
/>
),
{
width: 1200,
height: 630,
},
);
}
export function generateStaticParams() {
return source.getPages().map((page) => ({
lang: page.locale,
slug: getPageImage(page).segments,
}));
}
+277 -1278
View File
File diff suppressed because it is too large Load Diff
+13
View File
@@ -0,0 +1,13 @@
{
"$schema": "node_modules/@fumadocs/cli/dist/schema/default.json",
"aliases": {
"uiDir": "./components/ui",
"componentsDir": "./components",
"blockDir": "./components",
"cssDir": "./styles",
"libDir": "./lib"
},
"baseDir": "",
"uiLibrary": "radix-ui",
"commands": {}
}
+247
View File
@@ -0,0 +1,247 @@
'use client';
import { type ComponentProps, useMemo, useState } from 'react';
import { Check, ChevronDown, Copy, ExternalLinkIcon, TextIcon } from 'lucide-react';
import { cn } from '../../lib/cn';
import { useCopyButton } from 'fumadocs-ui/utils/use-copy-button';
import { Popover, PopoverTrigger, PopoverContent } from '../ui/popover';
import { buttonVariants } from '../ui/button';
const cache = new Map<string, Promise<string>>();
export function MarkdownCopyButton({
markdownUrl,
...props
}: ComponentProps<'button'> & {
/**
* A URL to fetch the raw Markdown/MDX content of page
*/
markdownUrl: string;
}) {
const [isLoading, setLoading] = useState(false);
const [checked, onClick] = useCopyButton(async () => {
const cached = cache.get(markdownUrl);
if (cached) return navigator.clipboard.writeText(await cached);
setLoading(true);
try {
const promise = fetch(markdownUrl).then((res) => res.text());
cache.set(markdownUrl, promise);
await navigator.clipboard.write([
new ClipboardItem({
'text/plain': promise,
}),
]);
} finally {
setLoading(false);
}
});
return (
<button
disabled={isLoading}
onClick={onClick}
{...props}
className={cn(
buttonVariants({
variant: 'secondary',
size: 'sm',
className: 'gap-2 [&_svg]:size-3.5 [&_svg]:text-fd-muted-foreground',
}),
props.className,
)}
>
{checked ? <Check /> : <Copy />}
Copy Markdown
</button>
);
}
export function ViewOptionsPopover({
markdownUrl,
githubUrl,
...props
}: ComponentProps<typeof PopoverTrigger> & {
/**
* A URL to the raw Markdown/MDX content of page
*/
markdownUrl: string;
/**
* Source file URL on GitHub
*/
githubUrl: string;
}) {
const items = useMemo(() => {
const pageUrl = typeof window !== 'undefined' ? window.location.href : 'loading';
const q = `Read ${pageUrl}, I want to ask questions about it.`;
return [
{
title: 'Open in GitHub',
href: githubUrl,
icon: (
<svg fill="currentColor" role="img" viewBox="0 0 24 24">
<title>GitHub</title>
<path d="M12 .297c-6.63 0-12 5.373-12 12 0 5.303 3.438 9.8 8.205 11.385.6.113.82-.258.82-.577 0-.285-.01-1.04-.015-2.04-3.338.724-4.042-1.61-4.042-1.61C4.422 18.07 3.633 17.7 3.633 17.7c-1.087-.744.084-.729.084-.729 1.205.084 1.838 1.236 1.838 1.236 1.07 1.835 2.809 1.305 3.495.998.108-.776.417-1.305.76-1.605-2.665-.3-5.466-1.332-5.466-5.93 0-1.31.465-2.38 1.235-3.22-.135-.303-.54-1.523.105-3.176 0 0 1.005-.322 3.3 1.23.96-.267 1.98-.399 3-.405 1.02.006 2.04.138 3 .405 2.28-1.552 3.285-1.23 3.285-1.23.645 1.653.24 2.873.12 3.176.765.84 1.23 1.91 1.23 3.22 0 4.61-2.805 5.625-5.475 5.92.42.36.81 1.096.81 2.22 0 1.606-.015 2.896-.015 3.286 0 .315.21.69.825.57C20.565 22.092 24 17.592 24 12.297c0-6.627-5.373-12-12-12" />
</svg>
),
},
{
title: 'View as Markdown',
href: markdownUrl,
icon: <TextIcon />,
},
{
title: 'Open in Scira AI',
href: `https://scira.ai/?${new URLSearchParams({
q,
})}`,
icon: (
<svg
width="910"
height="934"
viewBox="0 0 910 934"
fill="none"
xmlns="http://www.w3.org/2000/svg"
>
<title>Scira AI</title>
<path
d="M647.664 197.775C569.13 189.049 525.5 145.419 516.774 66.8849C508.048 145.419 464.418 189.049 385.884 197.775C464.418 206.501 508.048 250.131 516.774 328.665C525.5 250.131 569.13 206.501 647.664 197.775Z"
fill="currentColor"
stroke="currentColor"
strokeWidth="8"
strokeLinejoin="round"
/>
<path
d="M516.774 304.217C510.299 275.491 498.208 252.087 480.335 234.214C462.462 216.341 439.058 204.251 410.333 197.775C439.059 191.3 462.462 179.209 480.335 161.336C498.208 143.463 510.299 120.06 516.774 91.334C523.25 120.059 535.34 143.463 553.213 161.336C571.086 179.209 594.49 191.3 623.216 197.775C594.49 204.251 571.086 216.341 553.213 234.214C535.34 252.087 523.25 275.491 516.774 304.217Z"
fill="currentColor"
stroke="currentColor"
strokeWidth="8"
strokeLinejoin="round"
/>
<path
d="M857.5 508.116C763.259 497.644 710.903 445.288 700.432 351.047C689.961 445.288 637.605 497.644 543.364 508.116C637.605 518.587 689.961 570.943 700.432 665.184C710.903 570.943 763.259 518.587 857.5 508.116Z"
stroke="currentColor"
strokeWidth="20"
strokeLinejoin="round"
/>
<path
d="M700.432 615.957C691.848 589.05 678.575 566.357 660.383 548.165C642.191 529.973 619.499 516.7 592.593 508.116C619.499 499.533 642.191 486.258 660.383 468.066C678.575 449.874 691.848 427.181 700.432 400.274C709.015 427.181 722.289 449.874 740.481 468.066C758.673 486.258 781.365 499.533 808.271 508.116C781.365 516.7 758.673 529.973 740.481 548.165C722.289 566.357 709.015 589.05 700.432 615.957Z"
stroke="currentColor"
strokeWidth="20"
strokeLinejoin="round"
/>
<path
d="M889.949 121.237C831.049 114.692 798.326 81.9698 791.782 23.0692C785.237 81.9698 752.515 114.692 693.614 121.237C752.515 127.781 785.237 160.504 791.782 219.404C798.326 160.504 831.049 127.781 889.949 121.237Z"
fill="currentColor"
stroke="currentColor"
strokeWidth="8"
strokeLinejoin="round"
/>
<path
d="M791.782 196.795C786.697 176.937 777.869 160.567 765.16 147.858C752.452 135.15 736.082 126.322 716.226 121.237C736.082 116.152 752.452 107.324 765.16 94.6152C777.869 81.9065 786.697 65.5368 791.782 45.6797C796.867 65.5367 805.695 81.9066 818.403 94.6152C831.112 107.324 847.481 116.152 867.338 121.237C847.481 126.322 831.112 135.15 818.403 147.858C805.694 160.567 796.867 176.937 791.782 196.795Z"
fill="currentColor"
stroke="currentColor"
strokeWidth="8"
strokeLinejoin="round"
/>
<path
d="M760.632 764.337C720.719 814.616 669.835 855.1 611.872 882.692C553.91 910.285 490.404 924.255 426.213 923.533C362.022 922.812 298.846 907.419 241.518 878.531C184.19 849.643 134.228 808.026 95.4548 756.863C56.6815 705.7 30.1238 646.346 17.8129 583.343C5.50207 520.339 7.76433 455.354 24.4266 393.359C41.089 331.364 71.7099 274.001 113.947 225.658C156.184 177.315 208.919 139.273 268.117 114.442"
stroke="currentColor"
strokeWidth="30"
strokeLinecap="round"
strokeLinejoin="round"
/>
</svg>
),
},
{
title: 'Open in ChatGPT',
href: `https://chatgpt.com/?${new URLSearchParams({
hints: 'search',
q,
})}`,
icon: (
<svg
role="img"
viewBox="0 0 24 24"
fill="currentColor"
xmlns="http://www.w3.org/2000/svg"
>
<title>OpenAI</title>
<path d="M22.2819 9.8211a5.9847 5.9847 0 0 0-.5157-4.9108 6.0462 6.0462 0 0 0-6.5098-2.9A6.0651 6.0651 0 0 0 4.9807 4.1818a5.9847 5.9847 0 0 0-3.9977 2.9 6.0462 6.0462 0 0 0 .7427 7.0966 5.98 5.98 0 0 0 .511 4.9107 6.051 6.051 0 0 0 6.5146 2.9001A5.9847 5.9847 0 0 0 13.2599 24a6.0557 6.0557 0 0 0 5.7718-4.2058 5.9894 5.9894 0 0 0 3.9977-2.9001 6.0557 6.0557 0 0 0-.7475-7.0729zm-9.022 12.6081a4.4755 4.4755 0 0 1-2.8764-1.0408l.1419-.0804 4.7783-2.7582a.7948.7948 0 0 0 .3927-.6813v-6.7369l2.02 1.1686a.071.071 0 0 1 .038.052v5.5826a4.504 4.504 0 0 1-4.4945 4.4944zm-9.6607-4.1254a4.4708 4.4708 0 0 1-.5346-3.0137l.142.0852 4.783 2.7582a.7712.7712 0 0 0 .7806 0l5.8428-3.3685v2.3324a.0804.0804 0 0 1-.0332.0615L9.74 19.9502a4.4992 4.4992 0 0 1-6.1408-1.6464zM2.3408 7.8956a4.485 4.485 0 0 1 2.3655-1.9728V11.6a.7664.7664 0 0 0 .3879.6765l5.8144 3.3543-2.0201 1.1685a.0757.0757 0 0 1-.071 0l-4.8303-2.7865A4.504 4.504 0 0 1 2.3408 7.872zm16.5963 3.8558L13.1038 8.364 15.1192 7.2a.0757.0757 0 0 1 .071 0l4.8303 2.7913a4.4944 4.4944 0 0 1-.6765 8.1042v-5.6772a.79.79 0 0 0-.407-.667zm2.0107-3.0231l-.142-.0852-4.7735-2.7818a.7759.7759 0 0 0-.7854 0L9.409 9.2297V6.8974a.0662.0662 0 0 1 .0284-.0615l4.8303-2.7866a4.4992 4.4992 0 0 1 6.6802 4.66zM8.3065 12.863l-2.02-1.1638a.0804.0804 0 0 1-.038-.0567V6.0742a4.4992 4.4992 0 0 1 7.3757-3.4537l-.142.0805L8.704 5.459a.7948.7948 0 0 0-.3927.6813zm1.0976-2.3654l2.602-1.4998 2.6069 1.4998v2.9994l-2.5974 1.4997-2.6067-1.4997Z" />
</svg>
),
},
{
title: 'Open in Claude',
href: `https://claude.ai/new?${new URLSearchParams({
q,
})}`,
icon: (
<svg
fill="currentColor"
role="img"
viewBox="0 0 24 24"
xmlns="http://www.w3.org/2000/svg"
>
<title>Anthropic</title>
<path d="M17.3041 3.541h-3.6718l6.696 16.918H24Zm-10.6082 0L0 20.459h3.7442l1.3693-3.5527h7.0052l1.3693 3.5528h3.7442L10.5363 3.5409Zm-.3712 10.2232 2.2914-5.9456 2.2914 5.9456Z" />
</svg>
),
},
{
title: 'Open in Cursor',
icon: (
<svg
fill="currentColor"
role="img"
viewBox="0 0 24 24"
xmlns="http://www.w3.org/2000/svg"
>
<title>Cursor</title>
<path d="M11.503.131 1.891 5.678a.84.84 0 0 0-.42.726v11.188c0 .3.162.575.42.724l9.609 5.55a1 1 0 0 0 .998 0l9.61-5.55a.84.84 0 0 0 .42-.724V6.404a.84.84 0 0 0-.42-.726L12.497.131a1.01 1.01 0 0 0-.996 0M2.657 6.338h18.55c.263 0 .43.287.297.515L12.23 22.918c-.062.107-.229.064-.229-.06V12.335a.59.59 0 0 0-.295-.51l-9.11-5.257c-.109-.063-.064-.23.061-.23" />
</svg>
),
href: `https://cursor.com/link/prompt?${new URLSearchParams({
text: q,
})}`,
},
];
}, [githubUrl, markdownUrl]);
return (
<Popover>
<PopoverTrigger
{...props}
className={cn(
buttonVariants({
variant: 'secondary',
size: 'sm',
}),
'gap-2 data-[state=open]:bg-fd-accent data-[state=open]:text-fd-accent-foreground',
props.className,
)}
>
Open
<ChevronDown className="size-3.5 text-fd-muted-foreground" />
</PopoverTrigger>
<PopoverContent className="flex flex-col">
{items.map((item) => (
<a
key={item.href}
href={item.href}
rel="noreferrer noopener"
target="_blank"
className="text-sm p-2 rounded-lg inline-flex items-center gap-2 hover:text-fd-accent-foreground hover:bg-fd-accent [&_svg]:size-4"
>
{item.icon}
{item.title}
<ExternalLinkIcon className="text-fd-muted-foreground size-3.5 ms-auto" />
</a>
))}
</PopoverContent>
</Popover>
);
}
+6
View File
@@ -0,0 +1,6 @@
'use client';
import { defineClientConfig } from 'fumadocs-openapi/ui/client';
export default defineClientConfig({
// Client-side configuration for API playground
});
+7
View File
@@ -0,0 +1,7 @@
import { openapi } from '@/lib/openapi';
import { createAPIPage } from 'fumadocs-openapi/ui';
import client from './api-page.client';
export const APIPage = createAPIPage(openapi, {
client,
});
+29
View File
@@ -0,0 +1,29 @@
import { cva, type VariantProps } from 'class-variance-authority';
const variants = {
primary:
'bg-fd-primary text-fd-primary-foreground hover:bg-fd-primary/80 disabled:bg-fd-secondary disabled:text-fd-secondary-foreground',
outline: 'border hover:bg-fd-accent hover:text-fd-accent-foreground',
ghost: 'hover:bg-fd-accent hover:text-fd-accent-foreground',
secondary:
'border bg-fd-secondary text-fd-secondary-foreground hover:bg-fd-accent hover:text-fd-accent-foreground',
} as const;
export const buttonVariants = cva(
'inline-flex items-center justify-center rounded-md p-2 text-sm font-medium transition-colors duration-100 disabled:pointer-events-none disabled:opacity-50 focus-visible:outline-none focus-visible:ring-2 focus-visible:ring-fd-ring',
{
variants: {
variant: variants,
// fumadocs use `color` instead of `variant`
color: variants,
size: {
sm: 'gap-1 px-2 py-1.5 text-xs',
icon: 'p-1.5 [&_svg]:size-5',
'icon-sm': 'p-1.5 [&_svg]:size-4.5',
'icon-xs': 'p-1 [&_svg]:size-4',
},
},
},
);
export type ButtonProps = VariantProps<typeof buttonVariants>;
+32
View File
@@ -0,0 +1,32 @@
'use client';
import * as PopoverPrimitive from '@radix-ui/react-popover';
import * as React from 'react';
import { cn } from '../../lib/cn';
const Popover = PopoverPrimitive.Root;
const PopoverTrigger = PopoverPrimitive.Trigger;
const PopoverContent = React.forwardRef<
React.ComponentRef<typeof PopoverPrimitive.Content>,
React.ComponentPropsWithoutRef<typeof PopoverPrimitive.Content>
>(({ className, align = 'center', sideOffset = 4, ...props }, ref) => (
<PopoverPrimitive.Portal>
<PopoverPrimitive.Content
ref={ref}
align={align}
sideOffset={sideOffset}
side="bottom"
className={cn(
'z-50 origin-(--radix-popover-content-transform-origin) overflow-y-auto max-h-(--radix-popover-content-available-height) min-w-[240px] max-w-[98vw] rounded-xl border bg-fd-popover/60 backdrop-blur-lg p-2 text-sm text-fd-popover-foreground shadow-lg focus-visible:outline-none data-[state=closed]:animate-fd-popover-out data-[state=open]:animate-fd-popover-in',
className,
)}
{...props}
/>
</PopoverPrimitive.Portal>
));
PopoverContent.displayName = PopoverPrimitive.Content.displayName;
const PopoverClose = PopoverPrimitive.PopoverClose;
export { Popover, PopoverTrigger, PopoverContent, PopoverClose };
+67
View File
@@ -0,0 +1,67 @@
---
title: "Documentation README"
description: "Voicebox documentation development guide"
---
This directory contains the documentation for Voicebox, built with [Fumadocs](https://fumadocs.dev).
## Development
### Prerequisites
Install Mintlify globally using bun:
```bash
bun add -g mintlify
```
Or use the helper script:
```bash
bun run install:mintlify
```
### Running Locally
```bash
bun run dev
```
This will start the Mintlify dev server.
The docs will be available at `http://localhost:3000`
### Structure
```
docs/
├── mint.json # Mintlify configuration
├── custom.css # Custom styles
├── overview/ # Getting started & feature docs
├── guides/ # User guides
├── api/ # API reference
├── development/ # Developer documentation
├── logo/ # Logo assets
└── public/ # Static assets
```
### Writing Docs
- Use `.mdx` files for all documentation pages
- Follow the existing structure in `mint.json` for navigation
- Use Mintlify components for enhanced formatting (Card, CardGroup, Accordion, etc.)
- Reference the [Mintlify documentation](https://mintlify.com/docs) for available components
## Deployment
Docs are automatically deployed when changes are pushed to the main branch.
To manually deploy:
```bash
mintlify deploy
```
## Contributing
See [CONTRIBUTING.md](../CONTRIBUTING.md) for contribution guidelines.
@@ -1,4 +1,7 @@
# Troubleshooting Guide
---
title: "Troubleshooting Guide"
description: "Common issues and solutions for Voicebox"
---
Common issues and solutions for Voicebox.
@@ -0,0 +1,5 @@
{
"title": "API Reference",
"defaultOpen": true,
"pages": ["unknown"]
}
@@ -0,0 +1,16 @@
---
title: Add Profile Sample
description: Add a sample to a voice profile.
full: true
_openapi:
method: POST
toc: []
structuredData:
headings: []
contents:
- content: Add a sample to a voice profile.
---
{/* This file was generated by Fumadocs. Do not edit this file directly. Any changes should be made by running the generation command again. */}
<APIPage document={"./openapi.json"} operations={[{"path":"/profiles/{profile_id}/samples","method":"post"}]} />
@@ -0,0 +1,16 @@
---
title: Create Profile
description: Create a new voice profile.
full: true
_openapi:
method: POST
toc: []
structuredData:
headings: []
contents:
- content: Create a new voice profile.
---
{/* This file was generated by Fumadocs. Do not edit this file directly. Any changes should be made by running the generation command again. */}
<APIPage document={"./openapi.json"} operations={[{"path":"/profiles","method":"post"}]} />
@@ -0,0 +1,16 @@
---
title: Delete Generation
description: Delete a generation.
full: true
_openapi:
method: DELETE
toc: []
structuredData:
headings: []
contents:
- content: Delete a generation.
---
{/* This file was generated by Fumadocs. Do not edit this file directly. Any changes should be made by running the generation command again. */}
<APIPage document={"./openapi.json"} operations={[{"path":"/history/{generation_id}","method":"delete"}]} />
@@ -0,0 +1,16 @@
---
title: Delete Profile
description: Delete a voice profile.
full: true
_openapi:
method: DELETE
toc: []
structuredData:
headings: []
contents:
- content: Delete a voice profile.
---
{/* This file was generated by Fumadocs. Do not edit this file directly. Any changes should be made by running the generation command again. */}
<APIPage document={"./openapi.json"} operations={[{"path":"/profiles/{profile_id}","method":"delete"}]} />
@@ -0,0 +1,16 @@
---
title: Delete Profile Sample
description: Delete a profile sample.
full: true
_openapi:
method: DELETE
toc: []
structuredData:
headings: []
contents:
- content: Delete a profile sample.
---
{/* This file was generated by Fumadocs. Do not edit this file directly. Any changes should be made by running the generation command again. */}
<APIPage document={"./openapi.json"} operations={[{"path":"/profiles/samples/{sample_id}","method":"delete"}]} />
@@ -0,0 +1,16 @@
---
title: Generate Speech
description: Generate speech from text using a voice profile.
full: true
_openapi:
method: POST
toc: []
structuredData:
headings: []
contents:
- content: Generate speech from text using a voice profile.
---
{/* This file was generated by Fumadocs. Do not edit this file directly. Any changes should be made by running the generation command again. */}
<APIPage document={"./openapi.json"} operations={[{"path":"/generate","method":"post"}]} />
@@ -0,0 +1,16 @@
---
title: Get Audio
description: Serve generated audio file.
full: true
_openapi:
method: GET
toc: []
structuredData:
headings: []
contents:
- content: Serve generated audio file.
---
{/* This file was generated by Fumadocs. Do not edit this file directly. Any changes should be made by running the generation command again. */}
<APIPage document={"./openapi.json"} operations={[{"path":"/audio/{generation_id}","method":"get"}]} />
@@ -0,0 +1,16 @@
---
title: Get Generation
description: Get a generation by ID.
full: true
_openapi:
method: GET
toc: []
structuredData:
headings: []
contents:
- content: Get a generation by ID.
---
{/* This file was generated by Fumadocs. Do not edit this file directly. Any changes should be made by running the generation command again. */}
<APIPage document={"./openapi.json"} operations={[{"path":"/history/{generation_id}","method":"get"}]} />
@@ -0,0 +1,16 @@
---
title: Get Model Progress
description: Get model download progress via Server-Sent Events.
full: true
_openapi:
method: GET
toc: []
structuredData:
headings: []
contents:
- content: Get model download progress via Server-Sent Events.
---
{/* This file was generated by Fumadocs. Do not edit this file directly. Any changes should be made by running the generation command again. */}
<APIPage document={"./openapi.json"} operations={[{"path":"/models/progress/{model_name}","method":"get"}]} />
@@ -0,0 +1,16 @@
---
title: Get Model Status
description: Get status of all available models.
full: true
_openapi:
method: GET
toc: []
structuredData:
headings: []
contents:
- content: Get status of all available models.
---
{/* This file was generated by Fumadocs. Do not edit this file directly. Any changes should be made by running the generation command again. */}
<APIPage document={"./openapi.json"} operations={[{"path":"/models/status","method":"get"}]} />
@@ -0,0 +1,16 @@
---
title: Get Profile
description: Get a voice profile by ID.
full: true
_openapi:
method: GET
toc: []
structuredData:
headings: []
contents:
- content: Get a voice profile by ID.
---
{/* This file was generated by Fumadocs. Do not edit this file directly. Any changes should be made by running the generation command again. */}
<APIPage document={"./openapi.json"} operations={[{"path":"/profiles/{profile_id}","method":"get"}]} />
@@ -0,0 +1,16 @@
---
title: Get Profile Samples
description: Get all samples for a profile.
full: true
_openapi:
method: GET
toc: []
structuredData:
headings: []
contents:
- content: Get all samples for a profile.
---
{/* This file was generated by Fumadocs. Do not edit this file directly. Any changes should be made by running the generation command again. */}
<APIPage document={"./openapi.json"} operations={[{"path":"/profiles/{profile_id}/samples","method":"get"}]} />
@@ -0,0 +1,16 @@
---
title: Get Stats
description: Get generation statistics.
full: true
_openapi:
method: GET
toc: []
structuredData:
headings: []
contents:
- content: Get generation statistics.
---
{/* This file was generated by Fumadocs. Do not edit this file directly. Any changes should be made by running the generation command again. */}
<APIPage document={"./openapi.json"} operations={[{"path":"/history/stats","method":"get"}]} />
@@ -0,0 +1,16 @@
---
title: Health
description: Health check endpoint.
full: true
_openapi:
method: GET
toc: []
structuredData:
headings: []
contents:
- content: Health check endpoint.
---
{/* This file was generated by Fumadocs. Do not edit this file directly. Any changes should be made by running the generation command again. */}
<APIPage document={"./openapi.json"} operations={[{"path":"/health","method":"get"}]} />
@@ -0,0 +1,16 @@
---
title: List History
description: List generation history with optional filters.
full: true
_openapi:
method: GET
toc: []
structuredData:
headings: []
contents:
- content: List generation history with optional filters.
---
{/* This file was generated by Fumadocs. Do not edit this file directly. Any changes should be made by running the generation command again. */}
<APIPage document={"./openapi.json"} operations={[{"path":"/history","method":"get"}]} />
@@ -0,0 +1,16 @@
---
title: List Profiles
description: List all voice profiles.
full: true
_openapi:
method: GET
toc: []
structuredData:
headings: []
contents:
- content: List all voice profiles.
---
{/* This file was generated by Fumadocs. Do not edit this file directly. Any changes should be made by running the generation command again. */}
<APIPage document={"./openapi.json"} operations={[{"path":"/profiles","method":"get"}]} />
@@ -0,0 +1,16 @@
---
title: Load Model
description: Manually load TTS model.
full: true
_openapi:
method: POST
toc: []
structuredData:
headings: []
contents:
- content: Manually load TTS model.
---
{/* This file was generated by Fumadocs. Do not edit this file directly. Any changes should be made by running the generation command again. */}
<APIPage document={"./openapi.json"} operations={[{"path":"/models/load","method":"post"}]} />
@@ -0,0 +1,27 @@
{
"title": "Endpoints",
"pages": [
"root__get",
"health_health_get",
"list_profiles_profiles_get",
"create_profile_profiles_post",
"get_profile_profiles__profile_id__get",
"update_profile_profiles__profile_id__put",
"delete_profile_profiles__profile_id__delete",
"add_profile_sample_profiles__profile_id__samples_post",
"get_profile_samples_profiles__profile_id__samples_get",
"delete_profile_sample_profiles_samples__sample_id__delete",
"generate_speech_generate_post",
"list_history_history_get",
"get_generation_history__generation_id__get",
"delete_generation_history__generation_id__delete",
"get_stats_history_stats_get",
"transcribe_audio_transcribe_post",
"get_audio_audio__generation_id__get",
"load_model_models_load_post",
"unload_model_models_unload_post",
"get_model_progress_models_progress__model_name__get",
"get_model_status_models_status_get",
"trigger_model_download_models_download_post"
]
}
@@ -0,0 +1,16 @@
---
title: Root
description: Root endpoint.
full: true
_openapi:
method: GET
toc: []
structuredData:
headings: []
contents:
- content: Root endpoint.
---
{/* This file was generated by Fumadocs. Do not edit this file directly. Any changes should be made by running the generation command again. */}
<APIPage document={"./openapi.json"} operations={[{"path":"/","method":"get"}]} />
@@ -0,0 +1,16 @@
---
title: Transcribe Audio
description: Transcribe audio file to text.
full: true
_openapi:
method: POST
toc: []
structuredData:
headings: []
contents:
- content: Transcribe audio file to text.
---
{/* This file was generated by Fumadocs. Do not edit this file directly. Any changes should be made by running the generation command again. */}
<APIPage document={"./openapi.json"} operations={[{"path":"/transcribe","method":"post"}]} />
@@ -0,0 +1,16 @@
---
title: Trigger Model Download
description: Trigger download of a specific model.
full: true
_openapi:
method: POST
toc: []
structuredData:
headings: []
contents:
- content: Trigger download of a specific model.
---
{/* This file was generated by Fumadocs. Do not edit this file directly. Any changes should be made by running the generation command again. */}
<APIPage document={"./openapi.json"} operations={[{"path":"/models/download","method":"post"}]} />
@@ -0,0 +1,16 @@
---
title: Unload Model
description: Unload TTS model to free memory.
full: true
_openapi:
method: POST
toc: []
structuredData:
headings: []
contents:
- content: Unload TTS model to free memory.
---
{/* This file was generated by Fumadocs. Do not edit this file directly. Any changes should be made by running the generation command again. */}
<APIPage document={"./openapi.json"} operations={[{"path":"/models/unload","method":"post"}]} />
@@ -0,0 +1,16 @@
---
title: Update Profile
description: Update a voice profile.
full: true
_openapi:
method: PUT
toc: []
structuredData:
headings: []
contents:
- content: Update a voice profile.
---
{/* This file was generated by Fumadocs. Do not edit this file directly. Any changes should be made by running the generation command again. */}
<APIPage document={"./openapi.json"} operations={[{"path":"/profiles/{profile_id}","method":"put"}]} />
@@ -26,19 +26,22 @@ These two layers communicate via HTTP, with the frontend making API requests to
### Component Structure
```
app/src/
├── components/ # React components
│ ├── profiles/ # Voice profile UI
│ ├── generation/ # Speech generation UI
│ ├── stories/ # Timeline editor
│ └── shared/ # Reusable components
├── lib/ # Utilities
│ ├── api/ # Generated API client
│ └── utils/ # Helper functions
├── hooks/ # React hooks
└── stores/ # Zustand state stores
```
<Files>
<Folder name="app/src" defaultOpen>
<Folder name="components">
<File name="profiles/" />
<File name="generation/" />
<File name="stories/" />
<File name="shared/" />
</Folder>
<Folder name="lib">
<File name="api/" />
<File name="utils/" />
</Folder>
<Folder name="hooks" />
<Folder name="stores" />
</Folder>
</Files>
### State Management
@@ -64,16 +67,69 @@ const useProfileStore = create((set) => ({
### API Structure
```python
# main.py - API routes
@app.post("/generate")
async def generate_speech(request: GenerateRequest):
# 1. Validate request
# 2. Load voice profile
# 3. Generate audio with TTS
# 4. Save to database
# 5. Return response
```
<Files>
<Folder name="backend" defaultOpen>
<File name="app.py" />
<File name="main.py" />
<File name="config.py" />
<File name="models.py" />
<File name="server.py" />
<Folder name="routes">
<File name="profiles.py" />
<File name="generate.py" />
<File name="history.py" />
<File name="..." />
</Folder>
<Folder name="services">
<File name="generation.py" />
<File name="task_queue.py" />
<File name="..." />
</Folder>
<Folder name="backends">
<File name="__init__.py" />
<File name="base.py" />
<File name="..." />
</Folder>
<Folder name="database">
<File name="models.py" />
<File name="session.py" />
</Folder>
<Folder name="utils">
<File name="audio.py" />
<File name="effects.py" />
<File name="..." />
</Folder>
</Folder>
</Files>
### Request Flow
HTTP request → **routes/** (validate input, parse params) → **services/** (business logic, orchestration) → **backends/** (TTS/STT inference) → **utils/** (audio processing)
Route handlers are intentionally thin. They validate input, delegate to a service function, and format the response. All business logic lives in `services/`.
### Key Modules
- **app.py** — FastAPI app factory, CORS, lifecycle events
- **main.py** — Entry point (imports app, runs uvicorn)
- **server.py** — Tauri sidecar launcher, parent-pid watchdog
- **services/generation.py** — Single function handling all generation modes (generate, retry, regenerate)
- **services/task_queue.py** — Serial generation queue for GPU inference
- **backends/__init__.py** — Protocol definitions and backend factory
- **backends/base.py** — Shared utilities across all engine implementations
### Backend Selection
The server detects the best inference backend at startup:
| Platform | Backend | Acceleration |
|----------|---------|-------------|
| macOS (Apple Silicon) | MLX | Metal / Neural Engine |
| Windows / Linux (NVIDIA) | PyTorch | CUDA |
| Linux (AMD) | PyTorch | ROCm |
| Intel Arc | PyTorch | IPEX / XPU |
| Windows (any GPU) | PyTorch | DirectML |
| Any | PyTorch | CPU fallback |
### Data Model
@@ -89,11 +145,13 @@ The database uses three main tables:
### Rust Backend
```rust
// Sidecar process management
// File system access
// Native integrations
```
<Files>
<Folder name="tauri/src-tauri" defaultOpen>
<File name="Cargo.toml" />
<File name="src/" />
<Folder name="binaries" />
</Folder>
</Files>
### Responsibilities
@@ -196,11 +254,11 @@ When a user generates speech, the data flows through the following stages:
## Next Steps
<CardGroup cols={2}>
<Card title="Development Setup" icon="code" href="/development/setup">
<Cards>
<Card title="Development Setup" href="/development/setup">
Set up your dev environment
</Card>
<Card title="Contributing" icon="code-pull-request" href="/development/contributing">
<Card title="Contributing" href="/development/contributing">
Contribute to Voicebox
</Card>
</CardGroup>
</Cards>
+226
View File
@@ -0,0 +1,226 @@
---
title: "Auto-Updater"
description: "How Voicebox automatic updates work"
---
## Overview
Voicebox uses Tauri's built-in auto-updater to deliver signed updates to users. The system verifies updates cryptographically before installation.
## How It Works
When Voicebox launches (in production Tauri builds only), it checks GitHub Releases for a `latest.json` manifest. If a newer version is available:
1. **Notification** - An update banner appears at the top of the app
2. **Download** - User clicks "Install Now" to download the update package
3. **Verification** - The downloaded package is cryptographically verified using the public key embedded in `tauri.conf.json`
4. **Installation** - After verification, the update is installed
5. **Restart** - The app restarts automatically with the new version
Users can also check for updates manually via **Settings → Check for Updates**.
## Configuration
The updater is configured in `tauri/src-tauri/tauri.conf.json`:
```json
{
"plugins": {
"updater": {
"active": true,
"dialog": false,
"endpoints": [
"https://github.com/jamiepine/voicebox/releases/latest/download/latest.json"
],
"pubkey": "PASTE_PUBLIC_KEY_CONTENT_HERE"
}
}
}
```
**Key settings:**
- `endpoints` - URL to the `latest.json` manifest (checked on app startup)
- `pubkey` - Public key for verifying update signatures
- `dialog` - Set to `false` (we use custom UI instead of Tauri's built-in dialog)
## Release Manifest
The `latest.json` file defines available updates per platform:
```json
{
"version": "0.2.0",
"notes": "Bug fixes and improvements",
"pub_date": "2026-01-25T12:00:00Z",
"platforms": {
"darwin-aarch64": {
"signature": "base64_encoded_signature",
"url": "https://github.com/jamiepine/voicebox/releases/download/v0.2.0/voicebox_0.2.0_aarch64.app.tar.gz"
},
"darwin-x86_64": {
"signature": "base64_encoded_signature",
"url": "https://github.com/jamiepine/voicebox/releases/download/v0.2.0/voicebox_0.2.0_x64.app.tar.gz"
},
"linux-x86_64": {
"signature": "base64_encoded_signature",
"url": "https://github.com/jamiepine/voicebox/releases/download/v0.2.0/voicebox_0.2.0_amd64.AppImage"
},
"windows-x86_64": {
"signature": "base64_encoded_signature",
"url": "https://github.com/jamiepine/voicebox/releases/download/v0.2.0/voicebox_0.2.0_x64_en-US.msi"
}
}
}
```
## Signing
Updates must be cryptographically signed to be accepted. The signing process:
1. **Generate keys** (one-time setup):
```bash
bun tauri signer generate -w ~/.tauri/voicebox.key
```
This creates:
- Private key: `~/.tauri/voicebox.key` (stored in GitHub Secrets, never committed)
- Public key: `~/.tauri/voicebox.key.pub` (pasted into `tauri.conf.json`)
2. **Build with signing** (GitHub Actions handles this):
- Set `TAURI_SIGNING_PRIVATE_KEY` environment variable
- Tauri signs the update package during build
- Generates `.sig` signature file alongside the installer
3. **Verification** - The updater compares the signature against the public key before installing
## GitHub Actions Workflow
The release workflow (`.github/workflows/release.yml`) automatically:
- Builds signed releases for macOS, Windows, and Linux
- Creates the `latest.json` manifest with signatures
- Uploads everything to the GitHub Release
Triggered by pushing a git tag:
```bash
git tag v0.2.0 && git push --tags
```
## Environment Variables
GitHub Actions needs these secrets set:
- `TAURI_SIGNING_PRIVATE_KEY` - Content of `~/.tauri/voicebox.key`
- `TAURI_SIGNING_PRIVATE_KEY_PASSWORD` - Password for the key (if set)
## Security
<Callout type="warn">
**Critical:** Never commit the private key. Store it only in GitHub Secrets. The public key in `tauri.conf.json` is safe to commit and distribute.
</Callout>
- Updates are cryptographically signed using Ed25519
- HTTP endpoints are blocked (HTTPS only)
- Signature verification happens before installation
- Failed verification aborts the update
## Troubleshooting
### "Invalid signature" error
- Public key in `tauri.conf.json` doesn't match the private key used to sign
- Signature file wasn't uploaded to the release
### "No update available" when one exists
- `latest.json` version isn't higher than current version
- Wrong endpoint URL in configuration
- Manifest hasn't propagated to GitHub's CDN yet
### Update check fails in dev mode
The updater only works in production Tauri builds. It doesn't run during `just dev` or web mode.
### Build fails with signing error
- GitHub Secrets aren't set correctly
- Private key file is missing or corrupted
- Key format is wrong (should start with `dW50cnVzdGVkIGNvbW1lbnQ6`)
## CUDA Backend Updates
The CUDA-enabled backend is distributed separately from the main app due to its large size (~2.43 GB). Unlike the Tauri auto-updater, this uses a custom download system built into the Python backend.
**Size comparison:**
- Standard app bundle: ~410 MB
- CUDA backend binary: ~2.43 GB (6× larger)
### Why Split?
GitHub Releases has file size limits, and the CUDA-enabled `voicebox-server` binary is too large to include in the main Tauri bundle. Instead:
- **Standard release**: Includes CPU-only backend (~50MB)
- **CUDA release**: Split into multiple parts and downloaded on-demand by users who need GPU acceleration
### Download Process
When a user clicks "Enable CUDA" in the settings:
1. **Manifest Fetch** - Backend fetches `{version}/voicebox-server-cuda.manifest` from GitHub Releases
2. **Part Download** - Downloads each split part sequentially (e.g., `voicebox-server-cuda.part1`, `.part2`, etc.)
3. **Assembly** - Concatenates parts into a single binary
4. **Verification** - SHA-256 checksum verification (optional, if `.sha256` file exists)
5. **Placement** - Binary moved to `{data_dir}/backends/voicebox-server-cuda.exe`
6. **Restart** - Backend must restart to use the CUDA binary
### Auto-Update on Startup
On server startup, `check_and_update_cuda_binary()` compares the installed CUDA binary version with the app version:
```python
# backend/services/cuda.py
cuda_version = get_cuda_binary_version() # runs `voicebox-server-cuda --version`
current_version = __version__
if cuda_version != current_version:
await download_cuda_binary() # Auto-download in background
```
If versions mismatch, the backend automatically downloads the matching CUDA binary version without user intervention.
### Storage Location
Downloaded CUDA binaries are stored in the app's data directory:
```
{data_dir}/
backends/
voicebox-server-cuda.exe # Windows
voicebox-server-cuda # macOS/Linux
```
### API Endpoints
| Endpoint | Method | Description |
|----------|--------|-------------|
| `/backend/cuda-status` | GET | Check if CUDA binary available/active |
| `/backend/download-cuda` | POST | Start download |
| `/backend/cuda-progress` | GET | SSE stream of download progress |
| `/backend/cuda` | DELETE | Remove downloaded binary |
### Progress Tracking
Downloads report progress via Server-Sent Events (SSE):
```
GET /backend/cuda-progress
event: progress
data: {"current": 52428800, "total": 104857600, "filename": "Downloading CUDA backend (2/4)", "status": "downloading"}
```
The frontend subscribes to this endpoint to show real-time download progress in the UI.
### Release Artifacts
For each release, these CUDA-related files are uploaded to GitHub:
- `voicebox-server-cuda.manifest` - List of split part filenames
- `voicebox-server-cuda.part1` through `voicebox-server-cuda.partN` - Binary chunks
- `voicebox-server-cuda.sha256` - SHA-256 checksum for integrity verification
+190
View File
@@ -0,0 +1,190 @@
---
title: "Building"
description: "How Voicebox is built for production"
---
## Overview
Voicebox uses a two-stage build process:
1. **Python Server Binary** — PyInstaller bundles the FastAPI backend into a standalone executable
2. **Tauri Desktop App** — Bundles the React frontend, Rust wrapper, and Python server as a sidecar
## Build Commands
```bash
just build # Build everything (server + Tauri)
just build-server # Build Python server binary only
just build-tauri # Build Tauri app only
```
## Server Binary Build
### Build Script
`scripts/build-server.sh` orchestrates the build:
```bash
# Determine platform (e.g., x86_64-apple-darwin)
PLATFORM=$(rustc --print host-tuple)
# Run PyInstaller via build_binary.py
cd backend
python build_binary.py
# Copy to Tauri's binaries directory
cp dist/voicebox-server ../tauri/src-tauri/binaries/voicebox-server-${PLATFORM}
```
### PyInstaller Configuration
`backend/build_binary.py` contains the PyInstaller configuration:
**Entry Point:** Uses `server.py` (not `main.py`) for Tauri sidecar support
**Key Options:**
- `--onefile` — Single executable
- `--hidden-import` — Explicitly import modules PyInstaller can't detect
- `--collect-all` — Bundle data files and native libraries for packages like `mlx`, `zipvoice`
- `--exclude-module` — Strip NVIDIA packages from CPU builds
**Platform-Specific Logic:**
```python
# Apple Silicon — include MLX backend
if is_apple_silicon() and not cuda:
args.extend([
"--hidden-import", "mlx",
"--collect-all", "mlx", # Bundles .dylib and .metallib files
])
# CUDA builds — include torch.cuda
if cuda:
args.extend(["--hidden-import", "torch.cuda"])
# CPU builds — exclude NVIDIA packages to save ~3GB
else:
for pkg in ["nvidia", "nvidia.cublas", "nvidia.cudnn", ...]:
args.extend(["--exclude-module", pkg])
```
**Environment Variable:**
```bash
export QWEN_TTS_PATH=~/path/to/Qwen3-TTS # Use local Qwen3-TTS source
```
### CUDA Binary
The CUDA-enabled server is built separately due to size (~2.43 GB vs ~410 MB CPU version):
```bash
cd backend
python build_binary.py --cuda
```
The resulting binary is too large for GitHub Releases, so it's split into parts for distribution (see Auto-Updater docs for the download mechanism).
## Tauri App Build
Tauri bundles everything together:
```bash
cd tauri
bun run tauri build
```
**What happens:**
1. Vite builds the React frontend
2. Rust compiles the Tauri wrapper
3. Sidecar binary is copied from `src-tauri/binaries/`
4. Platform-specific installer created (DMG, MSI, AppImage)
**Output locations:**
<Files>
<Folder name="tauri/src-tauri/target/release/bundle" defaultOpen>
<File name="dmg/" />
<File name="msi/" />
<File name="nsis/" />
<File name="appimage/" />
</Folder>
</Files>
### Sidecar Configuration
The server binary is declared as an external binary in `tauri.conf.json`:
```json
{
"tauri": {
"bundle": {
"externalBin": ["binaries/voicebox-server"]
}
}
}
```
Tauri looks for `voicebox-server-${PLATFORM}` in `src-tauri/binaries/` and bundles it.
## GitHub Actions Release
`.github/workflows/release.yml` automates the full build:
### Matrix Strategy
| Platform | Target | Backend | Notes |
|----------|--------|---------|-------|
| macos-latest | aarch64-apple-darwin | MLX | Apple Silicon native |
| macos-15-intel | x86_64-apple-darwin | PyTorch | Intel Macs |
| windows-latest | x86_64-pc-windows-msvc | PyTorch | Windows with CUDA optional |
### Build Steps
1. **Setup** — Python, Rust, Bun, dependencies
2. **Build Server** — `build-server.sh` (Unix) or `build_binary.py` (Windows)
3. **Build Tauri** — `tauri-action` with signing keys
4. **Upload** — Release artifacts and `latest.json`
### Code Signing
**macOS:**
- Apple Developer certificate imported from secrets
- Notarization via App Store Connect API
**Windows:**
- Tauri handles signing via `TAURI_SIGNING_PRIVATE_KEY`
### CUDA Binary (Separate Job)
The `build-cuda-windows` job runs separately:
1. Install PyTorch with CUDA 12.1
2. Build with `build_binary.py --cuda`
3. Split binary with `scripts/split_binary.py`
4. Upload parts as release artifacts
This binary is downloaded on-demand by users who enable CUDA in settings.
## Troubleshooting
<AccordionGroup>
<Accordion title="Binary not found in dist/">
PyInstaller failed to create the output. Check:
- Python venv is activated
- All dependencies installed: `pip install -r requirements.txt`
- PyInstaller installed: `pip install pyinstaller`
</Accordion>
<Accordion title="MLX/Metal libraries missing in bundle">
macOS Apple Silicon builds need `--collect-all mlx` to include `.dylib` and `.metallib` files, not just `--collect-data`.
</Accordion>
<Accordion title="CUDA DLLs bloating CPU build">
If building CPU version but CUDA torch is installed locally, the script auto-detects and swaps to CPU torch temporarily, then restores CUDA torch after.
</Accordion>
<Accordion title="Tauri can't find sidecar">
Ensure binary exists at `tauri/src-tauri/binaries/voicebox-server-${PLATFORM}` before running Tauri build.
</Accordion>
</AccordionGroup>
@@ -23,20 +23,20 @@ Before you start contributing, make sure you have:
## Ways to Contribute
<CardGroup cols={2}>
<Card title="Report Bugs" icon="bug">
<Cards>
<Card title="Report Bugs">
Found a bug? Open an issue with reproduction steps
</Card>
<Card title="Request Features" icon="lightbulb">
<Card title="Request Features">
Have an idea? Start a discussion or open an issue
</Card>
<Card title="Improve Docs" icon="book">
<Card title="Improve Docs">
Fix typos, add examples, or clarify instructions
</Card>
<Card title="Write Code" icon="code">
<Card title="Write Code">
Fix bugs, add features, or optimize performance
</Card>
</CardGroup>
</Cards>
## Development Workflow
@@ -164,27 +164,28 @@ When creating a pull request:
## Project Structure
Understanding the codebase:
```
voicebox/
├── app/ # Shared React frontend
│ ├── src/
│ │ ├── components/ # UI components
│ │ ├── lib/ # Utilities and API client
│ │ ├── hooks/ # React hooks
│ │ └── stores/ # Zustand state stores
├── backend/ # Python FastAPI server
│ ├── main.py # API routes
│ ├── tts.py # Voice synthesis logic
│ ├── database.py # SQLite operations
│ └── models.py # Pydantic models
├── tauri/ # Desktop app wrapper
│ └── src-tauri/ # Rust backend
├── web/ # Web deployment
├── landing/ # Marketing website
└── scripts/ # Build & release scripts
```
<Files>
<Folder name="voicebox" defaultOpen>
<Folder name="app/src">
<File name="components/" />
<File name="lib/" />
<File name="hooks/" />
<File name="stores/" />
</Folder>
<Folder name="backend">
<File name="main.py" />
<File name="tts.py" />
<File name="database.py" />
<File name="models.py" />
</Folder>
<Folder name="tauri">
<File name="src-tauri/" />
</Folder>
<File name="web/" />
<File name="landing/" />
<File name="scripts/" />
</Folder>
</Files>
## Areas for Contribution
@@ -259,7 +260,11 @@ When adding new API endpoints:
</Step>
<Step title="Update Docs">
Add documentation in `/docs/api/`
The API documentation is automatically generated from the OpenAPI schema. Ensure your endpoint has proper docstrings and type hints, then regenerate the docs:
```bash
bun run generate:api
```
</Step>
</Steps>
@@ -0,0 +1,257 @@
---
title: "Effects Pipeline"
description: "Audio post-processing effects and generation versioning"
---
The effects pipeline provides professional-grade DSP audio processing using Spotify's Pedalboard library. Each generation can have multiple versions with different effect chains applied.
## Overview
**Key concepts:**
- **Effects Chain** — JSON-serializable list of effect configurations applied sequentially
- **Generation Version** — A processed variant of a generation with its own audio file and effects chain
- **Effect Preset** — Saved effects chain configuration (built-in or user-created)
- **Clean Version** — The original unprocessed generation audio
**Flow:**
1. TTS Generation creates clean audio
2. Effects Chain processes the audio
3. Processed Version is saved as a new generation version
Each generation maintains a clean version (original) plus any number of processed versions with different effect chains applied.
## Effect Types
The following effect types are available, each with configurable parameters:
### Chorus / Flanger
Modulated delay effect. Short centre_delay_ms gives flanger; longer gives chorus.
**Parameters:**
- rate_hz: LFO speed in Hz (range: 0.01 to 20, default: 1.0)
- depth: Modulation depth (range: 0.0 to 1.0, default: 0.5)
- feedback: Feedback amount (range: 0.0 to 0.95, default: 0.0)
- centre_delay_ms: Centre delay in milliseconds (range: 0.5 to 50, default: 7.0)
- mix: Wet/dry mix (range: 0.0 to 1.0, default: 0.5)
### Reverb
Room reverb effect.
**Parameters:**
- room_size: Room size (range: 0.0 to 1.0, default: 0.5)
- damping: High frequency damping (range: 0.0 to 1.0, default: 0.5)
- wet_level: Wet level (range: 0.0 to 1.0, default: 0.33)
- dry_level: Dry level (range: 0.0 to 1.0, default: 0.4)
- width: Stereo width (range: 0.0 to 1.0, default: 1.0)
### Delay
Echo / delay line.
**Parameters:**
- delay_seconds: Delay time in seconds (range: 0.01 to 2.0, default: 0.3)
- feedback: Feedback amount (range: 0.0 to 0.95, default: 0.3)
- mix: Wet/dry mix (range: 0.0 to 1.0, default: 0.3)
### Compressor
Dynamic range compression for consistent loudness.
**Parameters:**
- threshold_db: Threshold in dB (range: -60 to 0, default: -20.0)
- ratio: Compression ratio (range: 1.0 to 20.0, default: 4.0)
- attack_ms: Attack time in ms (range: 0.1 to 100, default: 10.0)
- release_ms: Release time in ms (range: 10 to 1000, default: 100.0)
### Gain
Volume adjustment in decibels.
**Parameters:**
- gain_db: Gain in dB (range: -40 to 40, default: 0.0)
### High-Pass Filter
Removes frequencies below the cutoff.
**Parameters:**
- cutoff_frequency_hz: Cutoff frequency in Hz (range: 20 to 8000, default: 80.0)
### Low-Pass Filter
Removes frequencies above the cutoff.
**Parameters:**
- cutoff_frequency_hz: Cutoff frequency in Hz (range: 200 to 20000, default: 8000.0)
### Pitch Shift
Shift pitch up or down by semitones.
**Parameters:**
- semitones: Semitones to shift (range: -12 to 12, default: 0.0)
## Generation Versions
Each generation starts with a clean version (no effects). Users can create processed versions by applying effect chains.
**Version properties:**
- id — Unique version identifier
- label — User-defined name (e.g., "robotic", "with reverb")
- audio_path — Path to the processed audio file
- effects_chain — JSON array of effect configurations
- source_version_id — Which version this was derived from
- is_default — Whether this is the default audio for the generation
**File storage:**
<Files>
<Folder name="data/generations" defaultOpen>
<File name="{generation_id}.wav" />
<File name="{generation_id}_{version_id}.wav" />
</Folder>
</Files>
**Default version behavior:**
- One version per generation is marked as default
- The generation's audio_path always points to the default version's audio
- Deleting the default version automatically promotes another version
## Effect Presets
Presets are saved effects chains that can be reused across generations.
**Built-in presets:**
- **Robotic**: Metallic robotic voice using chorus (flanger-style)
- **Radio**: Thin AM-radio voice with band-pass filtering and light compression
- **Echo Chamber**: Spacious reverb with trailing echo
- **Deep Voice**: Lower pitch with added warmth using pitch shift and compression
**User presets:**
- Created via the effects UI
- Stored in the database (SQLite)
- Cannot modify/delete built-in presets
- Used to quickly apply favorite effect combinations
## API Endpoints
### Effects Management
| Endpoint | Method | Description |
|----------|--------|-------------|
| /effects/available | GET | List all effect types with parameter definitions |
| /effects/presets | GET | List all presets (built-in + user) |
| /effects/presets | POST | Create a new user preset |
| /effects/presets/:id | GET | Get a specific preset |
| /effects/presets/:id | PUT | Update a user preset |
| /effects/presets/:id | DELETE | Delete a user preset |
| /effects/preview/:generation_id | POST | Preview effects on a generation (returns audio stream) |
### Generation Versions
| Endpoint | Method | Description |
|----------|--------|-------------|
| /generations/:id/versions | GET | List all versions for a generation |
| /generations/:id/versions/apply-effects | POST | Apply effects chain, create new version |
| /generations/:id/versions/:version_id/set-default | PUT | Set a version as default |
| /generations/:id/versions/:version_id | DELETE | Delete a version |
### Request Body: Apply Effects
Request body for applying effects:
- effects_chain: Array of effect objects
- label: Version label (e.g., "with reverb")
- set_as_default: Whether to set as default
- source_version_id: Source version ID (optional)
## Implementation
### Backend Architecture
**Files:**
| File | Purpose |
|------|---------|
| backend/utils/effects.py | Effect registry, validation, and audio processing |
| backend/services/versions.py | Generation version CRUD operations |
| backend/services/effects.py | Effect preset CRUD operations |
| backend/routes/effects.py | API endpoints for effects and versions |
**Effect Registry:**
The EFFECT_REGISTRY dict in utils/effects.py defines all available effects with their parameters, defaults, and ranges.
**Validation:**
Effects chains are validated before application:
- Each effect type must exist in the registry
- Parameters must be numbers within min/max bounds
- Unknown parameters are rejected
**Audio Processing:**
Uses Spotify's Pedalboard library:
```python
from pedalboard import Pedalboard
# Build pedalboard from chain
board = build_pedalboard(effects_chain)
# Apply to audio (async via thread)
processed = await asyncio.to_thread(lambda: board(audio, sample_rate))
```
### Frontend Integration
**Key components:**
| Component | Location |
|-----------|----------|
| Effects chain editor | app/src/components/Effects/ |
| Version selector | Generation detail view |
| Preset manager | Effects panel |
| Live preview | Preview button (streams processed audio) |
**State management:**
- Effects chains are stored as JSON arrays
- Live preview fetches processed audio without saving
- Applied effects create new versions via POST endpoint
## Adding New Effects
To add a new effect type:
1. **Add to registry** (backend/utils/effects.py):
- Add entry to EFFECT_REGISTRY with cls, label, description, and params
- Import the effect class from Pedalboard
2. **Update frontend types** if needed
The new effect automatically appears in /effects/available and the chain editor UI.
## Best Practices
**Effect ordering matters.** Process effects in this order for best results:
1. Pitch shift (if needed)
2. High/low-pass filters
3. Chorus/flanger (time-based)
4. Reverb/delay (spatial)
5. Compressor
6. Gain (final level adjustment)
**CPU usage:**
- Effects are applied in real-time during generation
- Pitch shift and reverb are the most CPU-intensive
- Consider previewing complex chains before applying
**Storage:**
- Each version creates a new audio file
- Clean version always exists (can be reverted to)
- Processed versions can be deleted to save space
@@ -30,11 +30,13 @@ class Generation(Base):
Generated audio is stored in:
```
data/
└── generations/
└── {generation_id}.wav
```
<Files>
<Folder name="data" defaultOpen>
<Folder name="generations">
<File name="{generation_id}.wav" />
</Folder>
</Folder>
</Files>
## Core Functions
@@ -171,11 +173,12 @@ async def delete_generations_by_profile(profile_id: str, db: Session) -> int:
Generations can be exported as ZIP archives:
```
generation_export.zip
├── generation.json # Metadata
└── audio.wav # Audio file
```
<Files>
<Folder name="generation_export.zip" defaultOpen>
<File name="generation.json" />
<File name="audio.wav" />
</Folder>
</Files>
### Importing a Generation
+20
View File
@@ -0,0 +1,20 @@
{
"title": "Developer",
"defaultOpen": true,
"pages": [
"setup",
"architecture",
"contributing",
"building",
"autoupdater",
"voice-profiles",
"tts-generation",
"tts-engines",
"effects-pipeline",
"history",
"stories",
"transcription",
"audio-channels",
"model-management"
]
}
@@ -36,13 +36,13 @@ Models are downloaded from HuggingFace Hub on first use and cached locally.
Models are cached in the HuggingFace cache directory:
```
~/.cache/huggingface/hub/
├── models--Qwen--Qwen3-TTS-12Hz-1.7B-Base/
├── models--Qwen--Qwen3-TTS-12Hz-0.6B-Base/
├── models--openai--whisper-base/
└── ...
```
<Files>
<Folder name="~/.cache/huggingface/hub" defaultOpen>
<File name="models--Qwen--Qwen3-TTS-12Hz-1.7B-Base/" />
<File name="models--Qwen--Qwen3-TTS-12Hz-0.6B-Base/" />
<File name="models--openai--whisper-base/" />
</Folder>
</Files>
## Progress Tracking
+248
View File
@@ -0,0 +1,248 @@
---
title: "Development Setup"
description: "Set up your local development environment for Voicebox"
---
## Quick Setup (Recommended)
Get started in two commands:
```bash
# Clone and enter the repository
git clone https://github.com/jamiepine/voicebox.git
cd voicebox
# Setup everything (Python venv, JS deps, dev sidecar)
just setup
# Start development (backend + desktop app)
just dev
```
The `just dev` command automatically starts the Python backend (if not already running) and launches the Tauri desktop app.
## Prerequisites
Ensure you have these installed:
<Cards>
<Card title="Bun" icon={<svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24" fill="none" stroke="currentColor" strokeWidth="2" strokeLinecap="round" strokeLinejoin="round"><path d="m7.5 4.27 9 5.15"/><path d="M21 8a2 2 0 0 0-1-1.73l-7-4a2 2 0 0 0-2 0l-7 4A2 2 0 0 0 3 8v8a2 2 0 0 0 1 1.73l7 4a2 2 0 0 0 2 0l7-4A2 2 0 0 0 21 16Z"/><path d="m3.3 7 8.7 5 8.7-5"/><path d="M12 22V12"/></svg>}>
[Download Bun](https://bun.sh)
```bash
curl -fsSL https://bun.sh/install | bash
```
</Card>
<Card title="Python 3.11+" icon={<svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24" fill="none" stroke="currentColor" strokeWidth="2" strokeLinecap="round" strokeLinejoin="round"><path d="M12 2L2 7l10 5 10-5-10-5z"/><path d="M2 17l10 5 10-5"/><path d="M2 12l10 5 10-5"/></svg>}>
[Download Python](https://python.org)
```bash
python --version
```
</Card>
<Card title="Rust" icon={<svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24" fill="none" stroke="currentColor" strokeWidth="2" strokeLinecap="round" strokeLinejoin="round"><path d="m6 9 6 6 6-6"/></svg>}>
[Install Rust](https://rustup.rs)
```bash
rustc --version
```
</Card>
<Card title="Just" icon={<svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24" fill="none" stroke="currentColor" strokeWidth="2" strokeLinecap="round" strokeLinejoin="round"><path d="M4 7V4h3"/><path d="M7 4h14v6h-2V6H7V4Z"/><path d="M4 10v10h16V10H4Z"/></svg>}>
[Install Just](https://github.com/casey/just)
```bash
brew install just # macOS
cargo install just # Linux/Windows
```
</Card>
</Cards>
<Callout type="info">
Just works on macOS, Linux, and Windows.
</Callout>
## Just Commands
Run `just --list` to see all available commands:
| Command | Description |
|---------|-------------|
| `just setup` | Full setup (Python venv + JS deps) |
| `just dev` | Start backend + desktop app |
| `just dev-web` | Start backend + web app (no Tauri) |
| `just dev-backend` | Start backend only |
| `just dev-frontend` | Start desktop app only (backend must be running) |
| `just build` | Build desktop app for production |
| `just build-web` | Build web app for production |
| `just check` | Run all checks (JS + Python lint + format) |
| `just fix` | Fix lint + format issues |
| `just test` | Run Python tests |
| `just db-init` | Initialize SQLite database |
| `just db-reset` | Reset database (delete + reinit) |
| `just clean` | Clean build artifacts |
| `just clean-all` | Nuclear clean (includes node_modules) |
## Project Structure
<Files>
<Folder name="voicebox" defaultOpen>
<Folder name="app">
<Folder name="src">
<File name="components/" />
<File name="lib/" />
<File name="hooks/" />
</Folder>
</Folder>
<Folder name="backend">
<File name="app.py" />
<File name="main.py" />
<File name="config.py" />
<File name="models.py" />
<File name="server.py" />
<Folder name="routes">
<File name="..." />
</Folder>
<Folder name="services">
<File name="..." />
</Folder>
<Folder name="backends">
<File name="..." />
</Folder>
<Folder name="database">
<File name="..." />
</Folder>
<Folder name="utils">
<File name="..." />
</Folder>
</Folder>
<Folder name="tauri">
<Folder name="src-tauri" />
</Folder>
<Folder name="web" />
<Folder name="scripts" />
</Folder>
</Files>
### Request Flow
HTTP request → **routes/** (validate input) → **services/** (business logic) → **backends/** (TTS/STT inference) → **utils/** (audio processing)
### Key Modules
- **app.py** — FastAPI app factory, CORS, lifecycle events
- **main.py** — Entry point (imports app, runs uvicorn)
- **server.py** — Tauri sidecar launcher, parent-pid watchdog
- **services/generation.py** — Single function handling all generation modes
- **backends/** — TTS/STT engine implementations (MLX, PyTorch, etc.)
## Model Downloads
Models are automatically downloaded from HuggingFace Hub on first use:
- **Whisper** (transcription): Auto-downloads on first transcription
- **Qwen3-TTS** (voice cloning): Auto-downloads on first generation (~2-4GB)
<Callout type="warn">
First-time usage will be slower due to model downloads, but subsequent runs will use cached models.
</Callout>
## Generate OpenAPI Client
After starting the backend server, generate the TypeScript API client:
```bash
just generate-api
```
This downloads the OpenAPI schema and generates the TypeScript client in `app/src/lib/api/`
## Manual Setup (Advanced)
If you prefer not to use Just, follow these manual steps:
### 1. Install JavaScript Dependencies
```bash
bun install
```
This installs dependencies for:
- `app/` - Shared React frontend
- `tauri/` - Tauri desktop wrapper
- `web/` - Web deployment wrapper
### 2. Set Up Python Backend
```bash
cd backend
# Create virtual environment
python -m venv venv
# Activate virtual environment
source venv/bin/activate # macOS/Linux
# or
venv\Scripts\activate # Windows
# Install Python dependencies
pip install -r requirements.txt
# Apple Silicon: install MLX dependencies
pip install -r requirements-mlx.txt
# Install Qwen3-TTS
pip install git+https://github.com/QwenLM/Qwen3-TTS.git
```
### 3. Start Development
Start the backend:
```bash
cd backend
source venv/bin/activate
uvicorn main:app --reload --port 17493
```
In a new terminal, start the desktop app:
```bash
cd tauri
bun run tauri dev
```
## Next Steps
<Cards>
<Card title="Architecture" href="/development/architecture">
Understand the system architecture
</Card>
<Card title="Contributing" href="/development/contributing">
Read the contribution guidelines
</Card>
<Card title="Building" href="/development/building">
Learn how to build production releases
</Card>
<Card title="API Reference" href="/api-reference">
Explore the REST API
</Card>
</Cards>
## Troubleshooting
<AccordionGroup>
<Accordion title="Backend won't start">
- Check Python version (must be 3.11+)
- Ensure virtual environment is activated: `source backend/venv/bin/activate`
- Verify all dependencies are installed: `pip install -r requirements.txt`
- Check if port 17493 is available
</Accordion>
<Accordion title="Tauri build fails">
- Ensure Rust is installed: `rustc --version`
- Clean the build: `cd tauri/src-tauri && cargo clean`
- Try rebuilding: `just dev`
</Accordion>
<Accordion title="OpenAPI client generation fails">
- Ensure backend is running: `curl http://localhost:17493/openapi.json`
- Check network connectivity
- Verify the backend is accessible at localhost:17493
</Accordion>
</AccordionGroup>
See the full [Troubleshooting Guide](/overview/troubleshooting) for more issues and solutions.
+257
View File
@@ -0,0 +1,257 @@
---
title: "TTS Engines"
description: "How to add new text-to-speech engines to Voicebox"
---
Adding an engine touches ~10 files across 4 layers. The backend protocol work is straightforward — the real time sink is dependency hell, upstream library bugs, and PyInstaller bundling.
## Architecture Overview
The backend is split into layers:
| Layer | Purpose | Files Touched |
|-------|---------|---------------|
| `routes/` | Thin HTTP handlers | None (auto-dispatch) |
| `services/` | Business logic | None (auto-dispatch) |
| `backends/` | Engine implementations | `your_engine_backend.py` |
| `utils/` | Shared utilities | As needed |
New engines only need to touch `backends/` and `models.py` on the backend side — the route and service layers use a model config registry that handles dispatch automatically.
## Phase 1: Backend Implementation
### 1.1 Create the Backend File
Create `backend/backends/<engine>_backend.py` (~200-300 lines) implementing the `TTSBackend` protocol:
```python
class YourBackend:
"""Must satisfy the TTSBackend protocol."""
async def load_model(self, model_size: str = "default") -> None: ...
async def create_voice_prompt(self, audio_path: str, reference_text: str, use_cache: bool = True) -> tuple[dict, bool]: ...
async def combine_voice_prompts(self, audio_paths: list[str], ref_texts: list[str]) -> tuple[np.ndarray, str]: ...
async def generate(self, text: str, voice_prompt: dict, language: str = "en", seed: int | None = None, instruct: str | None = None) -> tuple[np.ndarray, int]: ...
def unload_model(self) -> None: ...
def is_loaded(self) -> bool: ...
def _get_model_path(self, model_size: str) -> str: ...
```
**Key decisions per engine:**
| Decision | Options | Examples |
|----------|---------|---------|
| **Voice prompt storage** | Pre-computed tensors vs deferred file paths | Qwen stores tensor dicts; Chatterbox stores paths |
| **Caching** | Use voice prompt cache or skip it | LuxTTS caches with prefix; Chatterbox skips caching |
| **Device selection** | CUDA / MPS / CPU | Chatterbox forces CPU on macOS (MPS bugs) |
| **Model download** | Library handles it vs manual `snapshot_download` | Turbo uses manual download to bypass `token=True` bug |
| **Sample rate** | Engine-specific | LuxTTS outputs 48kHz, everything else is 24kHz |
### 1.2 Voice Prompt Patterns
**Pattern A: Pre-computed tensors** (Qwen, LuxTTS)
```python
encoded = model.encode_prompt(audio_path)
return encoded, False # (prompt_dict, was_cached)
```
**Pattern B: Deferred file paths** (Chatterbox, MLX)
```python
return {"ref_audio": audio_path, "ref_text": reference_text}, False
```
**Pattern C: Hybrid** (possible for new engines)
```python
embedding = model.extract_speaker(audio_path)
return {"embedding": embedding, "ref_audio": audio_path}, False
```
If caching, prefix your cache keys:
```python
cache_key = "yourengine_" + get_cache_key(audio_path, reference_text)
```
### 1.3 Register the Engine
In `backend/backends/__init__.py`:
**Add a `ModelConfig` entry:**
```python
ModelConfig(
model_name="your-engine",
display_name="Your Engine",
engine="your_engine",
hf_repo_id="org/model-repo",
size_mb=3200,
needs_trim=False, # set True if output needs trim_tts_output()
languages=["en", "fr", "de"],
),
```
**Add to `TTS_ENGINES` dict:**
```python
TTS_ENGINES = {
...
"your_engine": "Your Engine",
}
```
**Add factory branch:**
```python
elif engine == "your_engine":
from .your_backend import YourBackend
backend = YourBackend()
```
### 1.4 Update Request Models
In `backend/models.py`:
- Add engine name to `GenerationRequest.engine` regex pattern
- Add any new language codes to the language regex
## Phase 2: Route and Service Integration
With the model config registry, route and service layers have **zero per-engine dispatch points**. All endpoints use registry helpers like `get_model_config()`, `load_engine_model()`, `engine_needs_trim()`, `check_model_loaded()`, etc.
**You don't need to touch any route or service files** unless your engine needs custom behavior in the generate pipeline.
### Post-Processing
If your model produces trailing silence, set `needs_trim=True` on your `ModelConfig`. The generation service applies `trim_tts_output()` automatically.
## Phase 3: Frontend Integration
### 3.1 TypeScript Types
In `app/src/lib/api/types.ts`:
- Add to the `engine` union type on `GenerationRequest`
### 3.2 Language Maps
In `app/src/lib/constants/languages.ts`:
- Add entry to `ENGINE_LANGUAGES` record
- Add any new language codes to `ALL_LANGUAGES` if needed
### 3.3 Engine/Model Selector
In `app/src/components/Generation/EngineModelSelector.tsx`:
- Add entry to `ENGINE_OPTIONS` and `ENGINE_DESCRIPTIONS`
- Add to `ENGLISH_ONLY_ENGINES` if applicable
### 3.4 Form Hook
In `app/src/lib/hooks/useGenerationForm.ts`:
- Add to Zod schema enum for `engine`
- Add engine-to-model-name mapping
- Update payload construction for engine-specific fields
### 3.5 Model Management
In `app/src/components/ServerSettings/ModelManagement.tsx`:
- Add description to `MODEL_DESCRIPTIONS` record
## Phase 4: Dependencies
### 4.1 Python Dependencies
Add to `backend/requirements.txt`. Watch for:
**Pinned dependency conflicts** — If the model package pins old versions, install with `--no-deps`:
```bash
pip install --no-deps chatterbox-tts
```
Then list sub-dependencies manually in `requirements.txt`.
**Non-PyPI packages:**
```
linacodec @ git+https://github.com/user/repo.git
```
**Custom package indexes:**
```
--find-links https://k2-fsa.github.io/icefall/piper_phonemize.html
```
### 4.2 Identifying Hidden Sub-Dependencies
1. Install the package normally in a throwaway venv
2. Run `pip show <package>` to get its `Requires:` list
3. Cross-reference against existing requirements.txt
4. Test that the engine loads and generates
## Phase 5: PyInstaller Bundling
This is where most of the pain lives. Common issues:
| Issue | Symptom | Fix |
|-------|---------|-----|
| `inspect.getsource()` at import | "could not get source code" | `--collect-all <package>` |
| Data files (yaml, .pth.tar) | FileNotFoundError at runtime | `--collect-all <package>` |
| Native data paths (espeak-ng) | Library looks at `/usr/share/...` | Set env var in frozen builds |
| `importlib.metadata` lookups | "No package metadata found" | `--copy-metadata <package>` |
| Dynamic imports | ModuleNotFoundError | `--hidden-import <module>` |
### Testing Frozen Builds
You can't skip this. Models that work in `python -m uvicorn` will break in the PyInstaller binary.
1. Build: `just build`
2. Run and try download + load + generate
3. Check stderr for the actual error
4. Fix, rebuild, repeat
## Phase 6: Common Upstream Workarounds
### torch.load device mismatch
```python
_original_torch_load = torch.load
def _patched_torch_load(*args, **kwargs):
kwargs.setdefault("map_location", "cpu")
return _original_torch_load(*args, **kwargs)
torch.load = _patched_torch_load
```
### Float64/Float32 dtype mismatch
```python
original_fn = SomeClass.some_method
def patched_fn(self, *args, **kwargs):
result = original_fn(self, *args, **kwargs)
return result.float()
SomeClass.some_method = patched_fn
```
### HuggingFace token bug
```python
from huggingface_hub import snapshot_download
local_path = snapshot_download(repo_id=REPO, token=None)
model = ModelClass.from_local(local_path, device=device)
```
### MPS tensor issues
Skip MPS entirely if operators aren't supported:
```python
def _get_device(self):
if torch.cuda.is_available():
return "cuda"
return "cpu" # Skip MPS
```
## Upcoming Engines
Based on the current model landscape, these are candidates for future integration:
| Model | Languages | Size | Key Features | Status |
|-------|-----------|------|--------------|--------|
| **CosyVoice2-0.5B** | Multilingual | ~500MB | Instruct support (`inference_instruct2()`) | Ready |
| **Fish Speech** | 50+ | Medium | Word-level control via inline text | Ready |
| **Kokoro-82M** | English | 82M | CPU realtime, Apache 2.0 | Ready |
| **XTTS-v2** | 17+ | Medium | Zero-shot cloning | Ready |
| **HumeAI TADA** | EN (1B), Multi (3B) | Medium | 700s+ coherent audio, synced transcripts | Needs vetting |
| **MOSS-TTS** | Multilingual | Medium | Text-to-voice design, multi-speaker dialogue | Needs vetting |
| **Pocket TTS** | English | ~100M | CPU-first, >1× realtime | Needs vetting |
The multi-engine architecture is now in place, making new model integration straightforward (~1 day for a well-documented model with a PyPI package).
@@ -49,14 +49,16 @@ class ProfileSample(Base):
Profiles are stored in the data directory:
```
data/
└── profiles/
└── {profile_id}/
├── {sample_id_1}.wav
├── {sample_id_2}.wav
└── ...
```
<Files>
<Folder name="data" defaultOpen>
<Folder name="profiles">
<Folder name="{profile_id}">
<File name="{sample_id_1}.wav" />
<File name="{sample_id_2}.wav" />
</Folder>
</Folder>
</Folder>
</Files>
## Core Functions
@@ -158,14 +160,15 @@ Reference audio is validated before being accepted:
Profiles can be exported as ZIP archives for sharing:
```
profile_export.zip
├── profile.json # Metadata
├── samples/
│ ├── sample_1.wav
│ └── sample_1.json # Reference text
└── ...
```
<Files>
<Folder name="profile_export.zip" defaultOpen>
<File name="profile.json" />
<Folder name="samples">
<File name="sample_1.wav" />
<File name="sample_1.json" />
</Folder>
</Folder>
</Files>
## API Endpoints
+36
View File
@@ -0,0 +1,36 @@
---
title: "Voicebox Documentation"
description: "Voicebox is a local-first voice cloning studio -- a free and open-source alternative to ElevenLabs."
---
Voicebox is a **local-first voice cloning studio** -- a free and open-source alternative to ElevenLabs. Clone voices from a few seconds of audio, generate speech in 23 languages across 4 TTS engines, apply post-processing effects, and compose multi-voice projects with a timeline editor.
![Voicebox App Screenshot](/images/app-screenshot-1.webp)
- **Complete privacy** -- models and voice data stay on your machine
- **4 TTS engines** -- Qwen3-TTS, LuxTTS, Chatterbox Multilingual, and Chatterbox Turbo
- **23 languages** -- from English to Arabic, Japanese, Hindi, Swahili, and more
- **Post-processing effects** -- pitch shift, reverb, delay, chorus, compression, and filters
- **Expressive speech** -- paralinguistic tags like `[laugh]`, `[sigh]`, `[gasp]` via Chatterbox Turbo
- **Unlimited length** -- auto-chunking with crossfade for scripts, articles, and chapters
- **Stories editor** -- multi-track timeline for conversations, podcasts, and narratives
- **API-first** -- REST API for integrating voice synthesis into your own projects
- **Native performance** -- built with Tauri (Rust), not Electron
- **Runs everywhere** -- macOS (MLX/Metal), Windows (CUDA), Linux, AMD ROCm, Intel Arc, Docker
## Download
| Platform | Download |
|----------|----------|
| macOS (Apple Silicon) | [Download DMG](https://voicebox.sh/download/mac-arm) |
| macOS (Intel) | [Download DMG](https://voicebox.sh/download/mac-intel) |
| Windows | [Download MSI](https://voicebox.sh/download/windows) |
| Docker | `docker compose up` |
[View all releases](https://github.com/jamiepine/voicebox/releases/latest)
## Get Started
- [Installation](/docs/overview/installation) -- download and install Voicebox
- [Quick Start](/docs/overview/quick-start) -- get up and running in 5 minutes
- [API Reference](/docs/api-reference) -- integrate voice synthesis into your apps
+4
View File
@@ -0,0 +1,4 @@
{
"title": "Voicebox Documentation",
"pages": ["overview", "api-reference", "developer"]
}
@@ -28,32 +28,32 @@ Voice profiles are the foundation of voice cloning in Voicebox. This guide cover
### Ideal Sample Characteristics
<CardGroup cols={2}>
<Card title="Duration" icon="clock">
<Cards>
<Card title="Duration">
**10-30 seconds**
Too short: Poor quality
Too long: Unnecessary
</Card>
<Card title="Clarity" icon="volume">
<Card title="Clarity">
**Clear speech**
No background noise
No music or overlapping voices
</Card>
<Card title="Quality" icon="sparkles">
<Card title="Quality">
**High fidelity**
44.1kHz or 48kHz sample rate
Minimal compression
</Card>
<Card title="Content" icon="microphone">
<Card title="Content">
**Natural speech**
Conversational tone
Complete sentences
</Card>
</CardGroup>
</Cards>
### File Formats
@@ -63,9 +63,9 @@ Supported formats:
- **M4A** - Acceptable
- **FLAC** - Lossless alternative
<Tip>
<Callout type="info">
Use WAV for best results. Avoid heavily compressed formats.
</Tip>
</Callout>
## Recording Tips
@@ -108,20 +108,20 @@ Adding multiple samples can significantly improve quality:
### Why Multiple Samples?
<CardGroup cols={2}>
<Card title="Robustness" icon="shield">
<Cards>
<Card title="Robustness">
Model learns a more complete representation
</Card>
<Card title="Versatility" icon="palette">
<Card title="Versatility">
Handles different speaking styles better
</Card>
<Card title="Quality" icon="star">
<Card title="Quality">
Reduces artifacts and improves naturalness
</Card>
<Card title="Consistency" icon="check">
<Card title="Consistency">
More reliable across different texts
</Card>
</CardGroup>
</Cards>
### Sample Variety
@@ -144,9 +144,9 @@ Consider adding samples with:
- Phone call quality (if needed)
- Room acoustics
<Warning>
<Callout type="warn">
All samples should be from the **same speaker**. Mixing voices will produce poor results.
</Warning>
</Callout>
## Processing Existing Audio
@@ -286,11 +286,11 @@ The emotional tone of samples affects generation:
## Next Steps
<CardGroup cols={2}>
<Card title="Generate Speech" icon="waveform" href="/guides/generating-speech">
<Cards>
<Card title="Generate Speech" href="/overview/generating-speech">
Use your profile to generate speech
</Card>
<Card title="Build Stories" icon="film" href="/guides/building-stories">
<Card title="Build Stories" href="/overview/building-stories">
Create multi-voice narratives
</Card>
</CardGroup>
</Cards>
@@ -43,9 +43,9 @@ Use formatting to suggest emphasis:
- Bold for strong emphasis: "This is **very** important"
```
<Note>
<Callout type="info">
The model interprets these hints but results may vary.
</Note>
</Callout>
## Advanced Features
@@ -9,20 +9,20 @@ Voicebox keeps a complete history of all generated audio, making it easy to find
## Features
<CardGroup cols={2}>
<Card title="Full History" icon="clock">
<Cards>
<Card title="Full History" icon={<svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24" fill="none" stroke="currentColor" strokeWidth="2" strokeLinecap="round" strokeLinejoin="round"><circle cx="12" cy="12" r="10"/><polyline points="12 6 12 12 16 14"/></svg>}>
Every generation is automatically saved
</Card>
<Card title="Search & Filter" icon="search">
<Card title="Search & Filter" icon={<svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24" fill="none" stroke="currentColor" strokeWidth="2" strokeLinecap="round" strokeLinejoin="round"><circle cx="11" cy="11" r="8"/><path d="m21 21-4.3-4.3"/></svg>}>
Find by text, voice, or date
</Card>
<Card title="Re-generate" icon="rotate">
<Card title="Re-generate" icon={<svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24" fill="none" stroke="currentColor" strokeWidth="2" strokeLinecap="round" strokeLinejoin="round"><path d="M3 12a9 9 0 0 1 9-9 9.75 9.75 0 0 1 6.74 2.74L21 8"/><path d="M21 3v5h-5"/><path d="M21 12a9 9 0 0 1-9 9 9.75 9.75 0 0 1-6.74-2.74L3 16"/><path d="M8 16H3v5"/></svg>}>
Regenerate any past generation with one click
</Card>
<Card title="Export" icon="download">
<Card title="Export" icon={<svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24" fill="none" stroke="currentColor" strokeWidth="2" strokeLinecap="round" strokeLinejoin="round"><path d="M21 15v4a2 2 0 0 1-2 2H5a2 2 0 0 1-2-2v-4"/><polyline points="7 10 12 15 17 10"/><line x1="12" y1="15" x2="12" y2="3"/></svg>}>
Download individual or batch exports
</Card>
</CardGroup>
</Cards>
## Viewing History
@@ -54,20 +54,20 @@ Drag generations to the Stories Editor timeline.
## Search & Filter
<Tabs>
<Tab title="By Text">
<Tabs items={["By Text", "By Voice", "By Date"]}>
<Tab value="By Text">
Search for specific text content
```
"Hello world"
```
</Tab>
<Tab title="By Voice">
<Tab value="By Voice">
Filter by voice profile
```
Select from dropdown
```
</Tab>
<Tab title="By Date">
<Tab value="By Date">
Filter by date range
```
Last 7 days, Last 30 days, Custom range
@@ -83,6 +83,6 @@ History is stored locally:
- **Windows**: `%APPDATA%/com.voicebox.app/data/`
- **Linux**: `~/.config/com.voicebox.app/data/`
<Warning>
<Callout type="warn">
Deleting the data directory will remove all history. Export important files first.
</Warning>
</Callout>
@@ -7,19 +7,19 @@ description: "Download and install Voicebox on macOS, Windows, or Linux"
Voicebox is available for macOS and Windows, with Linux builds coming soon.
<CardGroup cols={2}>
<Card title="macOS" icon="apple">
<Cards>
<Card title="macOS" icon={<svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24" fill="none" stroke="currentColor" strokeWidth="2" strokeLinecap="round" strokeLinejoin="round"><path d="M12 2c-1.5 0-2.8.4-3.9 1.1A5.5 5.5 0 0 0 4 2.5C2.5 2.5 1 4 1 6c0 3.5 2.5 6 5 7.5C5 16 4 18 4 20c0 1.5.5 2.5 1.5 3C6.5 23.5 8 24 9.5 24c2 0 3.5-.5 5-2 1.5 1.5 3 2 5 2 1.5 0 3-.5 4-1 1-.5 1.5-1.5 1.5-3 0-2-1-4-2-6.5 2.5-1.5 5-4 5-7.5 0-2-1.5-3.5-3-3.5-.9 0-2.1.4-3.1 1.1A6.5 6.5 0 0 0 12 2Z"/></svg>}>
Download for Apple Silicon or Intel Macs
</Card>
<Card title="Windows" icon="windows">
<Card title="Windows" icon={<svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24" fill="none" stroke="currentColor" strokeWidth="2" strokeLinecap="round" strokeLinejoin="round"><rect x="2" y="3" width="20" height="14" rx="2" ry="2"/><line x1="8" y1="21" x2="16" y2="21"/><line x1="12" y1="17" x2="12" y2="21"/></svg>}>
Download MSI installer or Setup executable
</Card>
</CardGroup>
</Cards>
### macOS
<Tabs>
<Tab title="Apple Silicon">
<Tabs items={["Apple Silicon", "Intel"]}>
<Tab value="Apple Silicon">
Download: [voicebox_aarch64.app.tar.gz](https://github.com/jamiepine/voicebox/releases/latest/download/voicebox_aarch64.app.tar.gz)
```bash
@@ -30,7 +30,7 @@ Voicebox is available for macOS and Windows, with Linux builds coming soon.
mv Voicebox.app /Applications/
```
</Tab>
<Tab title="Intel">
<Tab value="Intel">
Download: [voicebox_x64.app.tar.gz](https://github.com/jamiepine/voicebox/releases/latest/download/voicebox_x64.app.tar.gz)
```bash
@@ -45,13 +45,13 @@ Voicebox is available for macOS and Windows, with Linux builds coming soon.
### Windows
<Tabs>
<Tab title="MSI Installer">
<Tabs items={["MSI Installer", "Setup Executable"]}>
<Tab value="MSI Installer">
Download: [voicebox_x64_en-US.msi](https://github.com/jamiepine/voicebox/releases/latest/download/voicebox_x64_en-US.msi)
Double-click the MSI file and follow the installation wizard.
</Tab>
<Tab title="Setup Executable">
<Tab value="Setup Executable">
Download: [voicebox_x64-setup.exe](https://github.com/jamiepine/voicebox/releases/latest/download/voicebox_x64-setup.exe)
Run the executable and follow the installation wizard.
@@ -60,9 +60,9 @@ Voicebox is available for macOS and Windows, with Linux builds coming soon.
### Linux
<Note>
<Callout type="info">
Linux builds are coming soon. Currently blocked by GitHub runner disk space limitations.
</Note>
</Callout>
## First Launch
@@ -76,9 +76,9 @@ When you launch Voicebox for the first time:
3. **Backend Server** — The bundled Python server starts automatically
<Tip>
<Callout type="info">
First generation will be slower due to model downloads. Subsequent runs use cached models.
</Tip>
</Callout>
## System Requirements
@@ -95,9 +95,9 @@ When you launch Voicebox for the first time:
- **GPU:** CUDA-capable NVIDIA GPU (for faster generation)
- **Storage:** 10GB+ free space
<Note>
<Callout type="info">
CPU inference is supported but significantly slower than GPU. A CUDA-capable GPU is highly recommended for real-time workflows.
</Note>
</Callout>
## Verification
@@ -108,12 +108,12 @@ After installation, verify everything works:
3. Navigate to **Profiles** and create a test profile
4. Generate a short audio clip to verify the TTS engine works
<Check>
<Callout type="success">
If you see a green status indicator and can generate audio, you're all set!
</Check>
</Callout>
## Next Steps
<Card title="Quick Start Guide" icon="rocket" href="/overview/quick-start">
<Card title="Quick Start Guide" href="/overview/quick-start">
Create your first voice profile and generate speech
</Card>
@@ -0,0 +1,64 @@
---
title: "Introduction"
description: "Voicebox is a local-first voice cloning studio -- a free and open-source alternative to ElevenLabs."
---
## What is Voicebox?
Voicebox is a **local-first voice cloning studio** -- a free and open-source alternative to ElevenLabs. Clone voices from a few seconds of audio, generate speech in 23 languages across 4 TTS engines, apply post-processing effects, and compose multi-voice projects with a timeline editor.
- **Complete privacy** -- models and voice data stay on your machine
- **4 TTS engines** -- Qwen3-TTS, LuxTTS, Chatterbox Multilingual, and Chatterbox Turbo
- **23 languages** -- from English to Arabic, Japanese, Hindi, Swahili, and more
- **Post-processing effects** -- pitch shift, reverb, delay, chorus, compression, and filters
- **Expressive speech** -- paralinguistic tags like `[laugh]`, `[sigh]`, `[gasp]` via Chatterbox Turbo
- **Unlimited length** -- auto-chunking with crossfade for scripts, articles, and chapters
- **Stories editor** -- multi-track timeline for conversations, podcasts, and narratives
- **API-first** -- REST API for integrating voice synthesis into your own projects
- **Native performance** -- built with Tauri (Rust), not Electron
- **Runs everywhere** -- macOS (MLX/Metal), Windows (CUDA), Linux, AMD ROCm, Intel Arc, Docker
## TTS Engines
Four engines with different strengths, switchable per-generation:
| Engine | Languages | Strengths |
|--------|-----------|-----------|
| **Qwen3-TTS** (0.6B / 1.7B) | 10 | High-quality multilingual cloning, delivery instructions |
| **LuxTTS** | English | Lightweight (~1GB VRAM), 48kHz output, 150x realtime on CPU |
| **Chatterbox Multilingual** | 23 | Broadest language coverage |
| **Chatterbox Turbo** | English | Fast 350M model with paralinguistic emotion/sound tags |
## GPU Support
| Platform | Backend | Notes |
|----------|---------|-------|
| macOS (Apple Silicon) | MLX (Metal) | 4-5x faster via Neural Engine |
| Windows / Linux (NVIDIA) | PyTorch (CUDA) | Auto-downloads CUDA binary from within the app |
| Linux (AMD) | PyTorch (ROCm) | Auto-configures HSA_OVERRIDE_GFX_VERSION |
| Windows (any GPU) | DirectML | Universal Windows GPU support |
| Intel Arc | IPEX/XPU | Intel discrete GPU acceleration |
| Any | CPU | Works everywhere, just slower |
## Use Cases
- **Game development** -- generate dynamic dialogue for characters
- **Content creation** -- produce podcasts and video voiceovers
- **Accessibility** -- build text-to-speech tools for users who need them
- **Voice assistants** -- create custom voice interfaces
- **Production pipelines** -- automate voiceover workflows via the REST API
## Tech Stack
| Layer | Technology |
|-------|------------|
| Desktop App | Tauri (Rust) |
| Frontend | React, TypeScript, Tailwind CSS |
| State | Zustand, React Query |
| Backend | FastAPI (Python) |
| TTS Engines | Qwen3-TTS, LuxTTS, Chatterbox, Chatterbox Turbo |
| Effects | Pedalboard (Spotify) |
| Transcription | Whisper / Whisper Turbo (PyTorch or MLX) |
| Inference | MLX (Apple Silicon) / PyTorch (CUDA/ROCm/XPU/CPU) |
| Database | SQLite |
| Audio | WaveSurfer.js, librosa |
+18
View File
@@ -0,0 +1,18 @@
{
"title": "Overview",
"defaultOpen": true,
"pages": [
"introduction",
"installation",
"quick-start",
"voice-cloning",
"stories-editor",
"recording-transcription",
"generation-history",
"remote-mode",
"creating-voice-profiles",
"generating-speech",
"building-stories",
"troubleshooting"
]
}
@@ -46,9 +46,9 @@ Voice profiles are the foundation of Voicebox. Each profile contains voice sampl
</Step>
</Steps>
<Tip>
<Callout type="info">
For best results, use clean audio with minimal background noise and consistent speaking tone.
</Tip>
</Callout>
## Step 2: Generate Speech
@@ -74,9 +74,9 @@ Now let's use your new voice profile to generate speech.
<Step title="Generate">
Click **Generate** and wait a few seconds
<Note>
<Callout type="info">
First generation may take longer due to model initialization. Subsequent generations will be faster.
</Note>
</Callout>
</Step>
<Step title="Play & Download">
@@ -114,20 +114,20 @@ The Stories Editor lets you create multi-voice narratives with a timeline-based
## What's Next?
<CardGroup cols={2}>
<Card title="Voice Cloning Guide" icon="microphone" href="/guides/creating-voice-profiles">
<Cards>
<Card title="Voice Cloning Guide" href="/overview/creating-voice-profiles">
Learn advanced techniques for high-quality voice cloning
</Card>
<Card title="API Integration" icon="code" href="/api/overview">
<Card title="API Integration" href="/api-reference">
Integrate Voicebox into your own applications
</Card>
<Card title="Stories Editor" icon="film" href="/overview/stories-editor">
<Card title="Stories Editor" href="/overview/stories-editor">
Master the multi-track timeline editor
</Card>
<Card title="Remote Mode" icon="server" href="/overview/remote-mode">
<Card title="Remote Mode" href="/overview/remote-mode">
Connect to a GPU server for faster generation
</Card>
</CardGroup>
</Cards>
## Tips for Success
@@ -149,6 +149,6 @@ The Stories Editor lets you create multi-voice narratives with a timeline-based
- **Server won't start:** Check if port 17493 is available
- **Poor audio quality:** Try adding more voice samples
- **Slow generation:** Verify GPU acceleration is enabled
- See the full [Troubleshooting Guide](/guides/troubleshooting) for more
- See the full [Troubleshooting Guide](/overview/troubleshooting) for more
</Accordion>
</AccordionGroup>
@@ -59,6 +59,6 @@ Automatic speech-to-text powered by OpenAI's Whisper model.
</Step>
</Steps>
<Tip>
<Callout type="info">
Transcription is useful for creating voice samples from existing audio or generating subtitles.
</Tip>
</Callout>
@@ -41,9 +41,9 @@ In Remote Mode, the Voicebox desktop app (running on your local machine) communi
uvicorn main:app --host 0.0.0.0 --port 17493
```
<Warning>
<Callout type="warn">
This exposes the server to your network. Use a firewall or VPN for security.
</Warning>
</Callout>
</Step>
<Step title="Open Firewall">
@@ -108,9 +108,9 @@ In Remote Mode, the Voicebox desktop app (running on your local machine) communi
## Security Considerations
<Warning>
<Callout type="warn">
The API currently has no authentication. Only use on trusted networks or with a VPN.
</Warning>
</Callout>
**Best Practices:**
- Use a VPN (WireGuard, Tailscale) instead of exposing to the internet
@@ -129,9 +129,9 @@ Expected performance on various GPUs:
| RTX 3060 | ~5-7s per 10 words |
| CPU (12-core) | ~20-30s per 10 words |
<Tip>
<Callout type="info">
A GPU with 8GB+ VRAM is recommended for best performance.
</Tip>
</Callout>
## Troubleshooting
@@ -9,20 +9,20 @@ The Stories Editor is a DAW-like timeline interface for creating multi-voice nar
## Features
<CardGroup cols={2}>
<Card title="Multi-Track Timeline" icon="timeline">
<Cards>
<Card title="Multi-Track Timeline">
Arrange multiple voice tracks in parallel
</Card>
<Card title="Inline Editing" icon="scissors">
<Card title="Inline Editing">
Trim and split clips directly in the timeline
</Card>
<Card title="Auto-Playback" icon="play">
<Card title="Auto-Playback">
Preview with synchronized playhead
</Card>
<Card title="Voice Mixing" icon="users">
<Card title="Voice Mixing">
Build conversations with multiple speakers
</Card>
</CardGroup>
</Cards>
## Creating a Story
@@ -25,9 +25,9 @@ Windows SmartScreen may warn that the app is unrecognized.
- Click "More info"
- Click "Run anyway"
<Note>
<Callout type="info">
This is expected for unsigned applications. We're working on code signing for future releases.
</Note>
</Callout>
## Server Issues
@@ -163,9 +163,9 @@ This is expected behavior. The first generation downloads the Qwen3-TTS model (~
Settings → Generation → Use CPU instead of GPU
<Warning>
<Callout type="warn">
CPU generation is 5-10x slower but uses system RAM instead of VRAM.
</Warning>
</Callout>
</Accordion>
<Accordion title="Reduce Batch Size">
@@ -322,9 +322,9 @@ bun run tauri build
**Solutions:**
<Warning>
<Callout type="warn">
This will delete all your voice profiles and generation history. Export important profiles first if possible.
</Warning>
</Callout>
```bash
# macOS
@@ -28,20 +28,20 @@ Voicebox uses **Qwen3-TTS** from Alibaba to achieve near-perfect voice cloning f
### Sample Quality
<CardGroup cols={2}>
<Card title="Do" icon="check">
<Cards>
<Card title="Do">
- Use 10-30 seconds of audio
- Clear, consistent speaking
- Minimal background noise
- Natural speaking pace
</Card>
<Card title="Don't" icon="xmark">
<Card title="Don't">
- Very short clips (< 5 seconds)
- Heavy background noise
- Music or overlapping voices
- Heavily processed audio
</Card>
</CardGroup>
</Cards>
### Multiple Samples
@@ -51,9 +51,9 @@ Adding multiple samples from the same speaker can improve quality:
- Different emotions (happy, serious)
- Different recording conditions
<Tip>
<Callout type="info">
The model will learn a more robust representation from diverse samples.
</Tip>
</Callout>
## Supported Languages
@@ -65,9 +65,9 @@ More languages coming soon.
## Limitations
<Warning>
<Callout type="warn">
Voice cloning should only be used with consent. Ensure you have permission to clone someone's voice.
</Warning>
</Callout>
- Quality depends on sample clarity
- Works best with consistent speaking tone
-15
View File
@@ -1,15 +0,0 @@
/* Anchor hover styles */
.nav-anchor:hover {
@apply text-[#BF9E40];
}
/* Icon wrapper on hover */
.nav-anchor:hover div {
background: #BF9E40 !important;
filter: brightness(1) !important;
}
/* Icon SVG on hover */
.nav-anchor:hover svg {
@apply bg-white !important;
}
-84
View File
@@ -1,84 +0,0 @@
---
title: "Auto-Updater"
description: "Configure and use the Tauri auto-updater"
---
## Overview
Voicebox uses Tauri's built-in auto-updater to deliver updates to users automatically.
## Quick Reference
For detailed setup instructions, see the existing documentation:
- [AUTOUPDATER_QUICKSTART.md](https://github.com/jamiepine/voicebox/blob/main/docs/AUTOUPDATER_QUICKSTART.md)
- [AUTOUPDATER.md](https://github.com/jamiepine/voicebox/blob/main/docs/AUTOUPDATER.md)
## How It Works
The auto-updater follows a secure update process:
1. **Check for Updates** - The Voicebox app periodically checks GitHub Releases for new versions
2. **Download Update** - If a new version is found, the update package is downloaded
3. **Verify Signature** - The downloaded package is cryptographically verified using the public key
4. **Install** - After verification, the update is installed
5. **Restart** - The app restarts with the new version
## Configuration
Updates are configured in `tauri/src-tauri/tauri.conf.json`:
```json
{
"updater": {
"active": true,
"endpoints": [
"https://github.com/jamiepine/voicebox/releases/latest/download/latest.json"
],
"dialog": true,
"pubkey": "YOUR_PUBLIC_KEY"
}
}
```
## Generating Keys
```bash
# Generate signing keys
bun run generate:keys
# Keys saved to ~/.tauri/voicebox.key
```
<Warning>
Keep your private key secure! Never commit it to the repository.
</Warning>
## Release Process
1. **Bump version** using bumpversion
2. **Push tag** to trigger CI/CD
3. **GitHub Actions** builds and signs releases
4. **Users** receive update notification
## User Experience
When an update is available:
1. User sees a notification dialog
2. User clicks "Update"
3. Update downloads in background
4. App restarts with new version
## For Developers
See the full documentation files for:
- Setting up signing keys
- Configuring GitHub releases
- Testing updates locally
- Troubleshooting update failures
<Card title="View Full Docs" href="https://github.com/jamiepine/voicebox/tree/main/docs">
Access AUTOUPDATER.md and AUTOUPDATER_QUICKSTART.md in the repository
</Card>
-270
View File
@@ -1,270 +0,0 @@
---
title: "Building"
description: "Build Voicebox for production"
---
## Overview
Voicebox uses a multi-step build process to create platform-specific installers.
## Quick Build
```bash
# Build for your current platform (automatically builds server binary first)
make build
# Or manually
bun run build
```
This automatically:
1. Builds the Python server binary (`bun run build:server`)
2. Builds the Tauri app (`cd tauri && bun run tauri build`)
## Build Process
The build process consists of two steps, but `bun run build` handles both automatically:
### 1. Server Binary Build (Automatic)
The Python backend is compiled into a standalone executable using PyInstaller. This happens automatically when you run `bun run build`.
**Platform-specific binaries:**
- macOS (Apple Silicon): `voicebox-server-aarch64-apple-darwin` (includes MLX backend)
- macOS (Intel): `voicebox-server-x86_64-apple-darwin` (PyTorch backend)
- Windows: `voicebox-server-x86_64-pc-windows-msvc.exe` (PyTorch backend)
- Linux: `voicebox-server-x86_64-unknown-linux-gnu` (PyTorch backend)
<Note>
The build script automatically detects your platform and includes the appropriate backend (MLX for Apple Silicon, PyTorch for others).
</Note>
**Manual build (if needed):**
```bash
bun run build:server
```
### 2. Tauri App Build (Automatic)
The Tauri app build is also handled automatically, which:
1. Builds the React frontend (Vite)
2. Compiles the Rust backend
3. Bundles the server binary as a sidecar
4. Creates platform-specific installers
**Manual build (if needed):**
```bash
cd tauri && bun run tauri build
```
### 3. Output
Installers are created in `tauri/src-tauri/target/release/bundle/`:
**macOS:**
- `dmg/` - Disk image installer
- `macos/` - App bundle
**Windows:**
- `msi/` - MSI installer
- `nsis/` - NSIS installer
**Linux:**
- `deb/` - Debian package
- `appimage/` - AppImage
## Advanced Options
### Building for Specific Platform
```bash
# Build for macOS (Apple Silicon)
bun run tauri build -- --target aarch64-apple-darwin
# Build for macOS (Intel)
bun run tauri build -- --target x86_64-apple-darwin
# Build for Windows
bun run tauri build -- --target x86_64-pc-windows-msvc
# Build for Linux
bun run tauri build -- --target x86_64-unknown-linux-gnu
```
### Using Local Qwen3-TTS
If you're developing Qwen3-TTS locally:
```bash
export QWEN_TTS_PATH=~/path/to/Qwen3-TTS
bun run build:server # Build server binary only
# or
bun run build # Build everything
```
This makes PyInstaller use your local version instead of the pip package.
### Debug Build
```bash
cd tauri
bun run tauri build --debug
```
Creates a debug build with symbols and logging.
## Build Configuration
### Tauri Config
Edit `tauri/src-tauri/tauri.conf.json`:
```json
{
"bundle": {
"identifier": "com.voicebox.app",
"icon": [
"icons/32x32.png",
"icons/128x128.png",
"icons/icon.icns",
"icons/icon.ico"
]
}
}
```
### Sidecar Configuration
The Python server is bundled as a sidecar:
```json
{
"tauri": {
"bundle": {
"externalBin": [
"binaries/voicebox-server"
]
}
}
}
```
## Code Signing
### macOS
To sign the app for distribution:
```bash
# Set signing identity
export APPLE_SIGNING_IDENTITY="Developer ID Application: Your Name"
# Build with signing
bun run tauri build
```
For notarization:
```bash
# Set credentials
export APPLE_ID="[email protected]"
export APPLE_PASSWORD="app-specific-password"
# Build and notarize
bun run tauri build
```
### Windows
For Windows code signing:
```bash
# Set certificate
export WINDOWS_CERTIFICATE_PATH="/path/to/cert.pfx"
export WINDOWS_CERTIFICATE_PASSWORD="password"
# Build with signing
bun run tauri build
```
## Release Process
The full release process is automated:
```bash
# 1. Bump version
bumpversion patch # or minor/major
# 2. Build all platforms (CI/CD handles this)
git push --tags
# 3. GitHub Actions creates releases
```
See [CONTRIBUTING.md](/development/contributing) for the full release workflow.
## Troubleshooting
<AccordionGroup>
<Accordion title="Server Binary Build Fails">
**Common issues:**
- Missing Python dependencies: `pip install -r requirements.txt`
- PyInstaller not found: `pip install pyinstaller`
- Qwen3-TTS not installed: `pip install git+https://github.com/QwenLM/Qwen3-TTS.git`
**Solution:**
```bash
cd backend
source venv/bin/activate
pip install -r requirements.txt
pip install pyinstaller
```
</Accordion>
<Accordion title="Tauri Build Fails">
**Common issues:**
- Rust not installed: `curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh`
- Server binary missing: Usually auto-built, but can run manually: `./scripts/build-server.sh`
- Node modules outdated: `bun install`
**Solution:**
```bash
# Clean and rebuild
cd tauri/src-tauri
cargo clean
cd ../..
bun run build # Automatically builds server binary first
```
</Accordion>
<Accordion title="App Won't Launch After Build">
**Check:**
- Server binary has execute permissions
- All dependencies are bundled
- Check logs in the app's data directory
**macOS:**
```bash
tail -f ~/Library/Application\ Support/com.voicebox.app/logs/server.log
```
**Windows:**
```bash
type %APPDATA%\com.voicebox.app\logs\server.log
```
</Accordion>
</AccordionGroup>
## CI/CD
GitHub Actions automatically builds releases when tags are pushed:
```yaml
# .github/workflows/release.yml
on:
push:
tags:
- 'v*'
```
See the [repository](https://github.com/jamiepine/voicebox) for the full CI/CD configuration.
-239
View File
@@ -1,239 +0,0 @@
---
title: "Development Setup"
description: "Set up your local development environment for Voicebox"
---
## Prerequisites
Before you begin, ensure you have the following installed:
<CardGroup cols={3}>
<Card title="Bun" icon="package">
[Download Bun](https://bun.sh)
```bash
curl -fsSL https://bun.sh/install | bash
```
</Card>
<Card title="Python 3.11+" icon="python">
[Download Python](https://python.org)
```bash
python --version
```
</Card>
<Card title="Rust" icon="rust">
[Install Rust](https://rustup.rs)
```bash
rustc --version
```
</Card>
</CardGroup>
## Clone the Repository
```bash
git clone https://github.com/jamiepine/voicebox.git
cd voicebox
```
## Quick Setup (Recommended)
The easiest way to get started is using the Makefile:
```bash
# Setup everything
make setup
# Start development
make dev
```
<Note>
The Makefile is available on macOS and Linux. Windows users should follow the manual setup below.
</Note>
## Manual Setup
### 1. Install JavaScript Dependencies
```bash
bun install
```
This installs dependencies for:
- `app/` - Shared React frontend
- `tauri/` - Tauri desktop wrapper
- `web/` - Web deployment wrapper
### 2. Set Up Python Backend
```bash
cd backend
# Create virtual environment
python -m venv venv
# Activate virtual environment
source venv/bin/activate # macOS/Linux
# or
venv\Scripts\activate # Windows
# Install Python dependencies
pip install -r requirements.txt
# Install MLX dependencies (Apple Silicon only - for faster inference)
# On Apple Silicon, this enables native Metal acceleration
if [[ $(uname -m) == "arm64" ]]; then
pip install -r requirements-mlx.txt
fi
# Install Qwen3-TTS
pip install git+https://github.com/QwenLM/Qwen3-TTS.git
```
## Running in Development
Development requires **two terminals**: one for the Python backend, one for the Tauri app.
<Tabs>
<Tab title="Terminal 1: Backend">
Start the Python server first:
```bash
cd backend
source venv/bin/activate # Activate venv
bun run dev:server
```
Or manually:
```bash
uvicorn main:app --reload --port 17493
```
Backend will be available at `http://localhost:17493`
</Tab>
<Tab title="Terminal 2: Desktop App">
Then start the Tauri app:
```bash
bun run dev
```
This will:
- Create a placeholder sidecar binary
- Start Vite dev server on port 5173
- Launch Tauri window
- Enable hot reload
</Tab>
</Tabs>
<Info>
In dev mode, the app connects to your manually-started Python server. The bundled server binary is only used in production builds.
</Info>
### Optional: Web App
```bash
bun run dev:web
```
Web app will be available at `http://localhost:5174`
## Model Downloads
Models are automatically downloaded from HuggingFace Hub on first use:
- **Whisper** (transcription): Auto-downloads on first transcription
- **Qwen3-TTS** (voice cloning): Auto-downloads on first generation (~2-4GB)
<Warning>
First-time usage will be slower due to model downloads, but subsequent runs will use cached models.
</Warning>
## Project Structure
```
voicebox/
├── app/ # Shared React frontend
│ └── src/
│ ├── components/ # UI components
│ ├── lib/ # Utilities and API client
│ └── hooks/ # React hooks
├── backend/ # Python FastAPI server
│ ├── main.py # API routes
│ ├── tts.py # Voice synthesis
│ └── database.py # SQLite operations
├── tauri/ # Desktop app wrapper
│ └── src-tauri/ # Rust backend
├── web/ # Web deployment
├── landing/ # Marketing website
└── scripts/ # Build & release scripts
```
## Available Make Commands
Run `make help` to see all available commands:
```bash
make setup # Install all dependencies
make dev # Start development servers
make dev-web # Start web development server
make build # Build desktop app
make build-web # Build web app
make clean # Clean build artifacts
make test # Run tests
```
## Generate OpenAPI Client
After starting the backend server, generate the TypeScript API client:
```bash
./scripts/generate-api.sh
# or
bun run generate:api
```
This downloads the OpenAPI schema and generates the TypeScript client in `app/src/lib/api/`
## Next Steps
<CardGroup cols={2}>
<Card title="Architecture" icon="diagram-project" href="/development/architecture">
Understand the system architecture
</Card>
<Card title="Contributing" icon="code-pull-request" href="/development/contributing">
Read the contribution guidelines
</Card>
<Card title="Building" icon="hammer" href="/development/building">
Learn how to build production releases
</Card>
<Card title="API Reference" icon="code" href="/api/overview">
Explore the REST API
</Card>
</CardGroup>
## Troubleshooting
<AccordionGroup>
<Accordion title="Backend won't start">
- Check Python version (must be 3.11+)
- Ensure virtual environment is activated
- Verify all dependencies are installed: `pip install -r requirements.txt`
- Check if port 17493 is available
</Accordion>
<Accordion title="Tauri build fails">
- Ensure Rust is installed: `rustc --version`
- Clean the build: `cd tauri/src-tauri && cargo clean`
- Try rebuilding: `bun run dev`
</Accordion>
<Accordion title="OpenAPI client generation fails">
- Ensure backend is running: `curl http://localhost:17493/openapi.json`
- Check network connectivity
- Verify the backend is accessible at localhost:17493
</Accordion>
</AccordionGroup>
See the full [Troubleshooting Guide](/guides/troubleshooting) for more issues and solutions.
+1
View File
@@ -0,0 +1 @@
export { twMerge as cn } from 'tailwind-merge';
+9
View File
@@ -0,0 +1,9 @@
import type { BaseLayoutProps } from 'fumadocs-ui/layouts/shared';
export function baseOptions(): BaseLayoutProps {
return {
nav: {
title: 'Voicebox',
},
};
}
+5
View File
@@ -0,0 +1,5 @@
import { createOpenAPI } from 'fumadocs-openapi/server';
export const openapi = createOpenAPI({
input: ['./openapi.json'],
});
+27
View File
@@ -0,0 +1,27 @@
import { docs } from '@/.source';
import { type InferPageType, loader } from 'fumadocs-core/source';
import { lucideIconsPlugin } from 'fumadocs-core/source/lucide-icons';
// See https://fumadocs.dev/docs/headless/source-api for more info
export const source = loader({
baseUrl: '/docs',
source: docs.toFumadocsSource(),
plugins: [lucideIconsPlugin()],
});
export function getPageImage(page: InferPageType<typeof source>) {
const segments = [...page.slugs, 'image.png'];
return {
segments,
url: `/og/docs/${segments.join('/')}`,
};
}
export async function getLLMText(page: InferPageType<typeof source>) {
const processed = await page.data.getText('processed');
return `# ${page.data.title}
${processed}`;
}
+53
View File
@@ -0,0 +1,53 @@
import { Callout } from 'fumadocs-ui/components/callout';
import { Card, Cards } from 'fumadocs-ui/components/card';
import { File, Files, Folder } from 'fumadocs-ui/components/files';
import { Step, Steps } from 'fumadocs-ui/components/steps';
import { Tab, Tabs } from 'fumadocs-ui/components/tabs';
import defaultMdxComponents from 'fumadocs-ui/mdx';
import type { MDXComponents } from 'mdx/types';
import type { ReactNode } from 'react';
import { APIPage } from '@/components/api-page';
// Simple accordion using native HTML details/summary
function AccordionGroup({ children }: { children: ReactNode }) {
return <div className="my-6 space-y-2">{children}</div>;
}
function Accordion({ title, children }: { title: string; children: ReactNode }) {
return (
<details className="group border rounded-lg p-4">
<summary className="cursor-pointer font-semibold list-none">
<span className="group-open:rotate-90 transition-transform inline-block mr-2">▶</span>
{title}
</summary>
<div className="mt-4 pl-6">{children}</div>
</details>
);
}
export function getMDXComponents(components?: MDXComponents): MDXComponents {
return {
...defaultMdxComponents,
// Layout components
Card,
Cards,
// Files
Files,
Folder,
File,
// Callouts
Callout,
// Tabs
Tabs,
Tab,
// Steps
Steps,
Step,
// Accordion (native HTML-based)
AccordionGroup,
Accordion,
// OpenAPI component
APIPage,
...components,
};
}
-102
View File
@@ -1,102 +0,0 @@
{
"$schema": "https://mintlify.com/schema.json",
"name": "Voicebox",
"logo": {
"light": "/logo/icon-light.png",
"dark": "/logo/icon-dark.png"
},
"favicon": "/favicon.png",
"colors": {
"primary": "#BF9E40",
"light": "#D4B560",
"dark": "#A68A35"
},
"styles": {
"css": ["/custom.css"]
},
"anchors": [
{
"name": "Overview",
"icon": "book-open",
"url": "overview"
},
{
"name": "API",
"icon": "code",
"url": "api"
},
{
"name": "Developer",
"icon": "book-open-cover",
"url": "developer"
},
{
"name": "GitHub",
"icon": "github",
"url": "https://github.com/jamiepine/voicebox"
}
],
"navigation": [
{
"group": "Getting Started",
"icon": "rocket",
"pages": ["overview/introduction", "overview/installation", "overview/quick-start"]
},
{
"group": "Features",
"icon": "sparkles",
"pages": [
"overview/voice-cloning",
"overview/stories-editor",
"overview/recording-transcription",
"overview/generation-history",
"overview/remote-mode"
]
},
{
"group": "User Guides",
"icon": "compass",
"pages": [
"overview/creating-voice-profiles",
"overview/generating-speech",
"overview/building-stories",
"overview/troubleshooting"
]
},
{
"group": "Development",
"icon": "wrench",
"pages": [
"developer/setup",
"developer/architecture",
"developer/contributing",
"developer/building",
"developer/autoupdater"
]
},
{
"group": "API Reference",
"icon": "code",
"pages": [
"api/overview",
"api/authentication",
"api/voice-profiles",
"api/generation",
"api/recordings"
]
},
{
"group": "Architecture",
"icon": "book-open-cover",
"pages": [
"developer/voice-profiles",
"developer/tts-generation",
"developer/history",
"developer/stories",
"developer/transcription",
"developer/audio-channels",
"developer/model-management"
]
}
]
}
+25
View File
@@ -0,0 +1,25 @@
import { createMDX } from 'fumadocs-mdx/next';
const withMDX = createMDX();
/** @type {import('next').NextConfig} */
const config = {
reactStrictMode: true,
async rewrites() {
return [
{
source: '/docs/:path*.mdx',
destination: '/llms.mdx/docs/:path*',
},
];
},
webpack: (config) => {
config.experiments = {
...config.experiments,
topLevelAwait: true,
};
return config;
},
};
export default withMDX(config);
-11
View File
@@ -1,11 +0,0 @@
[phases.setup]
nixPkgs = ["nodejs_20", "bun"]
[phases.install]
cmds = ["bun install"]
[phases.build]
cmds = ["bun run build"]
[start]
cmd = "bun run start"
File diff suppressed because one or more lines are too long
-58
View File
@@ -1,58 +0,0 @@
---
title: "Introduction"
description: "Welcome to Voicebox - the open-source voice synthesis studio"
---
## What is Voicebox?
Voicebox is a **local-first voice cloning studio** with DAW-like features for professional voice synthesis. Think of it as a **local, free and open-source alternative to ElevenLabs** — download models, clone voices, and generate speech entirely on your machine.
<Frame>
<img src="/images/app-screenshot-1.webp" alt="Voicebox App Screenshot" />
</Frame>
Unlike cloud services that lock your voice data behind subscriptions, Voicebox gives you:
- **Complete privacy** — models and voice data stay on your machine
- **Professional tools** — multi-track timeline editor, audio trimming, conversation mixing
- **Model flexibility** — currently powered by Qwen3-TTS, with support for XTTS, Bark, and other models coming soon
- **API-first** — use the desktop app or integrate voice synthesis into your own projects
- **Native performance** — built with Tauri (Rust), not Electron
Download a voice model, clone any voice from a few seconds of audio, and compose multi-voice projects with studio-grade editing tools. No Python install required, no cloud dependency, no limits.
## Key Features
<CardGroup cols={2}>
<Card title="Voice Cloning" icon="microphone">
Instant cloning from just a few seconds of audio with Qwen3-TTS
</Card>
<Card title="Stories Editor" icon="film">
Multi-track timeline for creating conversations and narratives
</Card>
<Card title="Full API" icon="code">
REST API for integrating voice synthesis into your apps
</Card>
<Card title="Local-First" icon="shield">
Everything runs on your machine - complete privacy
</Card>
</CardGroup>
## Use Cases
- **Game Development** — Generate dynamic dialogue for characters
- **Content Creation** — Produce podcasts and video voiceovers
- **Accessibility** — Build text-to-speech tools
- **Voice Assistants** — Create custom voice interfaces
- **Production Pipelines** — Automate voiceover workflows
## Next Steps
<CardGroup cols={2}>
<Card title="Installation" icon="download" href="/overview/installation">
Download and install Voicebox on your machine
</Card>
<Card title="Quick Start" icon="rocket" href="/overview/quick-start">
Get up and running in 5 minutes
</Card>
</CardGroup>
+28 -10
View File
@@ -1,17 +1,35 @@
{
"name": "voicebox-docs",
"version": "0.1.0",
"name": "example-next-mdx",
"version": "0.0.0",
"private": true,
"scripts": {
"dev": "mintlify dev",
"build": "mintlify build",
"start": "mintlify serve",
"install:mintlify": "bun add -g mintlify"
"build": "fumadocs-mdx && next build",
"dev": "fumadocs-mdx && next dev",
"start": "next start",
"postinstall": "fumadocs-mdx"
},
"dependencies": {
"@radix-ui/react-popover": "^1.1.15",
"class-variance-authority": "^0.7.1",
"fumadocs-core": "^16.4.11",
"fumadocs-mdx": "13",
"fumadocs-openapi": "^10.2.7",
"fumadocs-ui": "^16.4.11",
"lucide-react": "^0.546.0",
"next": "^16.1.6",
"react": "^19.2.0",
"react-dom": "^19.2.0",
"shiki": "^3.22.0",
"tailwind-merge": "^3.5.0"
},
"devDependencies": {
"mintlify": "latest"
},
"engines": {
"bun": ">=1.0.0"
"@tailwindcss/postcss": "^4.1.15",
"@types/mdx": "^2.0.13",
"@types/node": "^24.9.1",
"@types/react": "^19.2.2",
"@types/react-dom": "^19.2.2",
"postcss": "^8.5.6",
"tailwindcss": "^4.1.15",
"typescript": "^5.9.3"
}
}
-363
View File
@@ -1,363 +0,0 @@
# Adding a TTS Engine to Voicebox
Guide for adding new TTS model backends. Based on the implementation of LuxTTS (#254), Chatterbox Multilingual (#257), Chatterbox Turbo (#258), and the PyInstaller fixes in v0.2.3.
---
## Overview
Adding an engine touches ~10 files across 4 layers. The backend protocol work is straightforward — the real time sink is dependency hell, upstream library bugs, and PyInstaller bundling.
The backend is split into layers: `routes/` (thin HTTP handlers), `services/` (business logic), `backends/` (engine implementations), and `utils/` (shared utilities). New engines only need to touch `backends/` and `models.py` on the backend side — the route and service layers use a model config registry that handles dispatch automatically.
---
## Phase 1: Backend Implementation
### 1.1 Create the backend file
`backend/backends/<engine>_backend.py` (~200-300 lines)
Implement the `TTSBackend` protocol from `backend/backends/__init__.py`:
```python
class YourBackend:
"""Must satisfy the TTSBackend protocol."""
async def load_model(self, model_size: str = "default") -> None: ...
async def create_voice_prompt(self, audio_path: str, reference_text: str, use_cache: bool = True) -> tuple[dict, bool]: ...
async def combine_voice_prompts(self, audio_paths: list[str], ref_texts: list[str]) -> tuple[np.ndarray, str]: ...
async def generate(self, text: str, voice_prompt: dict, language: str = "en", seed: int | None = None, instruct: str | None = None) -> tuple[np.ndarray, int]: ...
def unload_model(self) -> None: ...
def is_loaded(self) -> bool: ...
def _get_model_path(self, model_size: str) -> str: ...
```
Key decisions per engine:
| Decision | Options | Examples |
|----------|---------|---------|
| **Voice prompt storage** | Pre-computed tensors vs deferred file paths | Qwen PyTorch stores tensor dicts; Chatterbox stores `{"ref_audio": path, "ref_text": text}` |
| **Caching** | Use voice prompt cache or skip it | LuxTTS caches with `luxtts_` prefix; Chatterbox skips caching entirely |
| **Device selection** | CUDA / MPS / CPU | Chatterbox forces CPU on macOS (MPS tensor bugs); LuxTTS supports MPS |
| **Model download** | Library handles it vs manual `snapshot_download` | Turbo uses manual download to bypass upstream `token=True` bug |
| **Sample rate** | Engine-specific | LuxTTS outputs 48kHz, everything else is 24kHz |
### 1.2 Voice prompt patterns
There are three patterns in use. Pick the one that fits your model:
**Pattern A: Pre-computed tensors** (Qwen PyTorch, LuxTTS)
```python
# create_voice_prompt returns opaque dict of tensors
# Cached via torch.save(), reused across generations
encoded = model.encode_prompt(audio_path)
return encoded, False # (prompt_dict, was_cached)
```
**Pattern B: Deferred file paths** (Chatterbox, MLX)
```python
# Just store paths, process at generation time
return {"ref_audio": audio_path, "ref_text": reference_text}, False
```
**Pattern C: Hybrid** (possible for new engines)
```python
# Pre-compute speaker embeddings, store alongside paths
embedding = model.extract_speaker(audio_path)
return {"embedding": embedding, "ref_audio": audio_path}, False
```
If caching, prefix your cache keys to avoid collisions with other engines using the same reference audio:
```python
cache_key = "yourengine_" + get_cache_key(audio_path, reference_text)
```
### 1.3 Register the engine
In `backend/backends/__init__.py`, three things:
**1. Add a `ModelConfig` entry** in `_get_non_qwen_tts_configs()`:
```python
ModelConfig(
model_name="your-engine",
display_name="Your Engine",
engine="your_engine",
hf_repo_id="org/model-repo",
size_mb=3200,
needs_trim=False, # set True if output needs trim_tts_output()
languages=["en", "fr", "de"],
),
```
This single entry replaces what used to be 6+ scattered dicts in `main.py`. The registry helpers (`get_model_config()`, `check_model_loaded()`, `engine_needs_trim()`, etc.) all derive from this config automatically.
**2. Add to `TTS_ENGINES` dict:**
```python
TTS_ENGINES = {
...
"your_engine": "Your Engine",
}
```
**3. Add an elif branch in `get_tts_backend_for_engine()`:**
```python
elif engine == "your_engine":
from .your_backend import YourBackend
backend = YourBackend()
```
The import is deferred so platform-specific deps aren't loaded until the engine is first requested.
### 1.4 Update request models
In `backend/models.py`:
- Add engine name to `GenerationRequest.engine` regex pattern
- Add any new language codes to the language regex on both `GenerationRequest` and `VoiceProfileCreate`
---
## Phase 2: Route and Service Integration
With the model config registry, the route and service layers have **zero per-engine dispatch points**. All endpoints use registry helpers like `get_model_config()`, `load_engine_model()`, `engine_needs_trim()`, `check_model_loaded()`, etc.
**You don't need to touch any route or service files** unless your engine needs custom behavior in the generate pipeline (e.g. a new post-processing step beyond `trim_tts_output`).
### 2.1 What the registry handles automatically
| Route file | Registry function used |
|------------|----------------------|
| `routes/generations.py` | `load_engine_model(engine, size)` + `engine_needs_trim(engine)` |
| `routes/models.py` | `get_all_model_configs()` + `check_model_loaded(config)` |
| `routes/models.py` | `get_model_config(name)` + `get_model_load_func(config)` |
| `services/generation.py` | `get_tts_backend_for_engine()` + `ensure_model_cached_or_raise()` |
### 2.2 Post-processing
If your model produces trailing silence or hallucinated audio, set `needs_trim=True` on your `ModelConfig`. The generation service checks `engine_needs_trim(engine)` and applies `trim_tts_output()` automatically.
---
## Phase 3: Frontend Integration
### 3.1 TypeScript types
In `app/src/lib/api/types.ts`:
- Add to the `engine` union type on `GenerationRequest`
### 3.2 Language maps
In `app/src/lib/constants/languages.ts`:
- Add entry to `ENGINE_LANGUAGES` record
- Add any new language codes to `ALL_LANGUAGES` if needed
### 3.3 Engine/model selector (shared component)
The model selector is a shared component — update one file:
- `app/src/components/Generation/EngineModelSelector.tsx`
Add an entry to `ENGINE_OPTIONS` and `ENGINE_DESCRIPTIONS`. If the engine is English-only, add it to `ENGLISH_ONLY_ENGINES`. The `handleEngineChange()` function handles language validation automatically (resets to first available language if the current one isn't supported).
Both `GenerationForm.tsx` and `FloatingGenerateBox.tsx` use `<EngineModelSelector>` — no changes needed in either.
Handle engine-specific UI conditionals in the form components if needed:
- Hide instruct field for engines that don't support it
- Show engine-specific controls (e.g. `ParalinguisticInput` for Turbo)
### 3.4 Form hook
In `app/src/lib/hooks/useGenerationForm.ts`:
- Add to Zod schema enum for `engine`
- Add engine-to-model-name mapping (e.g. `"your_engine"` → `"your-engine"`)
- Update payload construction to conditionally include engine-specific fields
### 3.5 Model management
In `app/src/components/ServerSettings/ModelManagement.tsx`:
- Add description to `MODEL_DESCRIPTIONS` record
- The model list auto-renders from `/models/status` data
---
## Phase 4: Dependencies
### 4.1 Python dependencies
Add to `backend/requirements.txt`. Watch for:
**Pinned dependency conflicts** — If the model package pins old versions of numpy, torch, or transformers, install with `--no-deps` and list sub-dependencies manually. This is what Chatterbox requires:
```
# In justfile (NOT requirements.txt):
pip install --no-deps chatterbox-tts
# In requirements.txt — list the transitive deps:
conformer
diffusers
omegaconf
# ... etc
```
**Non-PyPI packages** — Some deps only exist as git repos:
```
linacodec @ git+https://github.com/user/repo.git
Zipvoice @ git+https://github.com/user/repo.git
```
**Custom package indexes** — Some packages need `--find-links`:
```
--find-links https://k2-fsa.github.io/icefall/piper_phonemize.html
```
### 4.2 Identifying hidden sub-dependencies
When using `--no-deps`, you need to manually figure out what the package actually imports. There's no shortcut:
1. Install the package normally in a throwaway venv
2. Run `pip show <package>` to get its `Requires:` list
3. Cross-reference against what's already in our requirements.txt
4. Test that the engine loads and generates without import errors
---
## Phase 5: PyInstaller Bundling
This is where most of the pain lives. If your model's Python package or its dependencies use any of the following at runtime, PyInstaller won't bundle them automatically:
### 5.1 Common PyInstaller issues
| Issue | Symptom | Fix |
|-------|---------|-----|
| **`inspect.getsource()` at import time** | "could not get source code" | `--collect-all <package>` (bundles `.py` source files, not just bytecode) |
| **Data files (yaml, .pth.tar, lang dicts)** | FileNotFoundError at runtime | `--collect-all <package>` or `--collect-data <package>` |
| **Native data paths (espeak-ng, etc.)** | Library looks at `/usr/share/...` | Set env var in frozen builds: `os.environ["ESPEAK_DATA_PATH"] = bundled_path` |
| **`importlib.metadata` lookups** | "No package metadata found" | `--copy-metadata <package>` |
| **Dynamic imports** | ModuleNotFoundError | `--hidden-import <module>` |
| **`typeguard` / `@typechecked`** | Calls `inspect.getsource()` on decorated functions | `--collect-all` for the decorated package |
### 5.2 Testing frozen builds
You can't skip this. Models that work in `python -m uvicorn` will break in the PyInstaller binary. The flow:
1. Build the binary: `just build` or the PyInstaller spec
2. Run it and try to download + load + generate with the new engine
3. Check stderr for the actual error (macOS/Linux: stdout/stderr go to Tauri sidecar logs)
4. Fix, rebuild, repeat
### 5.3 Real examples from v0.2.3
These were all models that worked perfectly in dev:
- **LuxTTS**: `typeguard`'s `@typechecked` calls `inspect.getsource()` at import → needed `--collect-all inflect`. `piper_phonemize` bundles `espeak-ng-data/` → needed `--collect-all piper_phonemize` + `ESPEAK_DATA_PATH` env var
- **Chatterbox**: `resemble-perth` bundles a pretrained watermark model (`.pth.tar`, `hparams.yaml`) → needed `--collect-all perth`
- **Both**: `huggingface_hub` silently disables tqdm based on logger level → progress bars showed 0% in frozen builds until we force-enabled the internal counter
---
## Phase 6: Common Upstream Workarounds
Almost every model library has bugs you'll need to work around. Here's the catalog:
### 6.1 torch.load device mismatch
If model weights were saved on CUDA but you're loading on CPU/MPS:
```python
_original_torch_load = torch.load
def _patched_torch_load(*args, **kwargs):
kwargs.setdefault("map_location", "cpu")
return _original_torch_load(*args, **kwargs)
torch.load = _patched_torch_load
```
Used by both Chatterbox backends. Use a threading lock if patching globally.
### 6.2 Float64/Float32 dtype mismatch
`librosa` returns float64, model weights are float32. Patch the offending methods:
```python
original_fn = SomeClass.some_method
def patched_fn(self, *args, **kwargs):
result = original_fn(self, *args, **kwargs)
return result.float() # float64 → float32
SomeClass.some_method = patched_fn
```
Used by Chatterbox for `S3Tokenizer.log_mel_spectrogram` and `VoiceEncoder.forward`.
### 6.3 Transformers attention implementation
If the model uses `output_attentions=True` with transformers >= 4.36:
```python
for module in model.modules():
if hasattr(module, '_attn_implementation'):
module._attn_implementation = "eager"
```
SDPA (the new default) doesn't support `output_attentions`. Force eager attention.
### 6.4 HuggingFace token bug
Some models' `from_pretrained()` passes `token=True` which requires a stored HF token even for public repos:
```python
from huggingface_hub import snapshot_download
local_path = snapshot_download(repo_id=REPO, token=None)
model = ModelClass.from_local(local_path, device=device)
```
Used by Chatterbox Turbo.
### 6.5 MPS tensor issues
MPS (Apple Silicon GPU) has incomplete operator coverage. If generation crashes on MPS:
```python
def _get_device(self):
if torch.cuda.is_available():
return "cuda"
return "cpu" # Skip MPS entirely
```
Used by both Chatterbox backends. LuxTTS works fine on MPS.
### 6.6 HuggingFace progress tracking
To get download progress bars in the UI, wrap model loading with `HFProgressTracker`:
```python
from ..utils.hf_progress import HFProgressTracker
tracker = HFProgressTracker(model_name, progress_manager)
with tracker.patch_download():
model = ModelClass.from_pretrained(repo_id)
```
The tracker monkey-patches tqdm to intercept HuggingFace's internal progress bars. Must be set up BEFORE importing the model library if it imports HF at module level.
---
## Checklist
### Backend
- [ ] `backend/backends/<engine>_backend.py` — implements TTSBackend protocol
- [ ] `backend/backends/__init__.py` — `ModelConfig` entry + `TTS_ENGINES` + `get_tts_backend_for_engine()` elif
- [ ] `backend/models.py` — engine name in regex, any new language codes
- [ ] `backend/requirements.txt` — dependencies added (check for `--no-deps` needs)
- [ ] `justfile` — `--no-deps` install step if needed
### Routes and services
No changes needed — the model config registry handles all dispatch automatically.
### Frontend
- [ ] `app/src/lib/api/types.ts` — engine union type
- [ ] `app/src/lib/constants/languages.ts` — `ENGINE_LANGUAGES` entry
- [ ] `app/src/components/Generation/EngineModelSelector.tsx` — `ENGINE_OPTIONS` + `ENGINE_DESCRIPTIONS` + `ENGLISH_ONLY_ENGINES`
- [ ] `app/src/lib/hooks/useGenerationForm.ts` — Zod schema + model mapping
- [ ] `app/src/components/ServerSettings/ModelManagement.tsx` — model description
### Production
- [ ] PyInstaller spec — `--collect-all`, `--hidden-import`, `--copy-metadata` as needed
- [ ] Test in frozen binary — download, load, generate all work
- [ ] Download progress — `HFProgressTracker` wired up, progress shows in UI
### Upstream workarounds (check which apply)
- [ ] torch.load device mapping (CUDA weights on CPU)
- [ ] Float64→Float32 patches (librosa interaction)
- [ ] Eager attention forcing (transformers >= 4.36)
- [ ] HF token bypass (snapshot_download + from_local)
- [ ] MPS skip (if operators not supported)
- [ ] espeak-ng / native data path env vars
-581
View File
@@ -1,581 +0,0 @@
# CUDA Backend Swap via Binary Replacement
> Status: Plan | Target: v0.2.0 | Created: 2026-03-12
## Problem
The CUDA PyTorch backend binary is ~2.4 GB. GitHub Releases has a 2 GB asset limit. The current release ships CPU-only PyTorch on Windows and Intel Mac — NVIDIA GPU users get no acceleration from official releases. This is the #1 reported issue category (19 open issues).
Users who want GPU today must clone the repo and run from source. That's not acceptable for a desktop app targeting non-technical users.
## Solution
Ship two backend binaries: a default CPU build (~150 MB) bundled with the app, and a downloadable CUDA build (~2.4 GB) hosted externally. When the user downloads the CUDA build, the app kills the current backend process, swaps in the CUDA binary, and relaunches — a backend-only restart. The frontend stays running, all UI state is preserved.
No subprocesses. No HTTP protocol between processes. No port allocation. No provider manager. The backend is still one monolithic process — just a different binary.
## Architecture
### What Exists Today
```
Tauri App
├── React Frontend (in-process webview)
└── voicebox-server (sidecar subprocess on :17493)
└── One PyInstaller binary: CPU PyTorch or MLX
```
**Sidecar lifecycle** (`tauri/src-tauri/src/main.rs`):
- `start_server` command spawns `voicebox-server` sidecar (line 181)
- Binary located at `tauri/src-tauri/binaries/voicebox-server-{platform-triple}`
- Tauri resolves the sidecar name via `externalBin` in `tauri.conf.json` (line 16)
- Waits up to 120s for "Uvicorn running" in stdout/stderr (line 286)
- `stop_server` kills the process tree (line 466)
**Frontend reconnection** (`app/src/lib/hooks/useServer.ts`):
- Health check polls `GET /health` every 30 seconds
- React Query cache retains data for 10 minutes after disconnect
- All UI state (Zustand stores, form data, open tabs) survives disconnection
- No active reconnect logic — just keeps polling until server responds
This means a backend restart is mostly invisible to the frontend: it sees a few seconds of failed health checks, then the server comes back. The only risk is in-flight operations (generation, transcription) failing mid-request.
### What Changes
```
Tauri App
├── React Frontend (in-process webview)
└── voicebox-server (sidecar subprocess on :17493)
└── One of:
├── voicebox-server-cpu (bundled, ~150 MB)
└── voicebox-server-cuda (downloaded, ~2.4 GB)
```
The CUDA binary is functionally identical to the CPU binary. Same FastAPI app, same endpoints, same code. The only difference is PyTorch is compiled with CUDA 12.1 support and the binary includes CUDA runtime libraries.
The user downloads it once. On every subsequent app launch, Tauri checks which binary variant exists and spawns the appropriate one.
## Implementation Plan
### Phase 1: Build Infrastructure
Build the CUDA binary in CI separately from the main release.
#### 1a. CUDA PyInstaller Build
Add a `build_binary_cuda.py` or parameterize the existing `build_binary.py`:
```python
# backend/build_binary.py — add flag
def build_server(cuda=False):
args = [
'server.py',
'--onefile',
'--name', f'voicebox-server-{"cuda" if cuda else "cpu"}',
]
if cuda:
args.extend([
'--hidden-import', 'torch.cuda',
'--hidden-import', 'torch.backends.cudnn',
])
# ... rest of existing build
```
The `--onefile` flag is already used, which produces a single executable. This is important — `--onedir` would complicate the swap (replacing a directory vs a file).
#### 1b. CI Workflow for CUDA Binary
New workflow: `.github/workflows/build-cuda.yml`
```yaml
name: Build CUDA Provider
on:
workflow_dispatch:
push:
tags: ["v*"]
jobs:
build-cuda:
runs-on: windows-latest # CUDA is Windows/Linux only
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with: { python-version: "3.12" }
- name: Install dependencies
run: |
pip install pyinstaller
pip install -r backend/requirements.txt
pip install torch --index-url https://download.pytorch.org/whl/cu121 --force-reinstall
- name: Build CUDA binary
run: python backend/build_binary.py --cuda
- name: Split binary for GitHub Releases
run: |
python scripts/split_binary.py backend/dist/voicebox-server-cuda.exe \
--chunk-size 1900MB \
--output release-assets/
- name: Upload to R2
# Full binary to R2 (no size limit)
run: |
aws s3 cp backend/dist/voicebox-server-cuda.exe \
s3://voicebox-downloads/cuda/v${{ github.ref_name }}/voicebox-server-cuda.exe \
--endpoint-url ${{ secrets.R2_ENDPOINT }}
- name: Upload split parts to GitHub Release
# Split parts as GitHub Release assets (each <2 GB)
uses: softprops/action-gh-release@v1
with:
files: release-assets/*
```
Two distribution paths for redundancy:
- **Cloudflare R2**: Full binary, direct download, no size limit.
- **GitHub Releases**: Split into <2 GB chunks as fallback.
#### 1c. Binary Splitting Script
```python
# scripts/split_binary.py
"""Split a large binary into chunks for GitHub Releases."""
import hashlib
import argparse
from pathlib import Path
def split(input_path: Path, chunk_size: int, output_dir: Path):
output_dir.mkdir(parents=True, exist_ok=True)
data = input_path.read_bytes()
# Write SHA-256 of the complete file
sha256 = hashlib.sha256(data).hexdigest()
(output_dir / f"{input_path.stem}.sha256").write_text(
f"{sha256} {input_path.name}\n"
)
# Split into chunks
parts = []
for i in range(0, len(data), chunk_size):
part_name = f"{input_path.stem}.part{len(parts):02d}{input_path.suffix}"
part_path = output_dir / part_name
part_path.write_bytes(data[i:i + chunk_size])
parts.append(part_name)
# Write manifest
(output_dir / f"{input_path.stem}.manifest").write_text(
"\n".join(parts) + "\n"
)
print(f"Split into {len(parts)} parts, SHA-256: {sha256}")
```
### Phase 2: Download & Assemble in App
#### 2a. Backend Download Endpoint
Add to `backend/main.py`:
```python
@app.post("/backend/download-cuda")
async def download_cuda_backend():
"""Download the CUDA backend binary."""
# Returns immediately, runs download in background
task = asyncio.create_task(_download_cuda_binary())
task.add_done_callback(lambda t: logger.error(f"CUDA download failed: {t.exception()}") if t.exception() else None)
return {"status": "downloading"}
@app.get("/backend/cuda-status")
async def cuda_status():
"""Check if CUDA binary is available."""
cuda_path = _get_cuda_binary_path()
return {
"available": cuda_path is not None and cuda_path.exists(),
"active": _is_cuda_active(),
"download_progress": progress_manager.get_progress("cuda-backend"),
}
```
#### 2b. Download + Assemble + Verify Logic
New file: `backend/cuda_download.py`
Core logic:
```python
import hashlib
from pathlib import Path
from backend.config import get_data_dir
from backend.utils.progress import get_progress_manager
CUDA_DOWNLOAD_URL = "https://downloads.voicebox.sh/cuda/{version}/voicebox-server-cuda{ext}"
CUDA_CHECKSUMS = {
# Populated per release
"0.2.0-windows": "sha256:abc123...",
"0.2.0-linux": "sha256:def456...",
}
def get_cuda_binary_dir() -> Path:
"""Where CUDA binaries live. Inside the app's data directory."""
return get_data_dir() / "backends"
def get_cuda_binary_path() -> Path | None:
"""Return path to CUDA binary if it exists and is verified."""
d = get_cuda_binary_dir()
for name in ["voicebox-server-cuda.exe", "voicebox-server-cuda"]:
p = d / name
if p.exists():
return p
return None
async def download_cuda_binary(version: str):
"""Download, assemble (if split), and verify the CUDA binary."""
progress = get_progress_manager()
dest_dir = get_cuda_binary_dir()
dest_dir.mkdir(parents=True, exist_ok=True)
ext = ".exe" if sys.platform == "win32" else ""
url = CUDA_DOWNLOAD_URL.format(version=version, ext=ext)
# Download with progress tracking
temp_path = dest_dir / f"voicebox-server-cuda{ext}.download"
async with httpx.AsyncClient(follow_redirects=True) as client:
async with client.stream("GET", url) as response:
total = int(response.headers.get("content-length", 0))
downloaded = 0
with open(temp_path, "wb") as f:
async for chunk in response.aiter_bytes(chunk_size=1024 * 1024):
f.write(chunk)
downloaded += len(chunk)
progress.update("cuda-backend", downloaded, total)
# Verify checksum
sha256 = hashlib.sha256(temp_path.read_bytes()).hexdigest()
expected = CUDA_CHECKSUMS.get(f"{version}-{sys.platform}")
if expected and not expected.endswith(sha256):
temp_path.unlink()
raise ValueError(f"Checksum mismatch: expected {expected}, got sha256:{sha256}")
# Atomic move into place
final_path = dest_dir / f"voicebox-server-cuda{ext}"
temp_path.rename(final_path)
# Make executable on Unix
if sys.platform != "win32":
final_path.chmod(0o755)
progress.complete("cuda-backend")
```
Key points:
- Downloads to a `.download` temp file, verifies checksum, then atomically renames. No partial binaries left on crash.
- Progress tracked via the existing `ProgressManager` so the frontend SSE system works unchanged.
- CUDA binary lives in the **app data directory** (`data/backends/`), not alongside the app bundle. This avoids code-signing issues on macOS (though CUDA isn't relevant on macOS) and survives app updates.
#### 2c. Reassembly from Split Parts (GitHub Releases Fallback)
If the R2 download fails, fall back to downloading split parts from GitHub Releases:
```python
async def download_cuda_from_github(version: str):
"""Fallback: download split parts from GitHub Releases, reassemble."""
base_url = f"https://github.com/jamiepine/voicebox/releases/download/v{version}"
# Get manifest
manifest_url = f"{base_url}/voicebox-server-cuda.manifest"
async with httpx.AsyncClient(follow_redirects=True) as client:
manifest = (await client.get(manifest_url)).text
parts = [p.strip() for p in manifest.strip().splitlines()]
# Download checksum
sha256_url = f"{base_url}/voicebox-server-cuda.sha256"
expected_sha = (await client.get(sha256_url)).text.split()[0]
# Download parts
dest_dir = get_cuda_binary_dir()
dest_dir.mkdir(parents=True, exist_ok=True)
temp_path = dest_dir / "voicebox-server-cuda.exe.download"
total_downloaded = 0
with open(temp_path, "wb") as f:
for i, part_name in enumerate(parts):
part_url = f"{base_url}/{part_name}"
async with client.stream("GET", part_url) as response:
async for chunk in response.aiter_bytes(chunk_size=1024 * 1024):
f.write(chunk)
total_downloaded += len(chunk)
get_progress_manager().update(
"cuda-backend", total_downloaded, None,
message=f"Downloading part {i+1}/{len(parts)}"
)
# Verify reassembled file
sha256 = hashlib.sha256(temp_path.read_bytes()).hexdigest()
if sha256 != expected_sha:
temp_path.unlink()
raise ValueError(f"Checksum mismatch after reassembly")
final_path = dest_dir / "voicebox-server-cuda.exe"
temp_path.rename(final_path)
get_progress_manager().complete("cuda-backend")
```
### Phase 3: Backend Restart (The Swap)
This is the core of the feature: kill the CPU backend, launch the CUDA backend, frontend reconnects automatically.
#### 3a. New Tauri Command: `restart_server`
Add to `tauri/src-tauri/src/main.rs`:
```rust
#[command]
async fn restart_server(
app: tauri::AppHandle,
state: State<'_, ServerState>,
use_cuda: Option<bool>,
) -> Result<String, String> {
println!("restart_server: use_cuda={:?}", use_cuda);
// 1. Stop the current server
stop_server(state.clone()).await?;
// 2. Brief wait for port release
tokio::time::sleep(tokio::time::Duration::from_millis(500)).await;
// 3. Start with the appropriate binary
// The start_server logic needs to check for CUDA binary
start_server(app, state, None).await
}
```
#### 3b. Modify `start_server` to Prefer CUDA Binary
The existing `start_server` uses `app.shell().sidecar("voicebox-server")` which resolves via Tauri's `externalBin` config. For the CUDA binary (which lives in the data directory, not the app bundle), we need an alternative launch path.
Modify `start_server` in `main.rs`:
```rust
// After the existing sidecar logic, before spawning:
// Check for CUDA binary in data directory
let cuda_binary = data_dir.join("backends")
.join(if cfg!(windows) { "voicebox-server-cuda.exe" } else { "voicebox-server-cuda" });
let (mut rx, child) = if cuda_binary.exists() {
println!("Found CUDA backend binary at {:?}", cuda_binary);
// Launch CUDA binary directly (not as Tauri sidecar)
let mut cmd = app.shell().command(cuda_binary.to_str().unwrap());
cmd = cmd.args([
"--data-dir",
data_dir.to_str().ok_or("Invalid data dir path")?,
"--port",
&SERVER_PORT.to_string(),
]);
if remote.unwrap_or(false) {
cmd = cmd.args(["--host", "0.0.0.0"]);
}
cmd.spawn().map_err(|e| format!("Failed to spawn CUDA backend: {}", e))?
} else {
// Existing sidecar launch (CPU binary bundled with app)
sidecar.spawn().map_err(|e| format!("Failed to spawn: {}", e))?
};
```
Key decisions:
- CUDA binary is launched via `app.shell().command()` (arbitrary path), not `app.shell().sidecar()` (bundled path). Tauri's sidecar system only resolves binaries within the app bundle.
- The CUDA binary gets the same args (`--data-dir`, `--port`) as the CPU binary. It's the same `server.py` entry point.
- Preference: if CUDA binary exists, use it. Otherwise fall back to bundled CPU. No user configuration needed.
#### 3c. Frontend: Trigger Restart After Download
Add to the platform lifecycle interface (`app/src/platform/types.ts`):
```typescript
interface PlatformLifecycle {
startServer(remote?: boolean): Promise<string>;
stopServer(): Promise<void>;
restartServer(useCuda?: boolean): Promise<string>; // new
// ...
}
```
Implement in `tauri/src/platform/lifecycle.ts`:
```typescript
async restartServer(useCuda?: boolean): Promise<string> {
const result = await invoke<string>('restart_server', { useCuda });
this.onServerReady?.();
return result;
}
```
#### 3d. Frontend: GPU Settings UI
Add a section to the Server Settings page (or Model Management). Minimal UI:
```
┌─────────────────────────────────────────────┐
│ GPU Acceleration │
│ │
│ Status: CPU only (no CUDA backend) │
│ │
│ [Download CUDA Backend (2.4 GB)] │
│ │
│ Requires an NVIDIA GPU with 4+ GB VRAM. │
│ The app will restart its backend process │
│ after download. Your work is preserved. │
└─────────────────────────────────────────────┘
```
After download:
```
┌─────────────────────────────────────────────┐
│ GPU Acceleration │
│ │
│ Status: ✓ CUDA backend active (RTX 4090) │
│ │
│ [Switch to CPU] [Delete CUDA Backend] │
└─────────────────────────────────────────────┘
```
#### 3e. Frontend: Reconnection During Restart
The current health poll interval is 30 seconds — too slow for a restart UX. During a restart, temporarily increase polling:
```typescript
// In the component that triggers restart:
const restart = async () => {
setRestarting(true);
try {
await platform.lifecycle.restartServer(true);
} catch (e) {
// Frontend will show "reconnecting" state
}
// Aggressively poll until health check succeeds
const interval = setInterval(async () => {
try {
await apiClient.getHealth();
clearInterval(interval);
setRestarting(false);
queryClient.invalidateQueries(); // Refresh all data
} catch {}
}, 1000); // Poll every 1s during restart
// Safety timeout
setTimeout(() => clearInterval(interval), 30000);
};
```
### Phase 4: Auto-Detection on Startup
No user action needed on subsequent launches. The preference logic in `start_server` (Phase 3b) handles this:
1. App launches → `start_server` called
2. Check `data/backends/voicebox-server-cuda{.exe}`
3. If exists → launch CUDA binary
4. If not → launch bundled CPU binary
The user downloads CUDA once, and every future app launch (including after updates) uses it automatically. The CUDA binary lives in the data directory, not the app bundle, so app updates don't overwrite it.
### Phase 5: Handling Version Mismatches
When the app updates but the CUDA binary is from an older version, the API might be incompatible. Handle this by:
1. Add `--version` flag to `server.py`:
```python
parser.add_argument("--version", action="store_true")
# If invoked with --version, print version and exit
if args.version:
from backend import __version__
print(f"voicebox-server {__version__}")
sys.exit(0)
```
2. In `start_server` (Rust), before launching the CUDA binary:
```rust
// Quick version check
let version_output = std::process::Command::new(cuda_binary.to_str().unwrap())
.arg("--version")
.output();
match version_output {
Ok(output) => {
let version = String::from_utf8_lossy(&output.stdout);
let app_version = env!("CARGO_PKG_VERSION");
if !version.contains(app_version) {
println!("CUDA binary version mismatch (app: {}, cuda: {}), falling back to CPU",
app_version, version.trim());
// Fall through to CPU sidecar launch
}
}
Err(_) => {
println!("Failed to check CUDA binary version, falling back to CPU");
}
}
```
3. Frontend shows a notification: "Your GPU backend needs an update. [Download latest] or [Use CPU for now]"
## Files Changed
### New Files
| File | Purpose |
|------|---------|
| `backend/cuda_download.py` | Download, reassemble, verify CUDA binary |
| `scripts/split_binary.py` | Split binary into <2 GB chunks for GitHub Releases |
| `.github/workflows/build-cuda.yml` | CI: build + upload CUDA binary |
### Modified Files
| File | Change |
|------|--------|
| `tauri/src-tauri/src/main.rs` | Add `restart_server` command, modify `start_server` to check for CUDA binary in data dir |
| `backend/server.py` | Add `--version` flag |
| `backend/main.py` | Add `/backend/download-cuda`, `/backend/cuda-status`, `/backend/progress/cuda-backend` endpoints |
| `backend/build_binary.py` | Accept `--cuda` flag to build CUDA variant |
| `app/src/platform/types.ts` | Add `restartServer` to lifecycle interface |
| `tauri/src/platform/lifecycle.ts` | Implement `restartServer` |
| `app/src/components/ServerSettings/` | New GPU acceleration section |
| `.github/workflows/release.yml` | Trigger CUDA build workflow on tag |
### NOT Changed
| File | Why |
|------|-----|
| `backend/backends/__init__.py` | No changes to the TTSBackend singleton or factory. CUDA binary runs the same code. |
| `backend/backends/pytorch_backend.py` | Already detects CUDA at runtime (line 28-49). No changes needed. |
| `app/src/lib/api/client.ts` | API is identical between CPU and CUDA backends. |
| `app/src/lib/hooks/useGenerationForm.ts` | Generation flow is unchanged. |
## What This Doesn't Solve
- **Multi-model support** — This is purely about GPU acceleration. LuxTTS, Chatterbox, etc. need the in-process model registry, which is an independent workstream.
- **AMD GPU support** — DirectML/ROCm needs a different PyTorch build. Same pattern applies (another binary variant) but deferred.
- **Linux CUDA** — Same approach works, just another CI matrix entry. Can be added in the same release or shortly after.
- **Remote server mode** — Users who want to run TTS on a different machine still need the external provider architecture. Separate concern.
## What This DOES Solve
- **19 "GPU not detected" issues** — Users download the CUDA backend, restart, GPU works.
- **2 GB GitHub Release limit** — Binary splitting + R2 hosting.
- **Update burden** — App updates don't re-download the 2.4 GB CUDA binary. It persists in the data directory.
- **First-run experience** — App works immediately on CPU. GPU is an optional enhancement, not a setup blocker.
## Rollout Plan
1. Build and test CUDA binary locally on Windows with an NVIDIA GPU.
2. Set up R2 bucket at `downloads.voicebox.sh/cuda/`.
3. Ship the backend restart + download UI in v0.2.0.
4. Announce: "GPU acceleration is here — one click in Settings."
## Risks
| Risk | Mitigation |
|------|-----------|
| CUDA binary doesn't work on some GPU/driver combos | `/health` endpoint reports GPU info. Fallback to CPU if CUDA init fails. Clear error message. |
| Antivirus flags downloaded binary (Windows) | Code-sign the CUDA binary in CI. Document AV exceptions. |
| Data dir CUDA binary survives app uninstall | Document in uninstall notes. Not a real problem — it's just a file. |
| Version mismatch after app update | Version check on startup (Phase 5). Auto-fallback to CPU. Prompt to re-download. |
| R2 downtime | GitHub Releases split-binary fallback. |
| Download interrupted | Temp file with `.download` extension. Atomic rename on completion. Resume not implemented in v1 — restart download from scratch. |

Some files were not shown because too many files have changed in this diff Show More