mirror of
https://github.com/jamiepine/voicebox.git
synced 2026-09-26 13:45:16 -07:00
Compare commits
14
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
192979a762 | ||
|
|
e16cc42d53 | ||
|
|
f10e965003 | ||
|
|
a8968d4081 | ||
|
|
7c4afbe4df | ||
|
|
a180fcc56f | ||
|
|
1860b8dc92 | ||
|
|
1597937535 | ||
|
|
ac41a89359 | ||
|
|
2e6efa00a2 | ||
|
|
788a04f265 | ||
|
|
5cb54ee03c | ||
|
|
0922845101 | ||
|
|
64dd29d35a |
+25
-2
@@ -1,3 +1,26 @@
|
||||
node_modules
|
||||
.mintlify
|
||||
# deps
|
||||
/node_modules
|
||||
|
||||
# generated content
|
||||
.source
|
||||
|
||||
# test & build
|
||||
/coverage
|
||||
/.next/
|
||||
/out/
|
||||
/build
|
||||
*.tsbuildinfo
|
||||
|
||||
# misc
|
||||
.DS_Store
|
||||
*.pem
|
||||
/.pnp
|
||||
.pnp.js
|
||||
npm-debug.log*
|
||||
yarn-debug.log*
|
||||
yarn-error.log*
|
||||
|
||||
# others
|
||||
.env*.local
|
||||
.vercel
|
||||
next-env.d.ts
|
||||
@@ -1,192 +0,0 @@
|
||||
# Auto-Updater Documentation
|
||||
|
||||
Voicebox includes automatic updates powered by Tauri's updater plugin. This document explains how it works for both users and developers.
|
||||
|
||||
## 1. Generate Signing Keys
|
||||
|
||||
Run this command to generate your signing keypair:
|
||||
|
||||
```bash
|
||||
cd tauri && bun tauri signer generate -w ~/.tauri/voicebox.key
|
||||
```
|
||||
|
||||
This creates:
|
||||
- **Private key**: `~/.tauri/voicebox.key` (keep this secret!)
|
||||
- **Public key**: `~/.tauri/voicebox.key.pub`
|
||||
|
||||
## 2. Update Configuration
|
||||
|
||||
Copy the content from `~/.tauri/voicebox.key.pub` and replace the placeholder in `tauri/src-tauri/tauri.conf.json`:
|
||||
|
||||
```json
|
||||
{
|
||||
"plugins": {
|
||||
"updater": {
|
||||
"pubkey": "PASTE_PUBLIC_KEY_CONTENT_HERE",
|
||||
"endpoints": [
|
||||
"https://github.com/YOUR_USERNAME/voicebox/releases/latest/download/latest.json"
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Update the endpoint URL with your actual GitHub username/organization.
|
||||
|
||||
## 3. Building with Signatures
|
||||
|
||||
When building releases, set these environment variables:
|
||||
|
||||
**macOS/Linux:**
|
||||
```bash
|
||||
export TAURI_SIGNING_PRIVATE_KEY="$(cat ~/.tauri/voicebox.key)"
|
||||
export TAURI_SIGNING_PRIVATE_KEY_PASSWORD=""
|
||||
bun run build
|
||||
```
|
||||
|
||||
**Windows PowerShell:**
|
||||
```powershell
|
||||
$env:TAURI_SIGNING_PRIVATE_KEY = Get-Content ~/.tauri/voicebox.key -Raw
|
||||
$env:TAURI_SIGNING_PRIVATE_KEY_PASSWORD = ""
|
||||
bun run build
|
||||
```
|
||||
|
||||
## 4. GitHub Release Setup
|
||||
|
||||
When you create a GitHub release, the build process will generate:
|
||||
- Installers for each platform
|
||||
- `.sig` signature files
|
||||
- `latest.json` update manifest
|
||||
|
||||
### Manual Release Process
|
||||
|
||||
1. Build the app with signing keys set
|
||||
2. Create a new GitHub release
|
||||
3. Upload all files from `tauri/src-tauri/target/release/bundle/`
|
||||
4. Create `latest.json` in your release assets:
|
||||
|
||||
```json
|
||||
{
|
||||
"version": "0.2.0",
|
||||
"notes": "Bug fixes and improvements",
|
||||
"pub_date": "2026-01-25T12:00:00Z",
|
||||
"platforms": {
|
||||
"darwin-aarch64": {
|
||||
"signature": "CONTENT_FROM_.app.tar.gz.sig",
|
||||
"url": "https://github.com/YOUR_USERNAME/voicebox/releases/download/v0.2.0/voicebox_0.2.0_aarch64.dmg"
|
||||
},
|
||||
"darwin-x86_64": {
|
||||
"signature": "CONTENT_FROM_.app.tar.gz.sig",
|
||||
"url": "https://github.com/YOUR_USERNAME/voicebox/releases/download/v0.2.0/voicebox_0.2.0_x64.dmg"
|
||||
},
|
||||
"linux-x86_64": {
|
||||
"signature": "CONTENT_FROM_.AppImage.sig",
|
||||
"url": "https://github.com/YOUR_USERNAME/voicebox/releases/download/v0.2.0/voicebox_0.2.0_amd64.AppImage"
|
||||
},
|
||||
"windows-x86_64": {
|
||||
"signature": "CONTENT_FROM_.msi.sig",
|
||||
"url": "https://github.com/YOUR_USERNAME/voicebox/releases/download/v0.2.0/voicebox_0.2.0_x64_en-US.msi"
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Automated GitHub Actions (Recommended)
|
||||
|
||||
Create `.github/workflows/release.yml`:
|
||||
|
||||
```yaml
|
||||
name: Release
|
||||
|
||||
on:
|
||||
push:
|
||||
tags:
|
||||
- 'v*'
|
||||
|
||||
jobs:
|
||||
release:
|
||||
strategy:
|
||||
matrix:
|
||||
platform: [macos-latest, ubuntu-22.04, windows-latest]
|
||||
|
||||
runs-on: ${{ matrix.platform }}
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- name: Setup Bun
|
||||
uses: oven-sh/setup-bun@v1
|
||||
|
||||
- name: Install Rust
|
||||
uses: dtolnay/rust-toolchain@stable
|
||||
|
||||
- name: Install dependencies (Ubuntu)
|
||||
if: matrix.platform == 'ubuntu-22.04'
|
||||
run: |
|
||||
sudo apt-get update
|
||||
sudo apt-get install -y libwebkit2gtk-4.1-dev libappindicator3-dev librsvg2-dev patchelf
|
||||
|
||||
- name: Install dependencies
|
||||
run: bun install
|
||||
|
||||
- name: Build
|
||||
env:
|
||||
TAURI_SIGNING_PRIVATE_KEY: ${{ secrets.TAURI_SIGNING_PRIVATE_KEY }}
|
||||
TAURI_SIGNING_PRIVATE_KEY_PASSWORD: ${{ secrets.TAURI_SIGNING_PRIVATE_KEY_PASSWORD }}
|
||||
run: bun run build
|
||||
|
||||
- name: Upload Release
|
||||
uses: softprops/action-gh-release@v1
|
||||
with:
|
||||
files: tauri/src-tauri/target/release/bundle/**/*
|
||||
env:
|
||||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
```
|
||||
|
||||
Add your private key to GitHub secrets:
|
||||
- Go to Settings → Secrets and variables → Actions
|
||||
- Add `TAURI_SIGNING_PRIVATE_KEY` with the content of `~/.tauri/voicebox.key`
|
||||
- Add `TAURI_SIGNING_PRIVATE_KEY_PASSWORD` (empty string if no password)
|
||||
|
||||
## Frontend Integration
|
||||
|
||||
The frontend integration is complete with automatic update notifications and manual update checks:
|
||||
|
||||
- **Update Notification Banner** - Appears automatically when updates are available
|
||||
- **Settings Panel** - Manual "Check for Updates" button in Settings tab
|
||||
- **Update Hook** - React hook handles all update operations
|
||||
|
||||
See `docs/AUTOUPDATER_QUICKSTART.md` for a quick setup guide.
|
||||
|
||||
## Security Notes
|
||||
|
||||
- Never commit your private key to version control
|
||||
- Store private keys securely (use GitHub secrets for CI/CD)
|
||||
- The public key in `tauri.conf.json` is safe to commit
|
||||
- Updates are cryptographically verified before installation
|
||||
- HTTP endpoints are blocked by default (HTTPS only)
|
||||
|
||||
## Testing Updates
|
||||
|
||||
1. Build version 0.1.0 and install it
|
||||
2. Update version in `tauri.conf.json` to 0.2.0
|
||||
3. Build version 0.2.0 with signatures
|
||||
4. Create a local server or GitHub release with `latest.json`
|
||||
5. Run version 0.1.0 and trigger update check
|
||||
6. Verify update downloads and installs correctly
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
**"Invalid signature" error:**
|
||||
- Verify public key matches the private key used to sign
|
||||
- Ensure signature files (.sig) are uploaded correctly
|
||||
|
||||
**"No update available" when one exists:**
|
||||
- Check endpoint URL is correct
|
||||
- Verify `latest.json` format matches specification
|
||||
- Ensure version in latest.json is higher than current version
|
||||
|
||||
**Build fails with signing:**
|
||||
- Confirm environment variables are set correctly
|
||||
- Check private key file exists and is readable
|
||||
- Verify private key format (should start with `dW50cnVzdGVkIGNvbW1lbnQ6`)
|
||||
@@ -1,116 +0,0 @@
|
||||
# Autoupdater Quick Start
|
||||
|
||||
The Tauri v2 autoupdater has been fully configured and integrated. Follow these steps to activate it.
|
||||
|
||||
## What's Already Done
|
||||
|
||||
✅ Rust plugin installed and initialized
|
||||
✅ Tauri configuration set up with updater settings
|
||||
✅ Permissions granted for update operations
|
||||
✅ GitHub Actions workflow updated with signing support
|
||||
✅ Frontend components created and integrated
|
||||
✅ Update notifications on app startup
|
||||
✅ Manual update check in Settings tab
|
||||
|
||||
## Required Steps (5 minutes)
|
||||
|
||||
### 1. Generate Signing Keys
|
||||
|
||||
```bash
|
||||
bun run generate:keys
|
||||
```
|
||||
|
||||
This creates:
|
||||
- Private key: `~/.tauri/voicebox.key` (keep secret!)
|
||||
- Public key: `~/.tauri/voicebox.key.pub` (safe to share)
|
||||
|
||||
### 2. Update Tauri Config
|
||||
|
||||
Open `tauri/src-tauri/tauri.conf.json` and:
|
||||
|
||||
1. Replace `"REPLACE_WITH_YOUR_PUBLIC_KEY"` with the content from `~/.tauri/voicebox.key.pub`
|
||||
2. Update the endpoint URL with your GitHub username:
|
||||
```json
|
||||
"endpoints": [
|
||||
"https://github.com/YOUR_USERNAME/voicebox/releases/latest/download/latest.json"
|
||||
]
|
||||
```
|
||||
|
||||
### 3. Add GitHub Secrets
|
||||
|
||||
Go to your repo Settings → Secrets and variables → Actions:
|
||||
|
||||
1. Add `TAURI_SIGNING_PRIVATE_KEY`:
|
||||
```bash
|
||||
cat ~/.tauri/voicebox.key
|
||||
```
|
||||
Copy the entire output and paste as the secret value
|
||||
|
||||
2. Add `TAURI_SIGNING_PRIVATE_KEY_PASSWORD`:
|
||||
Leave empty (or add your password if you set one)
|
||||
|
||||
### 4. Test the Setup
|
||||
|
||||
To test locally before creating a release:
|
||||
|
||||
```bash
|
||||
bun run build:release
|
||||
```
|
||||
|
||||
This will verify your keys are set up correctly.
|
||||
|
||||
## How It Works
|
||||
|
||||
### For Users
|
||||
1. App checks for updates on startup (only in Tauri builds)
|
||||
2. If an update is available, a banner appears at the top
|
||||
3. Users can click "Install Now" to download and install
|
||||
4. App restarts automatically after installation
|
||||
|
||||
### For Developers
|
||||
1. Create a new git tag: `git tag v0.2.0 && git push --tags`
|
||||
2. GitHub Actions builds signed releases for all platforms
|
||||
3. Uploads installers and generates `latest.json` manifest
|
||||
4. Users running older versions will be notified automatically
|
||||
|
||||
## UI Components
|
||||
|
||||
### Update Notification Banner
|
||||
- Shows at top of app when update is available
|
||||
- Appears automatically on startup
|
||||
- Displays download/install progress
|
||||
|
||||
### Settings Panel
|
||||
- Located in Settings tab
|
||||
- Shows current version
|
||||
- Manual "Check for Updates" button
|
||||
- Update status and progress
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
**"Public key not configured"**
|
||||
- Make sure you copied the entire content from `voicebox.key.pub`
|
||||
- The key should start with `dW50cnVzdGVkIGNvbW1lbnQ6`
|
||||
|
||||
**"Failed to check for updates"**
|
||||
- Endpoint URL might be incorrect
|
||||
- No releases published yet (expected for first setup)
|
||||
|
||||
**Build fails with signing error**
|
||||
- Check that GitHub secrets are set correctly
|
||||
- Verify private key file exists at `~/.tauri/voicebox.key`
|
||||
|
||||
## Next Release Workflow
|
||||
|
||||
1. Update version in `tauri/src-tauri/tauri.conf.json`
|
||||
2. Commit changes
|
||||
3. Create and push tag: `git tag v0.2.0 && git push --tags`
|
||||
4. GitHub Actions will automatically build and create a draft release
|
||||
5. Review the release and publish it
|
||||
6. Users will be notified of the update
|
||||
|
||||
## See Also
|
||||
|
||||
- Full documentation: `docs/AUTOUPDATER.md`
|
||||
- Build script: `scripts/prepare-release.sh`
|
||||
- GitHub workflow: `.github/workflows/release.yml`
|
||||
@@ -0,0 +1,87 @@
|
||||
# Documentation Migration: Mintlify → Fumadocs
|
||||
|
||||
This document summarizes the migration of documentation from `/docs` (Mintlify) to `/docs2` (Fumadocs).
|
||||
|
||||
## What Was Done
|
||||
|
||||
### 1. Files Copied
|
||||
- ✅ All 29 MDX files from `/docs` folders (overview, api, developer, plans)
|
||||
- ✅ All 4 root-level markdown files (AUTOUPDATER.md, AUTOUPDATER_QUICKSTART.md, TROUBLESHOOTING.md, README.md)
|
||||
- ✅ All images (3 webp files) → `public/images/`
|
||||
- ✅ All logo files (2 png files) → `public/logo/`
|
||||
|
||||
### 2. Component Migration
|
||||
Created compatibility layer in `components/mintlify-compat.tsx` that maps Mintlify components to Fumadocs equivalents:
|
||||
|
||||
- `<Frame>` → Simple div wrapper (images are zoomable by default in Fumadocs)
|
||||
- `<CardGroup>` → `<Cards>` (Fumadocs component)
|
||||
- `<Card>` → `<Card>` (with icon string → Lucide icon mapping)
|
||||
- `<Steps>` / `<Step>` → Direct mapping to Fumadocs components
|
||||
- `<Tip>`, `<Note>`, `<Info>` → `<Callout type="info">`
|
||||
- `<Warning>` → `<Callout type="warn">`
|
||||
- `<Danger>` → `<Callout type="error">`
|
||||
- `<AccordionGroup>` / `<Accordion>` → HTML `<details>` / `<summary>` elements
|
||||
|
||||
### 3. Navigation Structure
|
||||
Created `meta.json` files for each folder:
|
||||
- `content/docs/meta.json` - Root documentation
|
||||
- `content/docs/overview/meta.json` - Overview pages
|
||||
- `content/docs/api/meta.json` - API reference
|
||||
- `content/docs/developer/meta.json` - Developer docs
|
||||
- `content/docs/plans/meta.json` - Plans/roadmap
|
||||
|
||||
### 4. Link Fixes
|
||||
- Fixed incorrect `/guides/...` paths → `/overview/...`
|
||||
- All internal links now use correct paths
|
||||
|
||||
### 5. Branding
|
||||
- Updated `lib/layout.shared.tsx` to use "Voicebox" as the nav title
|
||||
|
||||
## File Structure
|
||||
|
||||
```
|
||||
docs2/
|
||||
├── components/
|
||||
│ └── mintlify-compat.tsx # Mintlify → Fumadocs component mappings
|
||||
├── content/docs/
|
||||
│ ├── meta.json # Root navigation
|
||||
│ ├── overview/ # 12 MDX files
|
||||
│ ├── api/ # 5 MDX files
|
||||
│ ├── developer/ # 12 MDX files
|
||||
│ ├── plans/ # 4 MD files
|
||||
│ └── *.md # 4 root markdown files
|
||||
├── public/
|
||||
│ ├── images/ # 3 webp files
|
||||
│ └── logo/ # 2 png files
|
||||
└── mdx-components.tsx # MDX component configuration
|
||||
```
|
||||
|
||||
## Icon Mapping
|
||||
|
||||
The following icon strings are mapped to Lucide icons:
|
||||
- `microphone` → Mic
|
||||
- `film` → Film
|
||||
- `code` → Code
|
||||
- `shield` → Shield
|
||||
- `download` → Download
|
||||
- `rocket` → Rocket
|
||||
- `apple` → Apple
|
||||
- `windows` → Windows
|
||||
- `server` → Server
|
||||
- `user` → User
|
||||
- `waveform` → Waveform
|
||||
|
||||
## Next Steps
|
||||
|
||||
1. **Test the build**: Run `npm run build` (requires Node.js >= 20.9.0)
|
||||
2. **Start dev server**: Run `npm run dev` to preview
|
||||
3. **Customize styling**: Update `app/global.css` if needed
|
||||
4. **Add more icons**: Extend `iconMap` in `mintlify-compat.tsx` as needed
|
||||
5. **Review navigation**: Adjust `meta.json` files to customize page order
|
||||
|
||||
## Notes
|
||||
|
||||
- Image paths (`/images/...`) work as-is since Next.js serves from `public/`
|
||||
- All Mintlify components are now compatible with Fumadocs
|
||||
- Navigation structure follows Fumadocs conventions
|
||||
- No breaking changes to content - all MDX files work with compatibility layer
|
||||
+29
-48
@@ -1,64 +1,45 @@
|
||||
# Voicebox Documentation
|
||||
# fumadocs-ui-template
|
||||
|
||||
This directory contains the documentation for Voicebox, built with [Mintlify](https://mintlify.com).
|
||||
This is a Next.js application generated with
|
||||
[Create Fumadocs](https://github.com/fuma-nama/fumadocs).
|
||||
|
||||
## Development
|
||||
|
||||
### Prerequisites
|
||||
|
||||
Install Mintlify globally using bun:
|
||||
Run development server:
|
||||
|
||||
```bash
|
||||
bun add -g mintlify
|
||||
npm run dev
|
||||
# or
|
||||
pnpm dev
|
||||
# or
|
||||
yarn dev
|
||||
```
|
||||
|
||||
Or use the helper script:
|
||||
Open http://localhost:3000 with your browser to see the result.
|
||||
|
||||
```bash
|
||||
bun run install:mintlify
|
||||
```
|
||||
## Explore
|
||||
|
||||
### Running Locally
|
||||
In the project, you can see:
|
||||
|
||||
```bash
|
||||
bun run dev
|
||||
```
|
||||
- `lib/source.ts`: Code for content source adapter, [`loader()`](https://fumadocs.dev/docs/headless/source-api) provides the interface to access your content.
|
||||
- `lib/layout.shared.tsx`: Shared options for layouts, optional but preferred to keep.
|
||||
|
||||
This will start the Mintlify dev server.
|
||||
| Route | Description |
|
||||
| ------------------------- | ------------------------------------------------------ |
|
||||
| `app/(home)` | The route group for your landing page and other pages. |
|
||||
| `app/docs` | The documentation layout and pages. |
|
||||
| `app/api/search/route.ts` | The Route Handler for search. |
|
||||
|
||||
The docs will be available at `http://localhost:3000`
|
||||
### Fumadocs MDX
|
||||
|
||||
### Structure
|
||||
A `source.config.ts` config file has been included, you can customise different options like frontmatter schema.
|
||||
|
||||
```
|
||||
docs/
|
||||
├── mint.json # Mintlify configuration
|
||||
├── custom.css # Custom styles
|
||||
├── overview/ # Getting started & feature docs
|
||||
├── guides/ # User guides
|
||||
├── api/ # API reference
|
||||
├── development/ # Developer documentation
|
||||
├── logo/ # Logo assets
|
||||
└── public/ # Static assets
|
||||
```
|
||||
Read the [Introduction](https://fumadocs.dev/docs/mdx) for further details.
|
||||
|
||||
### Writing Docs
|
||||
## Learn More
|
||||
|
||||
- Use `.mdx` files for all documentation pages
|
||||
- Follow the existing structure in `mint.json` for navigation
|
||||
- Use Mintlify components for enhanced formatting (Card, CardGroup, Accordion, etc.)
|
||||
- Reference the [Mintlify documentation](https://mintlify.com/docs) for available components
|
||||
To learn more about Next.js and Fumadocs, take a look at the following
|
||||
resources:
|
||||
|
||||
## Deployment
|
||||
|
||||
Docs are automatically deployed when changes are pushed to the main branch.
|
||||
|
||||
To manually deploy:
|
||||
|
||||
```bash
|
||||
mintlify deploy
|
||||
```
|
||||
|
||||
## Contributing
|
||||
|
||||
See [CONTRIBUTING.md](../CONTRIBUTING.md) for contribution guidelines.
|
||||
- [Next.js Documentation](https://nextjs.org/docs) - learn about Next.js
|
||||
features and API.
|
||||
- [Learn Next.js](https://nextjs.org/learn) - an interactive Next.js tutorial.
|
||||
- [Fumadocs](https://fumadocs.dev) - learn about Fumadocs
|
||||
|
||||
@@ -1,55 +0,0 @@
|
||||
---
|
||||
title: "Authentication"
|
||||
description: "API authentication and security"
|
||||
---
|
||||
|
||||
## Current Status
|
||||
|
||||
<Warning>
|
||||
Authentication is not currently implemented in Voicebox. The API is intended for local use only.
|
||||
</Warning>
|
||||
|
||||
## Local Usage
|
||||
|
||||
For local development and usage:
|
||||
- API runs on `localhost:17493`
|
||||
- No authentication required
|
||||
- Access restricted to local machine
|
||||
|
||||
## Future Implementation
|
||||
|
||||
Authentication will be added in a future release for:
|
||||
- Remote deployments
|
||||
- Multi-user access
|
||||
- Production environments
|
||||
|
||||
Planned authentication methods:
|
||||
- API keys
|
||||
- OAuth 2.0
|
||||
- JWT tokens
|
||||
|
||||
## Security Best Practices
|
||||
|
||||
Until authentication is implemented:
|
||||
|
||||
<CardGroup cols={2}>
|
||||
<Card title="Use VPN" icon="shield">
|
||||
Use WireGuard or Tailscale for remote access
|
||||
</Card>
|
||||
<Card title="Reverse Proxy" icon="server">
|
||||
Run behind nginx with basic auth
|
||||
</Card>
|
||||
<Card title="Firewall" icon="fire">
|
||||
Restrict access to trusted IPs only
|
||||
</Card>
|
||||
<Card title="Local Only" icon="laptop">
|
||||
Don't expose to public internet
|
||||
</Card>
|
||||
</CardGroup>
|
||||
|
||||
## Coming Soon
|
||||
|
||||
- API key management
|
||||
- User accounts
|
||||
- Rate limiting
|
||||
- Access control
|
||||
@@ -1,119 +0,0 @@
|
||||
---
|
||||
title: "Generation API"
|
||||
description: "Generate speech from text"
|
||||
---
|
||||
|
||||
## Generate Speech
|
||||
|
||||
```http
|
||||
POST /generate
|
||||
```
|
||||
|
||||
**Request:**
|
||||
```json
|
||||
{
|
||||
"text": "Hello world",
|
||||
"profile_id": "abc123",
|
||||
"language": "en"
|
||||
}
|
||||
```
|
||||
|
||||
**Response:**
|
||||
```json
|
||||
{
|
||||
"id": "gen123",
|
||||
"text": "Hello world",
|
||||
"profile_id": "abc123",
|
||||
"language": "en",
|
||||
"audio_url": "/audio/gen123.wav",
|
||||
"duration": 2.3,
|
||||
"created_at": "2024-01-29T12:00:00Z"
|
||||
}
|
||||
```
|
||||
|
||||
## List History
|
||||
|
||||
```http
|
||||
GET /history
|
||||
```
|
||||
|
||||
**Query Parameters:**
|
||||
- `profile_id` (optional) - Filter by voice profile
|
||||
- `limit` (optional) - Number of results (default: 50)
|
||||
- `offset` (optional) - Pagination offset
|
||||
|
||||
**Response:**
|
||||
```json
|
||||
{
|
||||
"generations": [
|
||||
{
|
||||
"id": "gen123",
|
||||
"text": "Hello world",
|
||||
"profile_id": "abc123",
|
||||
"duration": 2.3,
|
||||
"created_at": "2024-01-29T12:00:00Z"
|
||||
}
|
||||
],
|
||||
"total": 100
|
||||
}
|
||||
```
|
||||
|
||||
## Get Generation
|
||||
|
||||
```http
|
||||
GET /history/{id}
|
||||
```
|
||||
|
||||
**Response:**
|
||||
```json
|
||||
{
|
||||
"id": "gen123",
|
||||
"text": "Hello world",
|
||||
"profile_id": "abc123",
|
||||
"language": "en",
|
||||
"audio_url": "/audio/gen123.wav",
|
||||
"duration": 2.3,
|
||||
"created_at": "2024-01-29T12:00:00Z"
|
||||
}
|
||||
```
|
||||
|
||||
## Delete Generation
|
||||
|
||||
```http
|
||||
DELETE /history/{id}
|
||||
```
|
||||
|
||||
**Response:**
|
||||
```json
|
||||
{
|
||||
"success": true
|
||||
}
|
||||
```
|
||||
|
||||
## TypeScript Example
|
||||
|
||||
```typescript
|
||||
import { VoiceboxClient } from '@/lib/api'
|
||||
|
||||
const client = new VoiceboxClient({
|
||||
baseUrl: 'http://localhost:17493'
|
||||
})
|
||||
|
||||
// Generate speech
|
||||
const generation = await client.generate({
|
||||
text: 'Hello world',
|
||||
profile_id: 'abc123',
|
||||
language: 'en'
|
||||
})
|
||||
|
||||
// Get audio URL
|
||||
const audioUrl = generation.audio_url
|
||||
|
||||
// List history
|
||||
const history = await client.listHistory({
|
||||
profile_id: 'abc123',
|
||||
limit: 20
|
||||
})
|
||||
```
|
||||
|
||||
For full API documentation, visit `http://localhost:17493/docs` when the server is running.
|
||||
@@ -1,219 +0,0 @@
|
||||
---
|
||||
title: "API Overview"
|
||||
description: "Integrate voice synthesis into your applications with the Voicebox REST API"
|
||||
---
|
||||
|
||||
## Introduction
|
||||
|
||||
Voicebox exposes a full REST API that allows you to integrate voice synthesis into your own applications. The API runs on `http://localhost:17493` by default.
|
||||
|
||||
<Card title="Interactive API Docs" icon="book" href="http://localhost:17493/docs">
|
||||
When Voicebox is running, visit the auto-generated API documentation at `http://localhost:17493/docs`
|
||||
</Card>
|
||||
|
||||
## Base URL
|
||||
|
||||
```
|
||||
http://localhost:17493
|
||||
```
|
||||
|
||||
For remote deployments, replace `localhost` with your server's IP or hostname.
|
||||
|
||||
## Authentication
|
||||
|
||||
<Note>
|
||||
Currently, the API does not require authentication for local development. Authentication will be added in a future release for production deployments.
|
||||
</Note>
|
||||
|
||||
## Quick Example
|
||||
|
||||
Here's a simple example of generating speech:
|
||||
|
||||
```bash
|
||||
# Generate speech
|
||||
curl -X POST http://localhost:17493/generate \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"text": "Hello world",
|
||||
"profile_id": "abc123",
|
||||
"language": "en"
|
||||
}'
|
||||
```
|
||||
|
||||
## API Endpoints
|
||||
|
||||
The Voicebox API is organized into several categories:
|
||||
|
||||
<CardGroup cols={2}>
|
||||
<Card title="Voice Profiles" icon="user" href="/api/voice-profiles">
|
||||
Create, list, update, and delete voice profiles
|
||||
</Card>
|
||||
<Card title="Generation" icon="waveform" href="/api/generation">
|
||||
Generate speech from text using voice profiles
|
||||
</Card>
|
||||
<Card title="Recordings" icon="microphone" href="/api/recordings">
|
||||
Record and transcribe audio
|
||||
</Card>
|
||||
<Card title="Stories" icon="film">
|
||||
Create and manage multi-voice stories (coming soon)
|
||||
</Card>
|
||||
</CardGroup>
|
||||
|
||||
## Core Endpoints
|
||||
|
||||
### Voice Profiles
|
||||
|
||||
```http
|
||||
GET /profiles # List all profiles
|
||||
POST /profiles # Create a new profile
|
||||
GET /profiles/{id} # Get profile details
|
||||
PUT /profiles/{id} # Update a profile
|
||||
DELETE /profiles/{id} # Delete a profile
|
||||
POST /profiles/{id}/samples # Add voice sample
|
||||
```
|
||||
|
||||
### Generation
|
||||
|
||||
```http
|
||||
POST /generate # Generate speech
|
||||
GET /history # List generation history
|
||||
GET /history/{id} # Get generation details
|
||||
DELETE /history/{id} # Delete from history
|
||||
```
|
||||
|
||||
### Recordings
|
||||
|
||||
```http
|
||||
POST /recordings # Start recording
|
||||
POST /recordings/stop # Stop recording
|
||||
POST /transcribe # Transcribe audio
|
||||
```
|
||||
|
||||
## Response Format
|
||||
|
||||
All API responses follow a consistent JSON format:
|
||||
|
||||
```json
|
||||
{
|
||||
"success": true,
|
||||
"data": {
|
||||
// Response data
|
||||
},
|
||||
"error": null
|
||||
}
|
||||
```
|
||||
|
||||
Error responses:
|
||||
|
||||
```json
|
||||
{
|
||||
"success": false,
|
||||
"data": null,
|
||||
"error": {
|
||||
"message": "Error description",
|
||||
"code": "ERROR_CODE"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Data Models
|
||||
|
||||
### Voice Profile
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "abc123",
|
||||
"name": "John Smith",
|
||||
"language": "en",
|
||||
"description": "Professional narrator voice",
|
||||
"created_at": "2024-01-29T12:00:00Z",
|
||||
"samples": [
|
||||
{
|
||||
"id": "sample123",
|
||||
"audio_path": "/path/to/sample.wav",
|
||||
"duration": 15.5
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
### Generation
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "gen123",
|
||||
"text": "Hello world",
|
||||
"profile_id": "abc123",
|
||||
"language": "en",
|
||||
"audio_path": "/path/to/output.wav",
|
||||
"duration": 2.3,
|
||||
"created_at": "2024-01-29T12:00:00Z"
|
||||
}
|
||||
```
|
||||
|
||||
## TypeScript Client
|
||||
|
||||
Voicebox provides an auto-generated TypeScript client with full type safety:
|
||||
|
||||
```typescript
|
||||
import { VoiceboxClient } from '@/lib/api'
|
||||
|
||||
const client = new VoiceboxClient({
|
||||
baseUrl: 'http://localhost:17493'
|
||||
})
|
||||
|
||||
// Create a profile
|
||||
const profile = await client.createProfile({
|
||||
name: 'John Smith',
|
||||
language: 'en'
|
||||
})
|
||||
|
||||
// Generate speech
|
||||
const generation = await client.generate({
|
||||
text: 'Hello world',
|
||||
profile_id: profile.id,
|
||||
language: 'en'
|
||||
})
|
||||
```
|
||||
|
||||
The client is automatically generated from the OpenAPI schema. See [Development Setup](/development/setup#generate-openapi-client) for details.
|
||||
|
||||
## Rate Limiting
|
||||
|
||||
<Info>
|
||||
Currently, there are no rate limits for local usage. Rate limiting will be added in a future release for production deployments.
|
||||
</Info>
|
||||
|
||||
## WebSocket Support
|
||||
|
||||
<Note>
|
||||
Real-time streaming generation via WebSockets is planned for a future release.
|
||||
</Note>
|
||||
|
||||
## Use Cases
|
||||
|
||||
<CardGroup cols={2}>
|
||||
<Card title="Game Development" icon="gamepad">
|
||||
Generate dynamic dialogue for NPCs and characters
|
||||
</Card>
|
||||
<Card title="Content Creation" icon="video">
|
||||
Automate voiceovers for videos and podcasts
|
||||
</Card>
|
||||
<Card title="Accessibility" icon="universal-access">
|
||||
Build text-to-speech tools for visually impaired users
|
||||
</Card>
|
||||
<Card title="Voice Assistants" icon="robot">
|
||||
Create custom voice interfaces
|
||||
</Card>
|
||||
</CardGroup>
|
||||
|
||||
## Next Steps
|
||||
|
||||
<CardGroup cols={2}>
|
||||
<Card title="Voice Profiles API" icon="user" href="/api/voice-profiles">
|
||||
Learn how to manage voice profiles
|
||||
</Card>
|
||||
<Card title="Generation API" icon="waveform" href="/api/generation">
|
||||
Generate speech from text
|
||||
</Card>
|
||||
</CardGroup>
|
||||
@@ -1,95 +0,0 @@
|
||||
---
|
||||
title: "Recordings API"
|
||||
description: "Record and transcribe audio"
|
||||
---
|
||||
|
||||
## Start Recording
|
||||
|
||||
```http
|
||||
POST /recordings/start
|
||||
```
|
||||
|
||||
**Request:**
|
||||
```json
|
||||
{
|
||||
"source": "microphone"
|
||||
}
|
||||
```
|
||||
|
||||
**Response:**
|
||||
```json
|
||||
{
|
||||
"recording_id": "rec123",
|
||||
"status": "recording"
|
||||
}
|
||||
```
|
||||
|
||||
## Stop Recording
|
||||
|
||||
```http
|
||||
POST /recordings/stop
|
||||
```
|
||||
|
||||
**Request:**
|
||||
```json
|
||||
{
|
||||
"recording_id": "rec123"
|
||||
}
|
||||
```
|
||||
|
||||
**Response:**
|
||||
```json
|
||||
{
|
||||
"recording_id": "rec123",
|
||||
"audio_url": "/audio/rec123.wav",
|
||||
"duration": 15.5
|
||||
}
|
||||
```
|
||||
|
||||
## Transcribe Audio
|
||||
|
||||
```http
|
||||
POST /transcribe
|
||||
```
|
||||
|
||||
**Request:** (multipart/form-data)
|
||||
```
|
||||
audio: <file>
|
||||
language: "en" (optional)
|
||||
```
|
||||
|
||||
**Response:**
|
||||
```json
|
||||
{
|
||||
"text": "Transcribed speech text here",
|
||||
"language": "en",
|
||||
"duration": 15.5,
|
||||
"confidence": 0.95
|
||||
}
|
||||
```
|
||||
|
||||
## TypeScript Example
|
||||
|
||||
```typescript
|
||||
import { VoiceboxClient } from '@/lib/api'
|
||||
|
||||
const client = new VoiceboxClient({
|
||||
baseUrl: 'http://localhost:17493'
|
||||
})
|
||||
|
||||
// Start recording
|
||||
const recording = await client.startRecording({
|
||||
source: 'microphone'
|
||||
})
|
||||
|
||||
// ... record audio ...
|
||||
|
||||
// Stop recording
|
||||
const result = await client.stopRecording(recording.id)
|
||||
|
||||
// Transcribe
|
||||
const transcription = await client.transcribe(audioFile, 'en')
|
||||
console.log(transcription.text)
|
||||
```
|
||||
|
||||
For full API documentation, visit `http://localhost:17493/docs` when the server is running.
|
||||
@@ -1,149 +0,0 @@
|
||||
---
|
||||
title: "Voice Profiles API"
|
||||
description: "Manage voice profiles programmatically"
|
||||
---
|
||||
|
||||
## Endpoints
|
||||
|
||||
### List Profiles
|
||||
|
||||
```http
|
||||
GET /profiles
|
||||
```
|
||||
|
||||
**Response:**
|
||||
```json
|
||||
{
|
||||
"profiles": [
|
||||
{
|
||||
"id": "abc123",
|
||||
"name": "John Smith",
|
||||
"language": "en",
|
||||
"description": "Professional narrator",
|
||||
"created_at": "2024-01-29T12:00:00Z",
|
||||
"sample_count": 2
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
### Get Profile
|
||||
|
||||
```http
|
||||
GET /profiles/{id}
|
||||
```
|
||||
|
||||
**Response:**
|
||||
```json
|
||||
{
|
||||
"id": "abc123",
|
||||
"name": "John Smith",
|
||||
"language": "en",
|
||||
"description": "Professional narrator",
|
||||
"created_at": "2024-01-29T12:00:00Z",
|
||||
"samples": [
|
||||
{
|
||||
"id": "sample123",
|
||||
"duration": 15.5,
|
||||
"created_at": "2024-01-29T12:00:00Z"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
### Create Profile
|
||||
|
||||
```http
|
||||
POST /profiles
|
||||
```
|
||||
|
||||
**Request:**
|
||||
```json
|
||||
{
|
||||
"name": "John Smith",
|
||||
"language": "en",
|
||||
"description": "Professional narrator"
|
||||
}
|
||||
```
|
||||
|
||||
**Response:**
|
||||
```json
|
||||
{
|
||||
"id": "abc123",
|
||||
"name": "John Smith",
|
||||
"language": "en",
|
||||
"description": "Professional narrator",
|
||||
"created_at": "2024-01-29T12:00:00Z"
|
||||
}
|
||||
```
|
||||
|
||||
### Update Profile
|
||||
|
||||
```http
|
||||
PUT /profiles/{id}
|
||||
```
|
||||
|
||||
**Request:**
|
||||
```json
|
||||
{
|
||||
"name": "Updated Name",
|
||||
"description": "Updated description"
|
||||
}
|
||||
```
|
||||
|
||||
### Delete Profile
|
||||
|
||||
```http
|
||||
DELETE /profiles/{id}
|
||||
```
|
||||
|
||||
**Response:**
|
||||
```json
|
||||
{
|
||||
"success": true
|
||||
}
|
||||
```
|
||||
|
||||
### Add Voice Sample
|
||||
|
||||
```http
|
||||
POST /profiles/{id}/samples
|
||||
```
|
||||
|
||||
**Request:** (multipart/form-data)
|
||||
```
|
||||
audio: <file>
|
||||
```
|
||||
|
||||
**Response:**
|
||||
```json
|
||||
{
|
||||
"sample_id": "sample123",
|
||||
"duration": 15.5
|
||||
}
|
||||
```
|
||||
|
||||
## TypeScript Example
|
||||
|
||||
```typescript
|
||||
import { VoiceboxClient } from '@/lib/api'
|
||||
|
||||
const client = new VoiceboxClient({
|
||||
baseUrl: 'http://localhost:17493'
|
||||
})
|
||||
|
||||
// Create profile
|
||||
const profile = await client.createProfile({
|
||||
name: 'John Smith',
|
||||
language: 'en',
|
||||
description: 'Professional narrator'
|
||||
})
|
||||
|
||||
// Add sample
|
||||
await client.addSample(profile.id, audioFile)
|
||||
|
||||
// List all profiles
|
||||
const profiles = await client.listProfiles()
|
||||
```
|
||||
|
||||
For full API documentation, visit `http://localhost:17493/docs` when the server is running.
|
||||
@@ -0,0 +1,6 @@
|
||||
import { HomeLayout } from 'fumadocs-ui/layouts/home';
|
||||
import { baseOptions } from '@/lib/layout.shared';
|
||||
|
||||
export default function Layout({ children }: LayoutProps<'/'>) {
|
||||
return <HomeLayout {...baseOptions()}>{children}</HomeLayout>;
|
||||
}
|
||||
@@ -0,0 +1,5 @@
|
||||
import { redirect } from 'next/navigation';
|
||||
|
||||
export default function HomePage() {
|
||||
redirect('/docs');
|
||||
}
|
||||
@@ -0,0 +1,7 @@
|
||||
import { source } from '@/lib/source';
|
||||
import { createFromSource } from 'fumadocs-core/search/server';
|
||||
|
||||
export const { GET } = createFromSource(source, {
|
||||
// https://docs.orama.com/docs/orama-js/supported-languages
|
||||
language: 'english',
|
||||
});
|
||||
@@ -0,0 +1,74 @@
|
||||
import { createRelativeLink } from 'fumadocs-ui/mdx';
|
||||
import { DocsBody, DocsDescription, DocsPage, DocsTitle } from 'fumadocs-ui/page';
|
||||
import type { Metadata } from 'next';
|
||||
import { notFound } from 'next/navigation';
|
||||
import { MarkdownCopyButton, ViewOptionsPopover } from '@/components/ai/page-actions';
|
||||
import { APIPage } from '@/components/api-page';
|
||||
import { getPageImage, source } from '@/lib/source';
|
||||
import { getMDXComponents } from '@/mdx-components';
|
||||
|
||||
export default async function Page(props: PageProps<'/docs/[[...slug]]'>) {
|
||||
const params = await props.params;
|
||||
const page = source.getPage(params.slug);
|
||||
if (!page) notFound();
|
||||
|
||||
const MDX = page.data.body;
|
||||
const markdownUrl = `${page.url}.mdx`;
|
||||
const githubUrl = `https://github.com/jamiepine/voicebox/blob/main/docs/content/docs/${page.path}`;
|
||||
|
||||
return (
|
||||
<DocsPage
|
||||
toc={page.data.toc}
|
||||
full={page.data.full}
|
||||
editOnGithub={{
|
||||
owner: 'jamiepine',
|
||||
repo: 'voicebox',
|
||||
sha: 'main',
|
||||
path: `docs/content/docs/${page.path}`,
|
||||
}}
|
||||
lastUpdate={page.data.lastModified}
|
||||
>
|
||||
<DocsTitle>{page.data.title}</DocsTitle>
|
||||
<DocsDescription className="mb-0">{page.data.description}</DocsDescription>
|
||||
<div className="flex flex-row gap-2 items-center">
|
||||
<MarkdownCopyButton markdownUrl={markdownUrl} />
|
||||
<ViewOptionsPopover markdownUrl={markdownUrl} githubUrl={githubUrl} />
|
||||
</div>
|
||||
<div
|
||||
role="separator"
|
||||
style={{
|
||||
height: '1px',
|
||||
background: 'currentColor',
|
||||
opacity: 0.15,
|
||||
marginTop: '8px',
|
||||
marginBottom: '24px',
|
||||
}}
|
||||
/>
|
||||
<DocsBody>
|
||||
<MDX
|
||||
components={getMDXComponents({
|
||||
a: createRelativeLink(source, page),
|
||||
})}
|
||||
/>
|
||||
</DocsBody>
|
||||
</DocsPage>
|
||||
);
|
||||
}
|
||||
|
||||
export async function generateStaticParams() {
|
||||
return source.generateParams();
|
||||
}
|
||||
|
||||
export async function generateMetadata(props: PageProps<'/docs/[[...slug]]'>): Promise<Metadata> {
|
||||
const params = await props.params;
|
||||
const page = source.getPage(params.slug);
|
||||
if (!page) notFound();
|
||||
|
||||
return {
|
||||
title: page.data.title,
|
||||
description: page.data.description,
|
||||
openGraph: {
|
||||
images: getPageImage(page).url,
|
||||
},
|
||||
};
|
||||
}
|
||||
@@ -0,0 +1,11 @@
|
||||
import { source } from '@/lib/source';
|
||||
import { DocsLayout } from 'fumadocs-ui/layouts/docs';
|
||||
import { baseOptions } from '@/lib/layout.shared';
|
||||
|
||||
export default function Layout({ children }: LayoutProps<'/docs'>) {
|
||||
return (
|
||||
<DocsLayout tree={source.pageTree} {...baseOptions()}>
|
||||
{children}
|
||||
</DocsLayout>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,14 @@
|
||||
@import 'tailwindcss';
|
||||
@import 'fumadocs-ui/css/neutral.css';
|
||||
@import 'fumadocs-ui/css/preset.css';
|
||||
@import 'fumadocs-openapi/css/preset.css';
|
||||
|
||||
:root {
|
||||
--color-fd-primary: hsl(43, 50%, 50%);
|
||||
--color-fd-primary-foreground: hsl(222.2, 47.4%, 11.2%);
|
||||
}
|
||||
|
||||
.dark {
|
||||
--color-fd-primary: hsl(43, 50%, 45%);
|
||||
--color-fd-primary-foreground: hsl(0, 0%, 95%);
|
||||
}
|
||||
@@ -0,0 +1,17 @@
|
||||
import { RootProvider } from 'fumadocs-ui/provider/next';
|
||||
import './global.css';
|
||||
import { Inter } from 'next/font/google';
|
||||
|
||||
const inter = Inter({
|
||||
subsets: ['latin'],
|
||||
});
|
||||
|
||||
export default function Layout({ children }: LayoutProps<'/'>) {
|
||||
return (
|
||||
<html lang="en" className={inter.className} suppressHydrationWarning>
|
||||
<body className="flex flex-col min-h-screen">
|
||||
<RootProvider>{children}</RootProvider>
|
||||
</body>
|
||||
</html>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,10 @@
|
||||
import { getLLMText, source } from '@/lib/source';
|
||||
|
||||
export const revalidate = false;
|
||||
|
||||
export async function GET() {
|
||||
const scan = source.getPages().map(getLLMText);
|
||||
const scanned = await Promise.all(scan);
|
||||
|
||||
return new Response(scanned.join('\n\n'));
|
||||
}
|
||||
@@ -0,0 +1,20 @@
|
||||
import { notFound } from 'next/navigation';
|
||||
import { getLLMText, source } from '@/lib/source';
|
||||
|
||||
export const revalidate = false;
|
||||
|
||||
export async function GET(_req: Request, { params }: RouteContext<'/llms.mdx/docs/[[...slug]]'>) {
|
||||
const { slug } = await params;
|
||||
const page = source.getPage(slug);
|
||||
if (!page) notFound();
|
||||
|
||||
return new Response(await getLLMText(page), {
|
||||
headers: {
|
||||
'Content-Type': 'text/markdown',
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
export function generateStaticParams() {
|
||||
return source.generateParams();
|
||||
}
|
||||
@@ -0,0 +1,36 @@
|
||||
import { getPageImage, source } from '@/lib/source';
|
||||
import { notFound } from 'next/navigation';
|
||||
import { ImageResponse } from 'next/og';
|
||||
import { generate as DefaultImage } from 'fumadocs-ui/og';
|
||||
|
||||
export const revalidate = false;
|
||||
|
||||
export async function GET(
|
||||
_req: Request,
|
||||
{ params }: RouteContext<'/og/docs/[...slug]'>,
|
||||
) {
|
||||
const { slug } = await params;
|
||||
const page = source.getPage(slug.slice(0, -1));
|
||||
if (!page) notFound();
|
||||
|
||||
return new ImageResponse(
|
||||
(
|
||||
<DefaultImage
|
||||
title={page.data.title}
|
||||
description={page.data.description}
|
||||
site="My App"
|
||||
/>
|
||||
),
|
||||
{
|
||||
width: 1200,
|
||||
height: 630,
|
||||
},
|
||||
);
|
||||
}
|
||||
|
||||
export function generateStaticParams() {
|
||||
return source.getPages().map((page) => ({
|
||||
lang: page.locale,
|
||||
slug: getPageImage(page).segments,
|
||||
}));
|
||||
}
|
||||
+277
-1278
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,13 @@
|
||||
{
|
||||
"$schema": "node_modules/@fumadocs/cli/dist/schema/default.json",
|
||||
"aliases": {
|
||||
"uiDir": "./components/ui",
|
||||
"componentsDir": "./components",
|
||||
"blockDir": "./components",
|
||||
"cssDir": "./styles",
|
||||
"libDir": "./lib"
|
||||
},
|
||||
"baseDir": "",
|
||||
"uiLibrary": "radix-ui",
|
||||
"commands": {}
|
||||
}
|
||||
@@ -0,0 +1,247 @@
|
||||
'use client';
|
||||
import { type ComponentProps, useMemo, useState } from 'react';
|
||||
import { Check, ChevronDown, Copy, ExternalLinkIcon, TextIcon } from 'lucide-react';
|
||||
import { cn } from '../../lib/cn';
|
||||
import { useCopyButton } from 'fumadocs-ui/utils/use-copy-button';
|
||||
import { Popover, PopoverTrigger, PopoverContent } from '../ui/popover';
|
||||
import { buttonVariants } from '../ui/button';
|
||||
|
||||
const cache = new Map<string, Promise<string>>();
|
||||
|
||||
export function MarkdownCopyButton({
|
||||
markdownUrl,
|
||||
...props
|
||||
}: ComponentProps<'button'> & {
|
||||
/**
|
||||
* A URL to fetch the raw Markdown/MDX content of page
|
||||
*/
|
||||
markdownUrl: string;
|
||||
}) {
|
||||
const [isLoading, setLoading] = useState(false);
|
||||
const [checked, onClick] = useCopyButton(async () => {
|
||||
const cached = cache.get(markdownUrl);
|
||||
if (cached) return navigator.clipboard.writeText(await cached);
|
||||
|
||||
setLoading(true);
|
||||
|
||||
try {
|
||||
const promise = fetch(markdownUrl).then((res) => res.text());
|
||||
cache.set(markdownUrl, promise);
|
||||
await navigator.clipboard.write([
|
||||
new ClipboardItem({
|
||||
'text/plain': promise,
|
||||
}),
|
||||
]);
|
||||
} finally {
|
||||
setLoading(false);
|
||||
}
|
||||
});
|
||||
|
||||
return (
|
||||
<button
|
||||
disabled={isLoading}
|
||||
onClick={onClick}
|
||||
{...props}
|
||||
className={cn(
|
||||
buttonVariants({
|
||||
variant: 'secondary',
|
||||
size: 'sm',
|
||||
className: 'gap-2 [&_svg]:size-3.5 [&_svg]:text-fd-muted-foreground',
|
||||
}),
|
||||
props.className,
|
||||
)}
|
||||
>
|
||||
{checked ? <Check /> : <Copy />}
|
||||
Copy Markdown
|
||||
</button>
|
||||
);
|
||||
}
|
||||
|
||||
export function ViewOptionsPopover({
|
||||
markdownUrl,
|
||||
githubUrl,
|
||||
...props
|
||||
}: ComponentProps<typeof PopoverTrigger> & {
|
||||
/**
|
||||
* A URL to the raw Markdown/MDX content of page
|
||||
*/
|
||||
markdownUrl: string;
|
||||
|
||||
/**
|
||||
* Source file URL on GitHub
|
||||
*/
|
||||
githubUrl: string;
|
||||
}) {
|
||||
const items = useMemo(() => {
|
||||
const pageUrl = typeof window !== 'undefined' ? window.location.href : 'loading';
|
||||
const q = `Read ${pageUrl}, I want to ask questions about it.`;
|
||||
|
||||
return [
|
||||
{
|
||||
title: 'Open in GitHub',
|
||||
href: githubUrl,
|
||||
icon: (
|
||||
<svg fill="currentColor" role="img" viewBox="0 0 24 24">
|
||||
<title>GitHub</title>
|
||||
<path d="M12 .297c-6.63 0-12 5.373-12 12 0 5.303 3.438 9.8 8.205 11.385.6.113.82-.258.82-.577 0-.285-.01-1.04-.015-2.04-3.338.724-4.042-1.61-4.042-1.61C4.422 18.07 3.633 17.7 3.633 17.7c-1.087-.744.084-.729.084-.729 1.205.084 1.838 1.236 1.838 1.236 1.07 1.835 2.809 1.305 3.495.998.108-.776.417-1.305.76-1.605-2.665-.3-5.466-1.332-5.466-5.93 0-1.31.465-2.38 1.235-3.22-.135-.303-.54-1.523.105-3.176 0 0 1.005-.322 3.3 1.23.96-.267 1.98-.399 3-.405 1.02.006 2.04.138 3 .405 2.28-1.552 3.285-1.23 3.285-1.23.645 1.653.24 2.873.12 3.176.765.84 1.23 1.91 1.23 3.22 0 4.61-2.805 5.625-5.475 5.92.42.36.81 1.096.81 2.22 0 1.606-.015 2.896-.015 3.286 0 .315.21.69.825.57C20.565 22.092 24 17.592 24 12.297c0-6.627-5.373-12-12-12" />
|
||||
</svg>
|
||||
),
|
||||
},
|
||||
{
|
||||
title: 'View as Markdown',
|
||||
href: markdownUrl,
|
||||
icon: <TextIcon />,
|
||||
},
|
||||
{
|
||||
title: 'Open in Scira AI',
|
||||
href: `https://scira.ai/?${new URLSearchParams({
|
||||
q,
|
||||
})}`,
|
||||
icon: (
|
||||
<svg
|
||||
width="910"
|
||||
height="934"
|
||||
viewBox="0 0 910 934"
|
||||
fill="none"
|
||||
xmlns="http://www.w3.org/2000/svg"
|
||||
>
|
||||
<title>Scira AI</title>
|
||||
<path
|
||||
d="M647.664 197.775C569.13 189.049 525.5 145.419 516.774 66.8849C508.048 145.419 464.418 189.049 385.884 197.775C464.418 206.501 508.048 250.131 516.774 328.665C525.5 250.131 569.13 206.501 647.664 197.775Z"
|
||||
fill="currentColor"
|
||||
stroke="currentColor"
|
||||
strokeWidth="8"
|
||||
strokeLinejoin="round"
|
||||
/>
|
||||
<path
|
||||
d="M516.774 304.217C510.299 275.491 498.208 252.087 480.335 234.214C462.462 216.341 439.058 204.251 410.333 197.775C439.059 191.3 462.462 179.209 480.335 161.336C498.208 143.463 510.299 120.06 516.774 91.334C523.25 120.059 535.34 143.463 553.213 161.336C571.086 179.209 594.49 191.3 623.216 197.775C594.49 204.251 571.086 216.341 553.213 234.214C535.34 252.087 523.25 275.491 516.774 304.217Z"
|
||||
fill="currentColor"
|
||||
stroke="currentColor"
|
||||
strokeWidth="8"
|
||||
strokeLinejoin="round"
|
||||
/>
|
||||
<path
|
||||
d="M857.5 508.116C763.259 497.644 710.903 445.288 700.432 351.047C689.961 445.288 637.605 497.644 543.364 508.116C637.605 518.587 689.961 570.943 700.432 665.184C710.903 570.943 763.259 518.587 857.5 508.116Z"
|
||||
stroke="currentColor"
|
||||
strokeWidth="20"
|
||||
strokeLinejoin="round"
|
||||
/>
|
||||
<path
|
||||
d="M700.432 615.957C691.848 589.05 678.575 566.357 660.383 548.165C642.191 529.973 619.499 516.7 592.593 508.116C619.499 499.533 642.191 486.258 660.383 468.066C678.575 449.874 691.848 427.181 700.432 400.274C709.015 427.181 722.289 449.874 740.481 468.066C758.673 486.258 781.365 499.533 808.271 508.116C781.365 516.7 758.673 529.973 740.481 548.165C722.289 566.357 709.015 589.05 700.432 615.957Z"
|
||||
stroke="currentColor"
|
||||
strokeWidth="20"
|
||||
strokeLinejoin="round"
|
||||
/>
|
||||
<path
|
||||
d="M889.949 121.237C831.049 114.692 798.326 81.9698 791.782 23.0692C785.237 81.9698 752.515 114.692 693.614 121.237C752.515 127.781 785.237 160.504 791.782 219.404C798.326 160.504 831.049 127.781 889.949 121.237Z"
|
||||
fill="currentColor"
|
||||
stroke="currentColor"
|
||||
strokeWidth="8"
|
||||
strokeLinejoin="round"
|
||||
/>
|
||||
<path
|
||||
d="M791.782 196.795C786.697 176.937 777.869 160.567 765.16 147.858C752.452 135.15 736.082 126.322 716.226 121.237C736.082 116.152 752.452 107.324 765.16 94.6152C777.869 81.9065 786.697 65.5368 791.782 45.6797C796.867 65.5367 805.695 81.9066 818.403 94.6152C831.112 107.324 847.481 116.152 867.338 121.237C847.481 126.322 831.112 135.15 818.403 147.858C805.694 160.567 796.867 176.937 791.782 196.795Z"
|
||||
fill="currentColor"
|
||||
stroke="currentColor"
|
||||
strokeWidth="8"
|
||||
strokeLinejoin="round"
|
||||
/>
|
||||
<path
|
||||
d="M760.632 764.337C720.719 814.616 669.835 855.1 611.872 882.692C553.91 910.285 490.404 924.255 426.213 923.533C362.022 922.812 298.846 907.419 241.518 878.531C184.19 849.643 134.228 808.026 95.4548 756.863C56.6815 705.7 30.1238 646.346 17.8129 583.343C5.50207 520.339 7.76433 455.354 24.4266 393.359C41.089 331.364 71.7099 274.001 113.947 225.658C156.184 177.315 208.919 139.273 268.117 114.442"
|
||||
stroke="currentColor"
|
||||
strokeWidth="30"
|
||||
strokeLinecap="round"
|
||||
strokeLinejoin="round"
|
||||
/>
|
||||
</svg>
|
||||
),
|
||||
},
|
||||
{
|
||||
title: 'Open in ChatGPT',
|
||||
href: `https://chatgpt.com/?${new URLSearchParams({
|
||||
hints: 'search',
|
||||
q,
|
||||
})}`,
|
||||
icon: (
|
||||
<svg
|
||||
role="img"
|
||||
viewBox="0 0 24 24"
|
||||
fill="currentColor"
|
||||
xmlns="http://www.w3.org/2000/svg"
|
||||
>
|
||||
<title>OpenAI</title>
|
||||
<path d="M22.2819 9.8211a5.9847 5.9847 0 0 0-.5157-4.9108 6.0462 6.0462 0 0 0-6.5098-2.9A6.0651 6.0651 0 0 0 4.9807 4.1818a5.9847 5.9847 0 0 0-3.9977 2.9 6.0462 6.0462 0 0 0 .7427 7.0966 5.98 5.98 0 0 0 .511 4.9107 6.051 6.051 0 0 0 6.5146 2.9001A5.9847 5.9847 0 0 0 13.2599 24a6.0557 6.0557 0 0 0 5.7718-4.2058 5.9894 5.9894 0 0 0 3.9977-2.9001 6.0557 6.0557 0 0 0-.7475-7.0729zm-9.022 12.6081a4.4755 4.4755 0 0 1-2.8764-1.0408l.1419-.0804 4.7783-2.7582a.7948.7948 0 0 0 .3927-.6813v-6.7369l2.02 1.1686a.071.071 0 0 1 .038.052v5.5826a4.504 4.504 0 0 1-4.4945 4.4944zm-9.6607-4.1254a4.4708 4.4708 0 0 1-.5346-3.0137l.142.0852 4.783 2.7582a.7712.7712 0 0 0 .7806 0l5.8428-3.3685v2.3324a.0804.0804 0 0 1-.0332.0615L9.74 19.9502a4.4992 4.4992 0 0 1-6.1408-1.6464zM2.3408 7.8956a4.485 4.485 0 0 1 2.3655-1.9728V11.6a.7664.7664 0 0 0 .3879.6765l5.8144 3.3543-2.0201 1.1685a.0757.0757 0 0 1-.071 0l-4.8303-2.7865A4.504 4.504 0 0 1 2.3408 7.872zm16.5963 3.8558L13.1038 8.364 15.1192 7.2a.0757.0757 0 0 1 .071 0l4.8303 2.7913a4.4944 4.4944 0 0 1-.6765 8.1042v-5.6772a.79.79 0 0 0-.407-.667zm2.0107-3.0231l-.142-.0852-4.7735-2.7818a.7759.7759 0 0 0-.7854 0L9.409 9.2297V6.8974a.0662.0662 0 0 1 .0284-.0615l4.8303-2.7866a4.4992 4.4992 0 0 1 6.6802 4.66zM8.3065 12.863l-2.02-1.1638a.0804.0804 0 0 1-.038-.0567V6.0742a4.4992 4.4992 0 0 1 7.3757-3.4537l-.142.0805L8.704 5.459a.7948.7948 0 0 0-.3927.6813zm1.0976-2.3654l2.602-1.4998 2.6069 1.4998v2.9994l-2.5974 1.4997-2.6067-1.4997Z" />
|
||||
</svg>
|
||||
),
|
||||
},
|
||||
{
|
||||
title: 'Open in Claude',
|
||||
href: `https://claude.ai/new?${new URLSearchParams({
|
||||
q,
|
||||
})}`,
|
||||
icon: (
|
||||
<svg
|
||||
fill="currentColor"
|
||||
role="img"
|
||||
viewBox="0 0 24 24"
|
||||
xmlns="http://www.w3.org/2000/svg"
|
||||
>
|
||||
<title>Anthropic</title>
|
||||
<path d="M17.3041 3.541h-3.6718l6.696 16.918H24Zm-10.6082 0L0 20.459h3.7442l1.3693-3.5527h7.0052l1.3693 3.5528h3.7442L10.5363 3.5409Zm-.3712 10.2232 2.2914-5.9456 2.2914 5.9456Z" />
|
||||
</svg>
|
||||
),
|
||||
},
|
||||
{
|
||||
title: 'Open in Cursor',
|
||||
icon: (
|
||||
<svg
|
||||
fill="currentColor"
|
||||
role="img"
|
||||
viewBox="0 0 24 24"
|
||||
xmlns="http://www.w3.org/2000/svg"
|
||||
>
|
||||
<title>Cursor</title>
|
||||
<path d="M11.503.131 1.891 5.678a.84.84 0 0 0-.42.726v11.188c0 .3.162.575.42.724l9.609 5.55a1 1 0 0 0 .998 0l9.61-5.55a.84.84 0 0 0 .42-.724V6.404a.84.84 0 0 0-.42-.726L12.497.131a1.01 1.01 0 0 0-.996 0M2.657 6.338h18.55c.263 0 .43.287.297.515L12.23 22.918c-.062.107-.229.064-.229-.06V12.335a.59.59 0 0 0-.295-.51l-9.11-5.257c-.109-.063-.064-.23.061-.23" />
|
||||
</svg>
|
||||
),
|
||||
href: `https://cursor.com/link/prompt?${new URLSearchParams({
|
||||
text: q,
|
||||
})}`,
|
||||
},
|
||||
];
|
||||
}, [githubUrl, markdownUrl]);
|
||||
|
||||
return (
|
||||
<Popover>
|
||||
<PopoverTrigger
|
||||
{...props}
|
||||
className={cn(
|
||||
buttonVariants({
|
||||
variant: 'secondary',
|
||||
size: 'sm',
|
||||
}),
|
||||
'gap-2 data-[state=open]:bg-fd-accent data-[state=open]:text-fd-accent-foreground',
|
||||
props.className,
|
||||
)}
|
||||
>
|
||||
Open
|
||||
<ChevronDown className="size-3.5 text-fd-muted-foreground" />
|
||||
</PopoverTrigger>
|
||||
<PopoverContent className="flex flex-col">
|
||||
{items.map((item) => (
|
||||
<a
|
||||
key={item.href}
|
||||
href={item.href}
|
||||
rel="noreferrer noopener"
|
||||
target="_blank"
|
||||
className="text-sm p-2 rounded-lg inline-flex items-center gap-2 hover:text-fd-accent-foreground hover:bg-fd-accent [&_svg]:size-4"
|
||||
>
|
||||
{item.icon}
|
||||
{item.title}
|
||||
<ExternalLinkIcon className="text-fd-muted-foreground size-3.5 ms-auto" />
|
||||
</a>
|
||||
))}
|
||||
</PopoverContent>
|
||||
</Popover>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,6 @@
|
||||
'use client';
|
||||
import { defineClientConfig } from 'fumadocs-openapi/ui/client';
|
||||
|
||||
export default defineClientConfig({
|
||||
// Client-side configuration for API playground
|
||||
});
|
||||
@@ -0,0 +1,7 @@
|
||||
import { openapi } from '@/lib/openapi';
|
||||
import { createAPIPage } from 'fumadocs-openapi/ui';
|
||||
import client from './api-page.client';
|
||||
|
||||
export const APIPage = createAPIPage(openapi, {
|
||||
client,
|
||||
});
|
||||
@@ -0,0 +1,29 @@
|
||||
import { cva, type VariantProps } from 'class-variance-authority';
|
||||
|
||||
const variants = {
|
||||
primary:
|
||||
'bg-fd-primary text-fd-primary-foreground hover:bg-fd-primary/80 disabled:bg-fd-secondary disabled:text-fd-secondary-foreground',
|
||||
outline: 'border hover:bg-fd-accent hover:text-fd-accent-foreground',
|
||||
ghost: 'hover:bg-fd-accent hover:text-fd-accent-foreground',
|
||||
secondary:
|
||||
'border bg-fd-secondary text-fd-secondary-foreground hover:bg-fd-accent hover:text-fd-accent-foreground',
|
||||
} as const;
|
||||
|
||||
export const buttonVariants = cva(
|
||||
'inline-flex items-center justify-center rounded-md p-2 text-sm font-medium transition-colors duration-100 disabled:pointer-events-none disabled:opacity-50 focus-visible:outline-none focus-visible:ring-2 focus-visible:ring-fd-ring',
|
||||
{
|
||||
variants: {
|
||||
variant: variants,
|
||||
// fumadocs use `color` instead of `variant`
|
||||
color: variants,
|
||||
size: {
|
||||
sm: 'gap-1 px-2 py-1.5 text-xs',
|
||||
icon: 'p-1.5 [&_svg]:size-5',
|
||||
'icon-sm': 'p-1.5 [&_svg]:size-4.5',
|
||||
'icon-xs': 'p-1 [&_svg]:size-4',
|
||||
},
|
||||
},
|
||||
},
|
||||
);
|
||||
|
||||
export type ButtonProps = VariantProps<typeof buttonVariants>;
|
||||
@@ -0,0 +1,32 @@
|
||||
'use client';
|
||||
import * as PopoverPrimitive from '@radix-ui/react-popover';
|
||||
import * as React from 'react';
|
||||
import { cn } from '../../lib/cn';
|
||||
|
||||
const Popover = PopoverPrimitive.Root;
|
||||
|
||||
const PopoverTrigger = PopoverPrimitive.Trigger;
|
||||
|
||||
const PopoverContent = React.forwardRef<
|
||||
React.ComponentRef<typeof PopoverPrimitive.Content>,
|
||||
React.ComponentPropsWithoutRef<typeof PopoverPrimitive.Content>
|
||||
>(({ className, align = 'center', sideOffset = 4, ...props }, ref) => (
|
||||
<PopoverPrimitive.Portal>
|
||||
<PopoverPrimitive.Content
|
||||
ref={ref}
|
||||
align={align}
|
||||
sideOffset={sideOffset}
|
||||
side="bottom"
|
||||
className={cn(
|
||||
'z-50 origin-(--radix-popover-content-transform-origin) overflow-y-auto max-h-(--radix-popover-content-available-height) min-w-[240px] max-w-[98vw] rounded-xl border bg-fd-popover/60 backdrop-blur-lg p-2 text-sm text-fd-popover-foreground shadow-lg focus-visible:outline-none data-[state=closed]:animate-fd-popover-out data-[state=open]:animate-fd-popover-in',
|
||||
className,
|
||||
)}
|
||||
{...props}
|
||||
/>
|
||||
</PopoverPrimitive.Portal>
|
||||
));
|
||||
PopoverContent.displayName = PopoverPrimitive.Content.displayName;
|
||||
|
||||
const PopoverClose = PopoverPrimitive.PopoverClose;
|
||||
|
||||
export { Popover, PopoverTrigger, PopoverContent, PopoverClose };
|
||||
@@ -0,0 +1,67 @@
|
||||
---
|
||||
title: "Documentation README"
|
||||
description: "Voicebox documentation development guide"
|
||||
---
|
||||
|
||||
This directory contains the documentation for Voicebox, built with [Fumadocs](https://fumadocs.dev).
|
||||
|
||||
## Development
|
||||
|
||||
### Prerequisites
|
||||
|
||||
Install Mintlify globally using bun:
|
||||
|
||||
```bash
|
||||
bun add -g mintlify
|
||||
```
|
||||
|
||||
Or use the helper script:
|
||||
|
||||
```bash
|
||||
bun run install:mintlify
|
||||
```
|
||||
|
||||
### Running Locally
|
||||
|
||||
```bash
|
||||
bun run dev
|
||||
```
|
||||
|
||||
This will start the Mintlify dev server.
|
||||
|
||||
The docs will be available at `http://localhost:3000`
|
||||
|
||||
### Structure
|
||||
|
||||
```
|
||||
docs/
|
||||
├── mint.json # Mintlify configuration
|
||||
├── custom.css # Custom styles
|
||||
├── overview/ # Getting started & feature docs
|
||||
├── guides/ # User guides
|
||||
├── api/ # API reference
|
||||
├── development/ # Developer documentation
|
||||
├── logo/ # Logo assets
|
||||
└── public/ # Static assets
|
||||
```
|
||||
|
||||
### Writing Docs
|
||||
|
||||
- Use `.mdx` files for all documentation pages
|
||||
- Follow the existing structure in `mint.json` for navigation
|
||||
- Use Mintlify components for enhanced formatting (Card, CardGroup, Accordion, etc.)
|
||||
- Reference the [Mintlify documentation](https://mintlify.com/docs) for available components
|
||||
|
||||
## Deployment
|
||||
|
||||
Docs are automatically deployed when changes are pushed to the main branch.
|
||||
|
||||
To manually deploy:
|
||||
|
||||
```bash
|
||||
mintlify deploy
|
||||
```
|
||||
|
||||
## Contributing
|
||||
|
||||
See [CONTRIBUTING.md](../CONTRIBUTING.md) for contribution guidelines.
|
||||
@@ -1,4 +1,7 @@
|
||||
# Troubleshooting Guide
|
||||
---
|
||||
title: "Troubleshooting Guide"
|
||||
description: "Common issues and solutions for Voicebox"
|
||||
---
|
||||
|
||||
Common issues and solutions for Voicebox.
|
||||
|
||||
@@ -0,0 +1,5 @@
|
||||
{
|
||||
"title": "API Reference",
|
||||
"defaultOpen": true,
|
||||
"pages": ["unknown"]
|
||||
}
|
||||
+16
@@ -0,0 +1,16 @@
|
||||
---
|
||||
title: Add Profile Sample
|
||||
description: Add a sample to a voice profile.
|
||||
full: true
|
||||
_openapi:
|
||||
method: POST
|
||||
toc: []
|
||||
structuredData:
|
||||
headings: []
|
||||
contents:
|
||||
- content: Add a sample to a voice profile.
|
||||
---
|
||||
|
||||
{/* This file was generated by Fumadocs. Do not edit this file directly. Any changes should be made by running the generation command again. */}
|
||||
|
||||
<APIPage document={"./openapi.json"} operations={[{"path":"/profiles/{profile_id}/samples","method":"post"}]} />
|
||||
@@ -0,0 +1,16 @@
|
||||
---
|
||||
title: Create Profile
|
||||
description: Create a new voice profile.
|
||||
full: true
|
||||
_openapi:
|
||||
method: POST
|
||||
toc: []
|
||||
structuredData:
|
||||
headings: []
|
||||
contents:
|
||||
- content: Create a new voice profile.
|
||||
---
|
||||
|
||||
{/* This file was generated by Fumadocs. Do not edit this file directly. Any changes should be made by running the generation command again. */}
|
||||
|
||||
<APIPage document={"./openapi.json"} operations={[{"path":"/profiles","method":"post"}]} />
|
||||
+16
@@ -0,0 +1,16 @@
|
||||
---
|
||||
title: Delete Generation
|
||||
description: Delete a generation.
|
||||
full: true
|
||||
_openapi:
|
||||
method: DELETE
|
||||
toc: []
|
||||
structuredData:
|
||||
headings: []
|
||||
contents:
|
||||
- content: Delete a generation.
|
||||
---
|
||||
|
||||
{/* This file was generated by Fumadocs. Do not edit this file directly. Any changes should be made by running the generation command again. */}
|
||||
|
||||
<APIPage document={"./openapi.json"} operations={[{"path":"/history/{generation_id}","method":"delete"}]} />
|
||||
+16
@@ -0,0 +1,16 @@
|
||||
---
|
||||
title: Delete Profile
|
||||
description: Delete a voice profile.
|
||||
full: true
|
||||
_openapi:
|
||||
method: DELETE
|
||||
toc: []
|
||||
structuredData:
|
||||
headings: []
|
||||
contents:
|
||||
- content: Delete a voice profile.
|
||||
---
|
||||
|
||||
{/* This file was generated by Fumadocs. Do not edit this file directly. Any changes should be made by running the generation command again. */}
|
||||
|
||||
<APIPage document={"./openapi.json"} operations={[{"path":"/profiles/{profile_id}","method":"delete"}]} />
|
||||
+16
@@ -0,0 +1,16 @@
|
||||
---
|
||||
title: Delete Profile Sample
|
||||
description: Delete a profile sample.
|
||||
full: true
|
||||
_openapi:
|
||||
method: DELETE
|
||||
toc: []
|
||||
structuredData:
|
||||
headings: []
|
||||
contents:
|
||||
- content: Delete a profile sample.
|
||||
---
|
||||
|
||||
{/* This file was generated by Fumadocs. Do not edit this file directly. Any changes should be made by running the generation command again. */}
|
||||
|
||||
<APIPage document={"./openapi.json"} operations={[{"path":"/profiles/samples/{sample_id}","method":"delete"}]} />
|
||||
@@ -0,0 +1,16 @@
|
||||
---
|
||||
title: Generate Speech
|
||||
description: Generate speech from text using a voice profile.
|
||||
full: true
|
||||
_openapi:
|
||||
method: POST
|
||||
toc: []
|
||||
structuredData:
|
||||
headings: []
|
||||
contents:
|
||||
- content: Generate speech from text using a voice profile.
|
||||
---
|
||||
|
||||
{/* This file was generated by Fumadocs. Do not edit this file directly. Any changes should be made by running the generation command again. */}
|
||||
|
||||
<APIPage document={"./openapi.json"} operations={[{"path":"/generate","method":"post"}]} />
|
||||
@@ -0,0 +1,16 @@
|
||||
---
|
||||
title: Get Audio
|
||||
description: Serve generated audio file.
|
||||
full: true
|
||||
_openapi:
|
||||
method: GET
|
||||
toc: []
|
||||
structuredData:
|
||||
headings: []
|
||||
contents:
|
||||
- content: Serve generated audio file.
|
||||
---
|
||||
|
||||
{/* This file was generated by Fumadocs. Do not edit this file directly. Any changes should be made by running the generation command again. */}
|
||||
|
||||
<APIPage document={"./openapi.json"} operations={[{"path":"/audio/{generation_id}","method":"get"}]} />
|
||||
+16
@@ -0,0 +1,16 @@
|
||||
---
|
||||
title: Get Generation
|
||||
description: Get a generation by ID.
|
||||
full: true
|
||||
_openapi:
|
||||
method: GET
|
||||
toc: []
|
||||
structuredData:
|
||||
headings: []
|
||||
contents:
|
||||
- content: Get a generation by ID.
|
||||
---
|
||||
|
||||
{/* This file was generated by Fumadocs. Do not edit this file directly. Any changes should be made by running the generation command again. */}
|
||||
|
||||
<APIPage document={"./openapi.json"} operations={[{"path":"/history/{generation_id}","method":"get"}]} />
|
||||
+16
@@ -0,0 +1,16 @@
|
||||
---
|
||||
title: Get Model Progress
|
||||
description: Get model download progress via Server-Sent Events.
|
||||
full: true
|
||||
_openapi:
|
||||
method: GET
|
||||
toc: []
|
||||
structuredData:
|
||||
headings: []
|
||||
contents:
|
||||
- content: Get model download progress via Server-Sent Events.
|
||||
---
|
||||
|
||||
{/* This file was generated by Fumadocs. Do not edit this file directly. Any changes should be made by running the generation command again. */}
|
||||
|
||||
<APIPage document={"./openapi.json"} operations={[{"path":"/models/progress/{model_name}","method":"get"}]} />
|
||||
@@ -0,0 +1,16 @@
|
||||
---
|
||||
title: Get Model Status
|
||||
description: Get status of all available models.
|
||||
full: true
|
||||
_openapi:
|
||||
method: GET
|
||||
toc: []
|
||||
structuredData:
|
||||
headings: []
|
||||
contents:
|
||||
- content: Get status of all available models.
|
||||
---
|
||||
|
||||
{/* This file was generated by Fumadocs. Do not edit this file directly. Any changes should be made by running the generation command again. */}
|
||||
|
||||
<APIPage document={"./openapi.json"} operations={[{"path":"/models/status","method":"get"}]} />
|
||||
@@ -0,0 +1,16 @@
|
||||
---
|
||||
title: Get Profile
|
||||
description: Get a voice profile by ID.
|
||||
full: true
|
||||
_openapi:
|
||||
method: GET
|
||||
toc: []
|
||||
structuredData:
|
||||
headings: []
|
||||
contents:
|
||||
- content: Get a voice profile by ID.
|
||||
---
|
||||
|
||||
{/* This file was generated by Fumadocs. Do not edit this file directly. Any changes should be made by running the generation command again. */}
|
||||
|
||||
<APIPage document={"./openapi.json"} operations={[{"path":"/profiles/{profile_id}","method":"get"}]} />
|
||||
+16
@@ -0,0 +1,16 @@
|
||||
---
|
||||
title: Get Profile Samples
|
||||
description: Get all samples for a profile.
|
||||
full: true
|
||||
_openapi:
|
||||
method: GET
|
||||
toc: []
|
||||
structuredData:
|
||||
headings: []
|
||||
contents:
|
||||
- content: Get all samples for a profile.
|
||||
---
|
||||
|
||||
{/* This file was generated by Fumadocs. Do not edit this file directly. Any changes should be made by running the generation command again. */}
|
||||
|
||||
<APIPage document={"./openapi.json"} operations={[{"path":"/profiles/{profile_id}/samples","method":"get"}]} />
|
||||
@@ -0,0 +1,16 @@
|
||||
---
|
||||
title: Get Stats
|
||||
description: Get generation statistics.
|
||||
full: true
|
||||
_openapi:
|
||||
method: GET
|
||||
toc: []
|
||||
structuredData:
|
||||
headings: []
|
||||
contents:
|
||||
- content: Get generation statistics.
|
||||
---
|
||||
|
||||
{/* This file was generated by Fumadocs. Do not edit this file directly. Any changes should be made by running the generation command again. */}
|
||||
|
||||
<APIPage document={"./openapi.json"} operations={[{"path":"/history/stats","method":"get"}]} />
|
||||
@@ -0,0 +1,16 @@
|
||||
---
|
||||
title: Health
|
||||
description: Health check endpoint.
|
||||
full: true
|
||||
_openapi:
|
||||
method: GET
|
||||
toc: []
|
||||
structuredData:
|
||||
headings: []
|
||||
contents:
|
||||
- content: Health check endpoint.
|
||||
---
|
||||
|
||||
{/* This file was generated by Fumadocs. Do not edit this file directly. Any changes should be made by running the generation command again. */}
|
||||
|
||||
<APIPage document={"./openapi.json"} operations={[{"path":"/health","method":"get"}]} />
|
||||
@@ -0,0 +1,16 @@
|
||||
---
|
||||
title: List History
|
||||
description: List generation history with optional filters.
|
||||
full: true
|
||||
_openapi:
|
||||
method: GET
|
||||
toc: []
|
||||
structuredData:
|
||||
headings: []
|
||||
contents:
|
||||
- content: List generation history with optional filters.
|
||||
---
|
||||
|
||||
{/* This file was generated by Fumadocs. Do not edit this file directly. Any changes should be made by running the generation command again. */}
|
||||
|
||||
<APIPage document={"./openapi.json"} operations={[{"path":"/history","method":"get"}]} />
|
||||
@@ -0,0 +1,16 @@
|
||||
---
|
||||
title: List Profiles
|
||||
description: List all voice profiles.
|
||||
full: true
|
||||
_openapi:
|
||||
method: GET
|
||||
toc: []
|
||||
structuredData:
|
||||
headings: []
|
||||
contents:
|
||||
- content: List all voice profiles.
|
||||
---
|
||||
|
||||
{/* This file was generated by Fumadocs. Do not edit this file directly. Any changes should be made by running the generation command again. */}
|
||||
|
||||
<APIPage document={"./openapi.json"} operations={[{"path":"/profiles","method":"get"}]} />
|
||||
@@ -0,0 +1,16 @@
|
||||
---
|
||||
title: Load Model
|
||||
description: Manually load TTS model.
|
||||
full: true
|
||||
_openapi:
|
||||
method: POST
|
||||
toc: []
|
||||
structuredData:
|
||||
headings: []
|
||||
contents:
|
||||
- content: Manually load TTS model.
|
||||
---
|
||||
|
||||
{/* This file was generated by Fumadocs. Do not edit this file directly. Any changes should be made by running the generation command again. */}
|
||||
|
||||
<APIPage document={"./openapi.json"} operations={[{"path":"/models/load","method":"post"}]} />
|
||||
@@ -0,0 +1,27 @@
|
||||
{
|
||||
"title": "Endpoints",
|
||||
"pages": [
|
||||
"root__get",
|
||||
"health_health_get",
|
||||
"list_profiles_profiles_get",
|
||||
"create_profile_profiles_post",
|
||||
"get_profile_profiles__profile_id__get",
|
||||
"update_profile_profiles__profile_id__put",
|
||||
"delete_profile_profiles__profile_id__delete",
|
||||
"add_profile_sample_profiles__profile_id__samples_post",
|
||||
"get_profile_samples_profiles__profile_id__samples_get",
|
||||
"delete_profile_sample_profiles_samples__sample_id__delete",
|
||||
"generate_speech_generate_post",
|
||||
"list_history_history_get",
|
||||
"get_generation_history__generation_id__get",
|
||||
"delete_generation_history__generation_id__delete",
|
||||
"get_stats_history_stats_get",
|
||||
"transcribe_audio_transcribe_post",
|
||||
"get_audio_audio__generation_id__get",
|
||||
"load_model_models_load_post",
|
||||
"unload_model_models_unload_post",
|
||||
"get_model_progress_models_progress__model_name__get",
|
||||
"get_model_status_models_status_get",
|
||||
"trigger_model_download_models_download_post"
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,16 @@
|
||||
---
|
||||
title: Root
|
||||
description: Root endpoint.
|
||||
full: true
|
||||
_openapi:
|
||||
method: GET
|
||||
toc: []
|
||||
structuredData:
|
||||
headings: []
|
||||
contents:
|
||||
- content: Root endpoint.
|
||||
---
|
||||
|
||||
{/* This file was generated by Fumadocs. Do not edit this file directly. Any changes should be made by running the generation command again. */}
|
||||
|
||||
<APIPage document={"./openapi.json"} operations={[{"path":"/","method":"get"}]} />
|
||||
@@ -0,0 +1,16 @@
|
||||
---
|
||||
title: Transcribe Audio
|
||||
description: Transcribe audio file to text.
|
||||
full: true
|
||||
_openapi:
|
||||
method: POST
|
||||
toc: []
|
||||
structuredData:
|
||||
headings: []
|
||||
contents:
|
||||
- content: Transcribe audio file to text.
|
||||
---
|
||||
|
||||
{/* This file was generated by Fumadocs. Do not edit this file directly. Any changes should be made by running the generation command again. */}
|
||||
|
||||
<APIPage document={"./openapi.json"} operations={[{"path":"/transcribe","method":"post"}]} />
|
||||
+16
@@ -0,0 +1,16 @@
|
||||
---
|
||||
title: Trigger Model Download
|
||||
description: Trigger download of a specific model.
|
||||
full: true
|
||||
_openapi:
|
||||
method: POST
|
||||
toc: []
|
||||
structuredData:
|
||||
headings: []
|
||||
contents:
|
||||
- content: Trigger download of a specific model.
|
||||
---
|
||||
|
||||
{/* This file was generated by Fumadocs. Do not edit this file directly. Any changes should be made by running the generation command again. */}
|
||||
|
||||
<APIPage document={"./openapi.json"} operations={[{"path":"/models/download","method":"post"}]} />
|
||||
@@ -0,0 +1,16 @@
|
||||
---
|
||||
title: Unload Model
|
||||
description: Unload TTS model to free memory.
|
||||
full: true
|
||||
_openapi:
|
||||
method: POST
|
||||
toc: []
|
||||
structuredData:
|
||||
headings: []
|
||||
contents:
|
||||
- content: Unload TTS model to free memory.
|
||||
---
|
||||
|
||||
{/* This file was generated by Fumadocs. Do not edit this file directly. Any changes should be made by running the generation command again. */}
|
||||
|
||||
<APIPage document={"./openapi.json"} operations={[{"path":"/models/unload","method":"post"}]} />
|
||||
@@ -0,0 +1,16 @@
|
||||
---
|
||||
title: Update Profile
|
||||
description: Update a voice profile.
|
||||
full: true
|
||||
_openapi:
|
||||
method: PUT
|
||||
toc: []
|
||||
structuredData:
|
||||
headings: []
|
||||
contents:
|
||||
- content: Update a voice profile.
|
||||
---
|
||||
|
||||
{/* This file was generated by Fumadocs. Do not edit this file directly. Any changes should be made by running the generation command again. */}
|
||||
|
||||
<APIPage document={"./openapi.json"} operations={[{"path":"/profiles/{profile_id}","method":"put"}]} />
|
||||
@@ -26,19 +26,22 @@ These two layers communicate via HTTP, with the frontend making API requests to
|
||||
|
||||
### Component Structure
|
||||
|
||||
```
|
||||
app/src/
|
||||
├── components/ # React components
|
||||
│ ├── profiles/ # Voice profile UI
|
||||
│ ├── generation/ # Speech generation UI
|
||||
│ ├── stories/ # Timeline editor
|
||||
│ └── shared/ # Reusable components
|
||||
├── lib/ # Utilities
|
||||
│ ├── api/ # Generated API client
|
||||
│ └── utils/ # Helper functions
|
||||
├── hooks/ # React hooks
|
||||
└── stores/ # Zustand state stores
|
||||
```
|
||||
<Files>
|
||||
<Folder name="app/src" defaultOpen>
|
||||
<Folder name="components">
|
||||
<File name="profiles/" />
|
||||
<File name="generation/" />
|
||||
<File name="stories/" />
|
||||
<File name="shared/" />
|
||||
</Folder>
|
||||
<Folder name="lib">
|
||||
<File name="api/" />
|
||||
<File name="utils/" />
|
||||
</Folder>
|
||||
<Folder name="hooks" />
|
||||
<Folder name="stores" />
|
||||
</Folder>
|
||||
</Files>
|
||||
|
||||
### State Management
|
||||
|
||||
@@ -64,16 +67,69 @@ const useProfileStore = create((set) => ({
|
||||
|
||||
### API Structure
|
||||
|
||||
```python
|
||||
# main.py - API routes
|
||||
@app.post("/generate")
|
||||
async def generate_speech(request: GenerateRequest):
|
||||
# 1. Validate request
|
||||
# 2. Load voice profile
|
||||
# 3. Generate audio with TTS
|
||||
# 4. Save to database
|
||||
# 5. Return response
|
||||
```
|
||||
<Files>
|
||||
<Folder name="backend" defaultOpen>
|
||||
<File name="app.py" />
|
||||
<File name="main.py" />
|
||||
<File name="config.py" />
|
||||
<File name="models.py" />
|
||||
<File name="server.py" />
|
||||
<Folder name="routes">
|
||||
<File name="profiles.py" />
|
||||
<File name="generate.py" />
|
||||
<File name="history.py" />
|
||||
<File name="..." />
|
||||
</Folder>
|
||||
<Folder name="services">
|
||||
<File name="generation.py" />
|
||||
<File name="task_queue.py" />
|
||||
<File name="..." />
|
||||
</Folder>
|
||||
<Folder name="backends">
|
||||
<File name="__init__.py" />
|
||||
<File name="base.py" />
|
||||
<File name="..." />
|
||||
</Folder>
|
||||
<Folder name="database">
|
||||
<File name="models.py" />
|
||||
<File name="session.py" />
|
||||
</Folder>
|
||||
<Folder name="utils">
|
||||
<File name="audio.py" />
|
||||
<File name="effects.py" />
|
||||
<File name="..." />
|
||||
</Folder>
|
||||
</Folder>
|
||||
</Files>
|
||||
|
||||
### Request Flow
|
||||
|
||||
HTTP request → **routes/** (validate input, parse params) → **services/** (business logic, orchestration) → **backends/** (TTS/STT inference) → **utils/** (audio processing)
|
||||
|
||||
Route handlers are intentionally thin. They validate input, delegate to a service function, and format the response. All business logic lives in `services/`.
|
||||
|
||||
### Key Modules
|
||||
|
||||
- **app.py** — FastAPI app factory, CORS, lifecycle events
|
||||
- **main.py** — Entry point (imports app, runs uvicorn)
|
||||
- **server.py** — Tauri sidecar launcher, parent-pid watchdog
|
||||
- **services/generation.py** — Single function handling all generation modes (generate, retry, regenerate)
|
||||
- **services/task_queue.py** — Serial generation queue for GPU inference
|
||||
- **backends/__init__.py** — Protocol definitions and backend factory
|
||||
- **backends/base.py** — Shared utilities across all engine implementations
|
||||
|
||||
### Backend Selection
|
||||
|
||||
The server detects the best inference backend at startup:
|
||||
|
||||
| Platform | Backend | Acceleration |
|
||||
|----------|---------|-------------|
|
||||
| macOS (Apple Silicon) | MLX | Metal / Neural Engine |
|
||||
| Windows / Linux (NVIDIA) | PyTorch | CUDA |
|
||||
| Linux (AMD) | PyTorch | ROCm |
|
||||
| Intel Arc | PyTorch | IPEX / XPU |
|
||||
| Windows (any GPU) | PyTorch | DirectML |
|
||||
| Any | PyTorch | CPU fallback |
|
||||
|
||||
### Data Model
|
||||
|
||||
@@ -89,11 +145,13 @@ The database uses three main tables:
|
||||
|
||||
### Rust Backend
|
||||
|
||||
```rust
|
||||
// Sidecar process management
|
||||
// File system access
|
||||
// Native integrations
|
||||
```
|
||||
<Files>
|
||||
<Folder name="tauri/src-tauri" defaultOpen>
|
||||
<File name="Cargo.toml" />
|
||||
<File name="src/" />
|
||||
<Folder name="binaries" />
|
||||
</Folder>
|
||||
</Files>
|
||||
|
||||
### Responsibilities
|
||||
|
||||
@@ -196,11 +254,11 @@ When a user generates speech, the data flows through the following stages:
|
||||
|
||||
## Next Steps
|
||||
|
||||
<CardGroup cols={2}>
|
||||
<Card title="Development Setup" icon="code" href="/development/setup">
|
||||
<Cards>
|
||||
<Card title="Development Setup" href="/development/setup">
|
||||
Set up your dev environment
|
||||
</Card>
|
||||
<Card title="Contributing" icon="code-pull-request" href="/development/contributing">
|
||||
<Card title="Contributing" href="/development/contributing">
|
||||
Contribute to Voicebox
|
||||
</Card>
|
||||
</CardGroup>
|
||||
</Cards>
|
||||
@@ -0,0 +1,226 @@
|
||||
---
|
||||
title: "Auto-Updater"
|
||||
description: "How Voicebox automatic updates work"
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
Voicebox uses Tauri's built-in auto-updater to deliver signed updates to users. The system verifies updates cryptographically before installation.
|
||||
|
||||
## How It Works
|
||||
|
||||
When Voicebox launches (in production Tauri builds only), it checks GitHub Releases for a `latest.json` manifest. If a newer version is available:
|
||||
|
||||
1. **Notification** - An update banner appears at the top of the app
|
||||
2. **Download** - User clicks "Install Now" to download the update package
|
||||
3. **Verification** - The downloaded package is cryptographically verified using the public key embedded in `tauri.conf.json`
|
||||
4. **Installation** - After verification, the update is installed
|
||||
5. **Restart** - The app restarts automatically with the new version
|
||||
|
||||
Users can also check for updates manually via **Settings → Check for Updates**.
|
||||
|
||||
## Configuration
|
||||
|
||||
The updater is configured in `tauri/src-tauri/tauri.conf.json`:
|
||||
|
||||
```json
|
||||
{
|
||||
"plugins": {
|
||||
"updater": {
|
||||
"active": true,
|
||||
"dialog": false,
|
||||
"endpoints": [
|
||||
"https://github.com/jamiepine/voicebox/releases/latest/download/latest.json"
|
||||
],
|
||||
"pubkey": "PASTE_PUBLIC_KEY_CONTENT_HERE"
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Key settings:**
|
||||
- `endpoints` - URL to the `latest.json` manifest (checked on app startup)
|
||||
- `pubkey` - Public key for verifying update signatures
|
||||
- `dialog` - Set to `false` (we use custom UI instead of Tauri's built-in dialog)
|
||||
|
||||
## Release Manifest
|
||||
|
||||
The `latest.json` file defines available updates per platform:
|
||||
|
||||
```json
|
||||
{
|
||||
"version": "0.2.0",
|
||||
"notes": "Bug fixes and improvements",
|
||||
"pub_date": "2026-01-25T12:00:00Z",
|
||||
"platforms": {
|
||||
"darwin-aarch64": {
|
||||
"signature": "base64_encoded_signature",
|
||||
"url": "https://github.com/jamiepine/voicebox/releases/download/v0.2.0/voicebox_0.2.0_aarch64.app.tar.gz"
|
||||
},
|
||||
"darwin-x86_64": {
|
||||
"signature": "base64_encoded_signature",
|
||||
"url": "https://github.com/jamiepine/voicebox/releases/download/v0.2.0/voicebox_0.2.0_x64.app.tar.gz"
|
||||
},
|
||||
"linux-x86_64": {
|
||||
"signature": "base64_encoded_signature",
|
||||
"url": "https://github.com/jamiepine/voicebox/releases/download/v0.2.0/voicebox_0.2.0_amd64.AppImage"
|
||||
},
|
||||
"windows-x86_64": {
|
||||
"signature": "base64_encoded_signature",
|
||||
"url": "https://github.com/jamiepine/voicebox/releases/download/v0.2.0/voicebox_0.2.0_x64_en-US.msi"
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Signing
|
||||
|
||||
Updates must be cryptographically signed to be accepted. The signing process:
|
||||
|
||||
1. **Generate keys** (one-time setup):
|
||||
```bash
|
||||
bun tauri signer generate -w ~/.tauri/voicebox.key
|
||||
```
|
||||
This creates:
|
||||
- Private key: `~/.tauri/voicebox.key` (stored in GitHub Secrets, never committed)
|
||||
- Public key: `~/.tauri/voicebox.key.pub` (pasted into `tauri.conf.json`)
|
||||
|
||||
2. **Build with signing** (GitHub Actions handles this):
|
||||
- Set `TAURI_SIGNING_PRIVATE_KEY` environment variable
|
||||
- Tauri signs the update package during build
|
||||
- Generates `.sig` signature file alongside the installer
|
||||
|
||||
3. **Verification** - The updater compares the signature against the public key before installing
|
||||
|
||||
## GitHub Actions Workflow
|
||||
|
||||
The release workflow (`.github/workflows/release.yml`) automatically:
|
||||
|
||||
- Builds signed releases for macOS, Windows, and Linux
|
||||
- Creates the `latest.json` manifest with signatures
|
||||
- Uploads everything to the GitHub Release
|
||||
|
||||
Triggered by pushing a git tag:
|
||||
|
||||
```bash
|
||||
git tag v0.2.0 && git push --tags
|
||||
```
|
||||
|
||||
## Environment Variables
|
||||
|
||||
GitHub Actions needs these secrets set:
|
||||
|
||||
- `TAURI_SIGNING_PRIVATE_KEY` - Content of `~/.tauri/voicebox.key`
|
||||
- `TAURI_SIGNING_PRIVATE_KEY_PASSWORD` - Password for the key (if set)
|
||||
|
||||
## Security
|
||||
|
||||
<Callout type="warn">
|
||||
**Critical:** Never commit the private key. Store it only in GitHub Secrets. The public key in `tauri.conf.json` is safe to commit and distribute.
|
||||
</Callout>
|
||||
|
||||
- Updates are cryptographically signed using Ed25519
|
||||
- HTTP endpoints are blocked (HTTPS only)
|
||||
- Signature verification happens before installation
|
||||
- Failed verification aborts the update
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### "Invalid signature" error
|
||||
- Public key in `tauri.conf.json` doesn't match the private key used to sign
|
||||
- Signature file wasn't uploaded to the release
|
||||
|
||||
### "No update available" when one exists
|
||||
- `latest.json` version isn't higher than current version
|
||||
- Wrong endpoint URL in configuration
|
||||
- Manifest hasn't propagated to GitHub's CDN yet
|
||||
|
||||
### Update check fails in dev mode
|
||||
The updater only works in production Tauri builds. It doesn't run during `just dev` or web mode.
|
||||
|
||||
### Build fails with signing error
|
||||
- GitHub Secrets aren't set correctly
|
||||
- Private key file is missing or corrupted
|
||||
- Key format is wrong (should start with `dW50cnVzdGVkIGNvbW1lbnQ6`)
|
||||
|
||||
## CUDA Backend Updates
|
||||
|
||||
The CUDA-enabled backend is distributed separately from the main app due to its large size (~2.43 GB). Unlike the Tauri auto-updater, this uses a custom download system built into the Python backend.
|
||||
|
||||
**Size comparison:**
|
||||
- Standard app bundle: ~410 MB
|
||||
- CUDA backend binary: ~2.43 GB (6× larger)
|
||||
|
||||
### Why Split?
|
||||
|
||||
GitHub Releases has file size limits, and the CUDA-enabled `voicebox-server` binary is too large to include in the main Tauri bundle. Instead:
|
||||
|
||||
- **Standard release**: Includes CPU-only backend (~50MB)
|
||||
- **CUDA release**: Split into multiple parts and downloaded on-demand by users who need GPU acceleration
|
||||
|
||||
### Download Process
|
||||
|
||||
When a user clicks "Enable CUDA" in the settings:
|
||||
|
||||
1. **Manifest Fetch** - Backend fetches `{version}/voicebox-server-cuda.manifest` from GitHub Releases
|
||||
2. **Part Download** - Downloads each split part sequentially (e.g., `voicebox-server-cuda.part1`, `.part2`, etc.)
|
||||
3. **Assembly** - Concatenates parts into a single binary
|
||||
4. **Verification** - SHA-256 checksum verification (optional, if `.sha256` file exists)
|
||||
5. **Placement** - Binary moved to `{data_dir}/backends/voicebox-server-cuda.exe`
|
||||
6. **Restart** - Backend must restart to use the CUDA binary
|
||||
|
||||
### Auto-Update on Startup
|
||||
|
||||
On server startup, `check_and_update_cuda_binary()` compares the installed CUDA binary version with the app version:
|
||||
|
||||
```python
|
||||
# backend/services/cuda.py
|
||||
cuda_version = get_cuda_binary_version() # runs `voicebox-server-cuda --version`
|
||||
current_version = __version__
|
||||
|
||||
if cuda_version != current_version:
|
||||
await download_cuda_binary() # Auto-download in background
|
||||
```
|
||||
|
||||
If versions mismatch, the backend automatically downloads the matching CUDA binary version without user intervention.
|
||||
|
||||
### Storage Location
|
||||
|
||||
Downloaded CUDA binaries are stored in the app's data directory:
|
||||
|
||||
```
|
||||
{data_dir}/
|
||||
backends/
|
||||
voicebox-server-cuda.exe # Windows
|
||||
voicebox-server-cuda # macOS/Linux
|
||||
```
|
||||
|
||||
### API Endpoints
|
||||
|
||||
| Endpoint | Method | Description |
|
||||
|----------|--------|-------------|
|
||||
| `/backend/cuda-status` | GET | Check if CUDA binary available/active |
|
||||
| `/backend/download-cuda` | POST | Start download |
|
||||
| `/backend/cuda-progress` | GET | SSE stream of download progress |
|
||||
| `/backend/cuda` | DELETE | Remove downloaded binary |
|
||||
|
||||
### Progress Tracking
|
||||
|
||||
Downloads report progress via Server-Sent Events (SSE):
|
||||
|
||||
```
|
||||
GET /backend/cuda-progress
|
||||
|
||||
event: progress
|
||||
data: {"current": 52428800, "total": 104857600, "filename": "Downloading CUDA backend (2/4)", "status": "downloading"}
|
||||
```
|
||||
|
||||
The frontend subscribes to this endpoint to show real-time download progress in the UI.
|
||||
|
||||
### Release Artifacts
|
||||
|
||||
For each release, these CUDA-related files are uploaded to GitHub:
|
||||
|
||||
- `voicebox-server-cuda.manifest` - List of split part filenames
|
||||
- `voicebox-server-cuda.part1` through `voicebox-server-cuda.partN` - Binary chunks
|
||||
- `voicebox-server-cuda.sha256` - SHA-256 checksum for integrity verification
|
||||
@@ -0,0 +1,190 @@
|
||||
---
|
||||
title: "Building"
|
||||
description: "How Voicebox is built for production"
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
Voicebox uses a two-stage build process:
|
||||
|
||||
1. **Python Server Binary** — PyInstaller bundles the FastAPI backend into a standalone executable
|
||||
2. **Tauri Desktop App** — Bundles the React frontend, Rust wrapper, and Python server as a sidecar
|
||||
|
||||
## Build Commands
|
||||
|
||||
```bash
|
||||
just build # Build everything (server + Tauri)
|
||||
just build-server # Build Python server binary only
|
||||
just build-tauri # Build Tauri app only
|
||||
```
|
||||
|
||||
## Server Binary Build
|
||||
|
||||
### Build Script
|
||||
|
||||
`scripts/build-server.sh` orchestrates the build:
|
||||
|
||||
```bash
|
||||
# Determine platform (e.g., x86_64-apple-darwin)
|
||||
PLATFORM=$(rustc --print host-tuple)
|
||||
|
||||
# Run PyInstaller via build_binary.py
|
||||
cd backend
|
||||
python build_binary.py
|
||||
|
||||
# Copy to Tauri's binaries directory
|
||||
cp dist/voicebox-server ../tauri/src-tauri/binaries/voicebox-server-${PLATFORM}
|
||||
```
|
||||
|
||||
### PyInstaller Configuration
|
||||
|
||||
`backend/build_binary.py` contains the PyInstaller configuration:
|
||||
|
||||
**Entry Point:** Uses `server.py` (not `main.py`) for Tauri sidecar support
|
||||
|
||||
**Key Options:**
|
||||
- `--onefile` — Single executable
|
||||
- `--hidden-import` — Explicitly import modules PyInstaller can't detect
|
||||
- `--collect-all` — Bundle data files and native libraries for packages like `mlx`, `zipvoice`
|
||||
- `--exclude-module` — Strip NVIDIA packages from CPU builds
|
||||
|
||||
**Platform-Specific Logic:**
|
||||
|
||||
```python
|
||||
# Apple Silicon — include MLX backend
|
||||
if is_apple_silicon() and not cuda:
|
||||
args.extend([
|
||||
"--hidden-import", "mlx",
|
||||
"--collect-all", "mlx", # Bundles .dylib and .metallib files
|
||||
])
|
||||
|
||||
# CUDA builds — include torch.cuda
|
||||
if cuda:
|
||||
args.extend(["--hidden-import", "torch.cuda"])
|
||||
|
||||
# CPU builds — exclude NVIDIA packages to save ~3GB
|
||||
else:
|
||||
for pkg in ["nvidia", "nvidia.cublas", "nvidia.cudnn", ...]:
|
||||
args.extend(["--exclude-module", pkg])
|
||||
```
|
||||
|
||||
**Environment Variable:**
|
||||
|
||||
```bash
|
||||
export QWEN_TTS_PATH=~/path/to/Qwen3-TTS # Use local Qwen3-TTS source
|
||||
```
|
||||
|
||||
### CUDA Binary
|
||||
|
||||
The CUDA-enabled server is built separately due to size (~2.43 GB vs ~410 MB CPU version):
|
||||
|
||||
```bash
|
||||
cd backend
|
||||
python build_binary.py --cuda
|
||||
```
|
||||
|
||||
The resulting binary is too large for GitHub Releases, so it's split into parts for distribution (see Auto-Updater docs for the download mechanism).
|
||||
|
||||
## Tauri App Build
|
||||
|
||||
Tauri bundles everything together:
|
||||
|
||||
```bash
|
||||
cd tauri
|
||||
bun run tauri build
|
||||
```
|
||||
|
||||
**What happens:**
|
||||
1. Vite builds the React frontend
|
||||
2. Rust compiles the Tauri wrapper
|
||||
3. Sidecar binary is copied from `src-tauri/binaries/`
|
||||
4. Platform-specific installer created (DMG, MSI, AppImage)
|
||||
|
||||
**Output locations:**
|
||||
|
||||
<Files>
|
||||
<Folder name="tauri/src-tauri/target/release/bundle" defaultOpen>
|
||||
<File name="dmg/" />
|
||||
<File name="msi/" />
|
||||
<File name="nsis/" />
|
||||
<File name="appimage/" />
|
||||
</Folder>
|
||||
</Files>
|
||||
|
||||
### Sidecar Configuration
|
||||
|
||||
The server binary is declared as an external binary in `tauri.conf.json`:
|
||||
|
||||
```json
|
||||
{
|
||||
"tauri": {
|
||||
"bundle": {
|
||||
"externalBin": ["binaries/voicebox-server"]
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Tauri looks for `voicebox-server-${PLATFORM}` in `src-tauri/binaries/` and bundles it.
|
||||
|
||||
## GitHub Actions Release
|
||||
|
||||
`.github/workflows/release.yml` automates the full build:
|
||||
|
||||
### Matrix Strategy
|
||||
|
||||
| Platform | Target | Backend | Notes |
|
||||
|----------|--------|---------|-------|
|
||||
| macos-latest | aarch64-apple-darwin | MLX | Apple Silicon native |
|
||||
| macos-15-intel | x86_64-apple-darwin | PyTorch | Intel Macs |
|
||||
| windows-latest | x86_64-pc-windows-msvc | PyTorch | Windows with CUDA optional |
|
||||
|
||||
### Build Steps
|
||||
|
||||
1. **Setup** — Python, Rust, Bun, dependencies
|
||||
2. **Build Server** — `build-server.sh` (Unix) or `build_binary.py` (Windows)
|
||||
3. **Build Tauri** — `tauri-action` with signing keys
|
||||
4. **Upload** — Release artifacts and `latest.json`
|
||||
|
||||
### Code Signing
|
||||
|
||||
**macOS:**
|
||||
- Apple Developer certificate imported from secrets
|
||||
- Notarization via App Store Connect API
|
||||
|
||||
**Windows:**
|
||||
- Tauri handles signing via `TAURI_SIGNING_PRIVATE_KEY`
|
||||
|
||||
### CUDA Binary (Separate Job)
|
||||
|
||||
The `build-cuda-windows` job runs separately:
|
||||
|
||||
1. Install PyTorch with CUDA 12.1
|
||||
2. Build with `build_binary.py --cuda`
|
||||
3. Split binary with `scripts/split_binary.py`
|
||||
4. Upload parts as release artifacts
|
||||
|
||||
This binary is downloaded on-demand by users who enable CUDA in settings.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
<AccordionGroup>
|
||||
<Accordion title="Binary not found in dist/">
|
||||
PyInstaller failed to create the output. Check:
|
||||
- Python venv is activated
|
||||
- All dependencies installed: `pip install -r requirements.txt`
|
||||
- PyInstaller installed: `pip install pyinstaller`
|
||||
</Accordion>
|
||||
|
||||
<Accordion title="MLX/Metal libraries missing in bundle">
|
||||
macOS Apple Silicon builds need `--collect-all mlx` to include `.dylib` and `.metallib` files, not just `--collect-data`.
|
||||
</Accordion>
|
||||
|
||||
<Accordion title="CUDA DLLs bloating CPU build">
|
||||
If building CPU version but CUDA torch is installed locally, the script auto-detects and swaps to CPU torch temporarily, then restores CUDA torch after.
|
||||
</Accordion>
|
||||
|
||||
<Accordion title="Tauri can't find sidecar">
|
||||
Ensure binary exists at `tauri/src-tauri/binaries/voicebox-server-${PLATFORM}` before running Tauri build.
|
||||
</Accordion>
|
||||
</AccordionGroup>
|
||||
@@ -23,20 +23,20 @@ Before you start contributing, make sure you have:
|
||||
|
||||
## Ways to Contribute
|
||||
|
||||
<CardGroup cols={2}>
|
||||
<Card title="Report Bugs" icon="bug">
|
||||
<Cards>
|
||||
<Card title="Report Bugs">
|
||||
Found a bug? Open an issue with reproduction steps
|
||||
</Card>
|
||||
<Card title="Request Features" icon="lightbulb">
|
||||
<Card title="Request Features">
|
||||
Have an idea? Start a discussion or open an issue
|
||||
</Card>
|
||||
<Card title="Improve Docs" icon="book">
|
||||
<Card title="Improve Docs">
|
||||
Fix typos, add examples, or clarify instructions
|
||||
</Card>
|
||||
<Card title="Write Code" icon="code">
|
||||
<Card title="Write Code">
|
||||
Fix bugs, add features, or optimize performance
|
||||
</Card>
|
||||
</CardGroup>
|
||||
</Cards>
|
||||
|
||||
## Development Workflow
|
||||
|
||||
@@ -164,27 +164,28 @@ When creating a pull request:
|
||||
|
||||
## Project Structure
|
||||
|
||||
Understanding the codebase:
|
||||
|
||||
```
|
||||
voicebox/
|
||||
├── app/ # Shared React frontend
|
||||
│ ├── src/
|
||||
│ │ ├── components/ # UI components
|
||||
│ │ ├── lib/ # Utilities and API client
|
||||
│ │ ├── hooks/ # React hooks
|
||||
│ │ └── stores/ # Zustand state stores
|
||||
├── backend/ # Python FastAPI server
|
||||
│ ├── main.py # API routes
|
||||
│ ├── tts.py # Voice synthesis logic
|
||||
│ ├── database.py # SQLite operations
|
||||
│ └── models.py # Pydantic models
|
||||
├── tauri/ # Desktop app wrapper
|
||||
│ └── src-tauri/ # Rust backend
|
||||
├── web/ # Web deployment
|
||||
├── landing/ # Marketing website
|
||||
└── scripts/ # Build & release scripts
|
||||
```
|
||||
<Files>
|
||||
<Folder name="voicebox" defaultOpen>
|
||||
<Folder name="app/src">
|
||||
<File name="components/" />
|
||||
<File name="lib/" />
|
||||
<File name="hooks/" />
|
||||
<File name="stores/" />
|
||||
</Folder>
|
||||
<Folder name="backend">
|
||||
<File name="main.py" />
|
||||
<File name="tts.py" />
|
||||
<File name="database.py" />
|
||||
<File name="models.py" />
|
||||
</Folder>
|
||||
<Folder name="tauri">
|
||||
<File name="src-tauri/" />
|
||||
</Folder>
|
||||
<File name="web/" />
|
||||
<File name="landing/" />
|
||||
<File name="scripts/" />
|
||||
</Folder>
|
||||
</Files>
|
||||
|
||||
## Areas for Contribution
|
||||
|
||||
@@ -259,7 +260,11 @@ When adding new API endpoints:
|
||||
</Step>
|
||||
|
||||
<Step title="Update Docs">
|
||||
Add documentation in `/docs/api/`
|
||||
The API documentation is automatically generated from the OpenAPI schema. Ensure your endpoint has proper docstrings and type hints, then regenerate the docs:
|
||||
|
||||
```bash
|
||||
bun run generate:api
|
||||
```
|
||||
</Step>
|
||||
</Steps>
|
||||
|
||||
@@ -0,0 +1,257 @@
|
||||
---
|
||||
title: "Effects Pipeline"
|
||||
description: "Audio post-processing effects and generation versioning"
|
||||
---
|
||||
|
||||
The effects pipeline provides professional-grade DSP audio processing using Spotify's Pedalboard library. Each generation can have multiple versions with different effect chains applied.
|
||||
|
||||
## Overview
|
||||
|
||||
**Key concepts:**
|
||||
|
||||
- **Effects Chain** — JSON-serializable list of effect configurations applied sequentially
|
||||
- **Generation Version** — A processed variant of a generation with its own audio file and effects chain
|
||||
- **Effect Preset** — Saved effects chain configuration (built-in or user-created)
|
||||
- **Clean Version** — The original unprocessed generation audio
|
||||
|
||||
**Flow:**
|
||||
|
||||
1. TTS Generation creates clean audio
|
||||
2. Effects Chain processes the audio
|
||||
3. Processed Version is saved as a new generation version
|
||||
|
||||
Each generation maintains a clean version (original) plus any number of processed versions with different effect chains applied.
|
||||
|
||||
## Effect Types
|
||||
|
||||
The following effect types are available, each with configurable parameters:
|
||||
|
||||
### Chorus / Flanger
|
||||
|
||||
Modulated delay effect. Short centre_delay_ms gives flanger; longer gives chorus.
|
||||
|
||||
**Parameters:**
|
||||
- rate_hz: LFO speed in Hz (range: 0.01 to 20, default: 1.0)
|
||||
- depth: Modulation depth (range: 0.0 to 1.0, default: 0.5)
|
||||
- feedback: Feedback amount (range: 0.0 to 0.95, default: 0.0)
|
||||
- centre_delay_ms: Centre delay in milliseconds (range: 0.5 to 50, default: 7.0)
|
||||
- mix: Wet/dry mix (range: 0.0 to 1.0, default: 0.5)
|
||||
|
||||
### Reverb
|
||||
|
||||
Room reverb effect.
|
||||
|
||||
**Parameters:**
|
||||
- room_size: Room size (range: 0.0 to 1.0, default: 0.5)
|
||||
- damping: High frequency damping (range: 0.0 to 1.0, default: 0.5)
|
||||
- wet_level: Wet level (range: 0.0 to 1.0, default: 0.33)
|
||||
- dry_level: Dry level (range: 0.0 to 1.0, default: 0.4)
|
||||
- width: Stereo width (range: 0.0 to 1.0, default: 1.0)
|
||||
|
||||
### Delay
|
||||
|
||||
Echo / delay line.
|
||||
|
||||
**Parameters:**
|
||||
- delay_seconds: Delay time in seconds (range: 0.01 to 2.0, default: 0.3)
|
||||
- feedback: Feedback amount (range: 0.0 to 0.95, default: 0.3)
|
||||
- mix: Wet/dry mix (range: 0.0 to 1.0, default: 0.3)
|
||||
|
||||
### Compressor
|
||||
|
||||
Dynamic range compression for consistent loudness.
|
||||
|
||||
**Parameters:**
|
||||
- threshold_db: Threshold in dB (range: -60 to 0, default: -20.0)
|
||||
- ratio: Compression ratio (range: 1.0 to 20.0, default: 4.0)
|
||||
- attack_ms: Attack time in ms (range: 0.1 to 100, default: 10.0)
|
||||
- release_ms: Release time in ms (range: 10 to 1000, default: 100.0)
|
||||
|
||||
### Gain
|
||||
|
||||
Volume adjustment in decibels.
|
||||
|
||||
**Parameters:**
|
||||
- gain_db: Gain in dB (range: -40 to 40, default: 0.0)
|
||||
|
||||
### High-Pass Filter
|
||||
|
||||
Removes frequencies below the cutoff.
|
||||
|
||||
**Parameters:**
|
||||
- cutoff_frequency_hz: Cutoff frequency in Hz (range: 20 to 8000, default: 80.0)
|
||||
|
||||
### Low-Pass Filter
|
||||
|
||||
Removes frequencies above the cutoff.
|
||||
|
||||
**Parameters:**
|
||||
- cutoff_frequency_hz: Cutoff frequency in Hz (range: 200 to 20000, default: 8000.0)
|
||||
|
||||
### Pitch Shift
|
||||
|
||||
Shift pitch up or down by semitones.
|
||||
|
||||
**Parameters:**
|
||||
- semitones: Semitones to shift (range: -12 to 12, default: 0.0)
|
||||
|
||||
## Generation Versions
|
||||
|
||||
Each generation starts with a clean version (no effects). Users can create processed versions by applying effect chains.
|
||||
|
||||
**Version properties:**
|
||||
- id — Unique version identifier
|
||||
- label — User-defined name (e.g., "robotic", "with reverb")
|
||||
- audio_path — Path to the processed audio file
|
||||
- effects_chain — JSON array of effect configurations
|
||||
- source_version_id — Which version this was derived from
|
||||
- is_default — Whether this is the default audio for the generation
|
||||
|
||||
**File storage:**
|
||||
|
||||
<Files>
|
||||
<Folder name="data/generations" defaultOpen>
|
||||
<File name="{generation_id}.wav" />
|
||||
<File name="{generation_id}_{version_id}.wav" />
|
||||
</Folder>
|
||||
</Files>
|
||||
|
||||
**Default version behavior:**
|
||||
- One version per generation is marked as default
|
||||
- The generation's audio_path always points to the default version's audio
|
||||
- Deleting the default version automatically promotes another version
|
||||
|
||||
## Effect Presets
|
||||
|
||||
Presets are saved effects chains that can be reused across generations.
|
||||
|
||||
**Built-in presets:**
|
||||
|
||||
- **Robotic**: Metallic robotic voice using chorus (flanger-style)
|
||||
- **Radio**: Thin AM-radio voice with band-pass filtering and light compression
|
||||
- **Echo Chamber**: Spacious reverb with trailing echo
|
||||
- **Deep Voice**: Lower pitch with added warmth using pitch shift and compression
|
||||
|
||||
**User presets:**
|
||||
- Created via the effects UI
|
||||
- Stored in the database (SQLite)
|
||||
- Cannot modify/delete built-in presets
|
||||
- Used to quickly apply favorite effect combinations
|
||||
|
||||
## API Endpoints
|
||||
|
||||
### Effects Management
|
||||
|
||||
| Endpoint | Method | Description |
|
||||
|----------|--------|-------------|
|
||||
| /effects/available | GET | List all effect types with parameter definitions |
|
||||
| /effects/presets | GET | List all presets (built-in + user) |
|
||||
| /effects/presets | POST | Create a new user preset |
|
||||
| /effects/presets/:id | GET | Get a specific preset |
|
||||
| /effects/presets/:id | PUT | Update a user preset |
|
||||
| /effects/presets/:id | DELETE | Delete a user preset |
|
||||
| /effects/preview/:generation_id | POST | Preview effects on a generation (returns audio stream) |
|
||||
|
||||
### Generation Versions
|
||||
|
||||
| Endpoint | Method | Description |
|
||||
|----------|--------|-------------|
|
||||
| /generations/:id/versions | GET | List all versions for a generation |
|
||||
| /generations/:id/versions/apply-effects | POST | Apply effects chain, create new version |
|
||||
| /generations/:id/versions/:version_id/set-default | PUT | Set a version as default |
|
||||
| /generations/:id/versions/:version_id | DELETE | Delete a version |
|
||||
|
||||
### Request Body: Apply Effects
|
||||
|
||||
Request body for applying effects:
|
||||
|
||||
- effects_chain: Array of effect objects
|
||||
- label: Version label (e.g., "with reverb")
|
||||
- set_as_default: Whether to set as default
|
||||
- source_version_id: Source version ID (optional)
|
||||
|
||||
## Implementation
|
||||
|
||||
### Backend Architecture
|
||||
|
||||
**Files:**
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| backend/utils/effects.py | Effect registry, validation, and audio processing |
|
||||
| backend/services/versions.py | Generation version CRUD operations |
|
||||
| backend/services/effects.py | Effect preset CRUD operations |
|
||||
| backend/routes/effects.py | API endpoints for effects and versions |
|
||||
|
||||
**Effect Registry:**
|
||||
|
||||
The EFFECT_REGISTRY dict in utils/effects.py defines all available effects with their parameters, defaults, and ranges.
|
||||
|
||||
**Validation:**
|
||||
|
||||
Effects chains are validated before application:
|
||||
- Each effect type must exist in the registry
|
||||
- Parameters must be numbers within min/max bounds
|
||||
- Unknown parameters are rejected
|
||||
|
||||
**Audio Processing:**
|
||||
|
||||
Uses Spotify's Pedalboard library:
|
||||
|
||||
```python
|
||||
from pedalboard import Pedalboard
|
||||
|
||||
# Build pedalboard from chain
|
||||
board = build_pedalboard(effects_chain)
|
||||
|
||||
# Apply to audio (async via thread)
|
||||
processed = await asyncio.to_thread(lambda: board(audio, sample_rate))
|
||||
```
|
||||
|
||||
### Frontend Integration
|
||||
|
||||
**Key components:**
|
||||
|
||||
| Component | Location |
|
||||
|-----------|----------|
|
||||
| Effects chain editor | app/src/components/Effects/ |
|
||||
| Version selector | Generation detail view |
|
||||
| Preset manager | Effects panel |
|
||||
| Live preview | Preview button (streams processed audio) |
|
||||
|
||||
**State management:**
|
||||
- Effects chains are stored as JSON arrays
|
||||
- Live preview fetches processed audio without saving
|
||||
- Applied effects create new versions via POST endpoint
|
||||
|
||||
## Adding New Effects
|
||||
|
||||
To add a new effect type:
|
||||
|
||||
1. **Add to registry** (backend/utils/effects.py):
|
||||
- Add entry to EFFECT_REGISTRY with cls, label, description, and params
|
||||
- Import the effect class from Pedalboard
|
||||
|
||||
2. **Update frontend types** if needed
|
||||
|
||||
The new effect automatically appears in /effects/available and the chain editor UI.
|
||||
|
||||
## Best Practices
|
||||
|
||||
**Effect ordering matters.** Process effects in this order for best results:
|
||||
1. Pitch shift (if needed)
|
||||
2. High/low-pass filters
|
||||
3. Chorus/flanger (time-based)
|
||||
4. Reverb/delay (spatial)
|
||||
5. Compressor
|
||||
6. Gain (final level adjustment)
|
||||
|
||||
**CPU usage:**
|
||||
- Effects are applied in real-time during generation
|
||||
- Pitch shift and reverb are the most CPU-intensive
|
||||
- Consider previewing complex chains before applying
|
||||
|
||||
**Storage:**
|
||||
- Each version creates a new audio file
|
||||
- Clean version always exists (can be reverted to)
|
||||
- Processed versions can be deleted to save space
|
||||
@@ -30,11 +30,13 @@ class Generation(Base):
|
||||
|
||||
Generated audio is stored in:
|
||||
|
||||
```
|
||||
data/
|
||||
└── generations/
|
||||
└── {generation_id}.wav
|
||||
```
|
||||
<Files>
|
||||
<Folder name="data" defaultOpen>
|
||||
<Folder name="generations">
|
||||
<File name="{generation_id}.wav" />
|
||||
</Folder>
|
||||
</Folder>
|
||||
</Files>
|
||||
|
||||
## Core Functions
|
||||
|
||||
@@ -171,11 +173,12 @@ async def delete_generations_by_profile(profile_id: str, db: Session) -> int:
|
||||
|
||||
Generations can be exported as ZIP archives:
|
||||
|
||||
```
|
||||
generation_export.zip
|
||||
├── generation.json # Metadata
|
||||
└── audio.wav # Audio file
|
||||
```
|
||||
<Files>
|
||||
<Folder name="generation_export.zip" defaultOpen>
|
||||
<File name="generation.json" />
|
||||
<File name="audio.wav" />
|
||||
</Folder>
|
||||
</Files>
|
||||
|
||||
### Importing a Generation
|
||||
|
||||
@@ -0,0 +1,20 @@
|
||||
{
|
||||
"title": "Developer",
|
||||
"defaultOpen": true,
|
||||
"pages": [
|
||||
"setup",
|
||||
"architecture",
|
||||
"contributing",
|
||||
"building",
|
||||
"autoupdater",
|
||||
"voice-profiles",
|
||||
"tts-generation",
|
||||
"tts-engines",
|
||||
"effects-pipeline",
|
||||
"history",
|
||||
"stories",
|
||||
"transcription",
|
||||
"audio-channels",
|
||||
"model-management"
|
||||
]
|
||||
}
|
||||
+7
-7
@@ -36,13 +36,13 @@ Models are downloaded from HuggingFace Hub on first use and cached locally.
|
||||
|
||||
Models are cached in the HuggingFace cache directory:
|
||||
|
||||
```
|
||||
~/.cache/huggingface/hub/
|
||||
├── models--Qwen--Qwen3-TTS-12Hz-1.7B-Base/
|
||||
├── models--Qwen--Qwen3-TTS-12Hz-0.6B-Base/
|
||||
├── models--openai--whisper-base/
|
||||
└── ...
|
||||
```
|
||||
<Files>
|
||||
<Folder name="~/.cache/huggingface/hub" defaultOpen>
|
||||
<File name="models--Qwen--Qwen3-TTS-12Hz-1.7B-Base/" />
|
||||
<File name="models--Qwen--Qwen3-TTS-12Hz-0.6B-Base/" />
|
||||
<File name="models--openai--whisper-base/" />
|
||||
</Folder>
|
||||
</Files>
|
||||
|
||||
## Progress Tracking
|
||||
|
||||
@@ -0,0 +1,248 @@
|
||||
---
|
||||
title: "Development Setup"
|
||||
description: "Set up your local development environment for Voicebox"
|
||||
---
|
||||
|
||||
## Quick Setup (Recommended)
|
||||
|
||||
Get started in two commands:
|
||||
|
||||
```bash
|
||||
# Clone and enter the repository
|
||||
git clone https://github.com/jamiepine/voicebox.git
|
||||
cd voicebox
|
||||
|
||||
# Setup everything (Python venv, JS deps, dev sidecar)
|
||||
just setup
|
||||
|
||||
# Start development (backend + desktop app)
|
||||
just dev
|
||||
```
|
||||
|
||||
The `just dev` command automatically starts the Python backend (if not already running) and launches the Tauri desktop app.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
Ensure you have these installed:
|
||||
|
||||
<Cards>
|
||||
<Card title="Bun" icon={<svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24" fill="none" stroke="currentColor" strokeWidth="2" strokeLinecap="round" strokeLinejoin="round"><path d="m7.5 4.27 9 5.15"/><path d="M21 8a2 2 0 0 0-1-1.73l-7-4a2 2 0 0 0-2 0l-7 4A2 2 0 0 0 3 8v8a2 2 0 0 0 1 1.73l7 4a2 2 0 0 0 2 0l7-4A2 2 0 0 0 21 16Z"/><path d="m3.3 7 8.7 5 8.7-5"/><path d="M12 22V12"/></svg>}>
|
||||
[Download Bun](https://bun.sh)
|
||||
```bash
|
||||
curl -fsSL https://bun.sh/install | bash
|
||||
```
|
||||
</Card>
|
||||
<Card title="Python 3.11+" icon={<svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24" fill="none" stroke="currentColor" strokeWidth="2" strokeLinecap="round" strokeLinejoin="round"><path d="M12 2L2 7l10 5 10-5-10-5z"/><path d="M2 17l10 5 10-5"/><path d="M2 12l10 5 10-5"/></svg>}>
|
||||
[Download Python](https://python.org)
|
||||
```bash
|
||||
python --version
|
||||
```
|
||||
</Card>
|
||||
<Card title="Rust" icon={<svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24" fill="none" stroke="currentColor" strokeWidth="2" strokeLinecap="round" strokeLinejoin="round"><path d="m6 9 6 6 6-6"/></svg>}>
|
||||
[Install Rust](https://rustup.rs)
|
||||
```bash
|
||||
rustc --version
|
||||
```
|
||||
</Card>
|
||||
<Card title="Just" icon={<svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24" fill="none" stroke="currentColor" strokeWidth="2" strokeLinecap="round" strokeLinejoin="round"><path d="M4 7V4h3"/><path d="M7 4h14v6h-2V6H7V4Z"/><path d="M4 10v10h16V10H4Z"/></svg>}>
|
||||
[Install Just](https://github.com/casey/just)
|
||||
```bash
|
||||
brew install just # macOS
|
||||
cargo install just # Linux/Windows
|
||||
```
|
||||
</Card>
|
||||
</Cards>
|
||||
|
||||
<Callout type="info">
|
||||
Just works on macOS, Linux, and Windows.
|
||||
</Callout>
|
||||
|
||||
## Just Commands
|
||||
|
||||
Run `just --list` to see all available commands:
|
||||
|
||||
| Command | Description |
|
||||
|---------|-------------|
|
||||
| `just setup` | Full setup (Python venv + JS deps) |
|
||||
| `just dev` | Start backend + desktop app |
|
||||
| `just dev-web` | Start backend + web app (no Tauri) |
|
||||
| `just dev-backend` | Start backend only |
|
||||
| `just dev-frontend` | Start desktop app only (backend must be running) |
|
||||
| `just build` | Build desktop app for production |
|
||||
| `just build-web` | Build web app for production |
|
||||
| `just check` | Run all checks (JS + Python lint + format) |
|
||||
| `just fix` | Fix lint + format issues |
|
||||
| `just test` | Run Python tests |
|
||||
| `just db-init` | Initialize SQLite database |
|
||||
| `just db-reset` | Reset database (delete + reinit) |
|
||||
| `just clean` | Clean build artifacts |
|
||||
| `just clean-all` | Nuclear clean (includes node_modules) |
|
||||
|
||||
## Project Structure
|
||||
|
||||
<Files>
|
||||
<Folder name="voicebox" defaultOpen>
|
||||
<Folder name="app">
|
||||
<Folder name="src">
|
||||
<File name="components/" />
|
||||
<File name="lib/" />
|
||||
<File name="hooks/" />
|
||||
</Folder>
|
||||
</Folder>
|
||||
<Folder name="backend">
|
||||
<File name="app.py" />
|
||||
<File name="main.py" />
|
||||
<File name="config.py" />
|
||||
<File name="models.py" />
|
||||
<File name="server.py" />
|
||||
<Folder name="routes">
|
||||
<File name="..." />
|
||||
</Folder>
|
||||
<Folder name="services">
|
||||
<File name="..." />
|
||||
</Folder>
|
||||
<Folder name="backends">
|
||||
<File name="..." />
|
||||
</Folder>
|
||||
<Folder name="database">
|
||||
<File name="..." />
|
||||
</Folder>
|
||||
<Folder name="utils">
|
||||
<File name="..." />
|
||||
</Folder>
|
||||
</Folder>
|
||||
<Folder name="tauri">
|
||||
<Folder name="src-tauri" />
|
||||
</Folder>
|
||||
<Folder name="web" />
|
||||
<Folder name="scripts" />
|
||||
</Folder>
|
||||
</Files>
|
||||
|
||||
### Request Flow
|
||||
|
||||
HTTP request → **routes/** (validate input) → **services/** (business logic) → **backends/** (TTS/STT inference) → **utils/** (audio processing)
|
||||
|
||||
### Key Modules
|
||||
|
||||
- **app.py** — FastAPI app factory, CORS, lifecycle events
|
||||
- **main.py** — Entry point (imports app, runs uvicorn)
|
||||
- **server.py** — Tauri sidecar launcher, parent-pid watchdog
|
||||
- **services/generation.py** — Single function handling all generation modes
|
||||
- **backends/** — TTS/STT engine implementations (MLX, PyTorch, etc.)
|
||||
|
||||
## Model Downloads
|
||||
|
||||
Models are automatically downloaded from HuggingFace Hub on first use:
|
||||
|
||||
- **Whisper** (transcription): Auto-downloads on first transcription
|
||||
- **Qwen3-TTS** (voice cloning): Auto-downloads on first generation (~2-4GB)
|
||||
|
||||
<Callout type="warn">
|
||||
First-time usage will be slower due to model downloads, but subsequent runs will use cached models.
|
||||
</Callout>
|
||||
|
||||
## Generate OpenAPI Client
|
||||
|
||||
After starting the backend server, generate the TypeScript API client:
|
||||
|
||||
```bash
|
||||
just generate-api
|
||||
```
|
||||
|
||||
This downloads the OpenAPI schema and generates the TypeScript client in `app/src/lib/api/`
|
||||
|
||||
## Manual Setup (Advanced)
|
||||
|
||||
If you prefer not to use Just, follow these manual steps:
|
||||
|
||||
### 1. Install JavaScript Dependencies
|
||||
|
||||
```bash
|
||||
bun install
|
||||
```
|
||||
|
||||
This installs dependencies for:
|
||||
- `app/` - Shared React frontend
|
||||
- `tauri/` - Tauri desktop wrapper
|
||||
- `web/` - Web deployment wrapper
|
||||
|
||||
### 2. Set Up Python Backend
|
||||
|
||||
```bash
|
||||
cd backend
|
||||
|
||||
# Create virtual environment
|
||||
python -m venv venv
|
||||
|
||||
# Activate virtual environment
|
||||
source venv/bin/activate # macOS/Linux
|
||||
# or
|
||||
venv\Scripts\activate # Windows
|
||||
|
||||
# Install Python dependencies
|
||||
pip install -r requirements.txt
|
||||
|
||||
# Apple Silicon: install MLX dependencies
|
||||
pip install -r requirements-mlx.txt
|
||||
|
||||
# Install Qwen3-TTS
|
||||
pip install git+https://github.com/QwenLM/Qwen3-TTS.git
|
||||
```
|
||||
|
||||
### 3. Start Development
|
||||
|
||||
Start the backend:
|
||||
```bash
|
||||
cd backend
|
||||
source venv/bin/activate
|
||||
uvicorn main:app --reload --port 17493
|
||||
```
|
||||
|
||||
In a new terminal, start the desktop app:
|
||||
```bash
|
||||
cd tauri
|
||||
bun run tauri dev
|
||||
```
|
||||
|
||||
## Next Steps
|
||||
|
||||
<Cards>
|
||||
<Card title="Architecture" href="/development/architecture">
|
||||
Understand the system architecture
|
||||
</Card>
|
||||
<Card title="Contributing" href="/development/contributing">
|
||||
Read the contribution guidelines
|
||||
</Card>
|
||||
<Card title="Building" href="/development/building">
|
||||
Learn how to build production releases
|
||||
</Card>
|
||||
<Card title="API Reference" href="/api-reference">
|
||||
Explore the REST API
|
||||
</Card>
|
||||
</Cards>
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
<AccordionGroup>
|
||||
<Accordion title="Backend won't start">
|
||||
- Check Python version (must be 3.11+)
|
||||
- Ensure virtual environment is activated: `source backend/venv/bin/activate`
|
||||
- Verify all dependencies are installed: `pip install -r requirements.txt`
|
||||
- Check if port 17493 is available
|
||||
</Accordion>
|
||||
|
||||
<Accordion title="Tauri build fails">
|
||||
- Ensure Rust is installed: `rustc --version`
|
||||
- Clean the build: `cd tauri/src-tauri && cargo clean`
|
||||
- Try rebuilding: `just dev`
|
||||
</Accordion>
|
||||
|
||||
<Accordion title="OpenAPI client generation fails">
|
||||
- Ensure backend is running: `curl http://localhost:17493/openapi.json`
|
||||
- Check network connectivity
|
||||
- Verify the backend is accessible at localhost:17493
|
||||
</Accordion>
|
||||
</AccordionGroup>
|
||||
|
||||
See the full [Troubleshooting Guide](/overview/troubleshooting) for more issues and solutions.
|
||||
@@ -0,0 +1,257 @@
|
||||
---
|
||||
title: "TTS Engines"
|
||||
description: "How to add new text-to-speech engines to Voicebox"
|
||||
---
|
||||
|
||||
Adding an engine touches ~10 files across 4 layers. The backend protocol work is straightforward — the real time sink is dependency hell, upstream library bugs, and PyInstaller bundling.
|
||||
|
||||
## Architecture Overview
|
||||
|
||||
The backend is split into layers:
|
||||
|
||||
| Layer | Purpose | Files Touched |
|
||||
|-------|---------|---------------|
|
||||
| `routes/` | Thin HTTP handlers | None (auto-dispatch) |
|
||||
| `services/` | Business logic | None (auto-dispatch) |
|
||||
| `backends/` | Engine implementations | `your_engine_backend.py` |
|
||||
| `utils/` | Shared utilities | As needed |
|
||||
|
||||
New engines only need to touch `backends/` and `models.py` on the backend side — the route and service layers use a model config registry that handles dispatch automatically.
|
||||
|
||||
## Phase 1: Backend Implementation
|
||||
|
||||
### 1.1 Create the Backend File
|
||||
|
||||
Create `backend/backends/<engine>_backend.py` (~200-300 lines) implementing the `TTSBackend` protocol:
|
||||
|
||||
```python
|
||||
class YourBackend:
|
||||
"""Must satisfy the TTSBackend protocol."""
|
||||
|
||||
async def load_model(self, model_size: str = "default") -> None: ...
|
||||
async def create_voice_prompt(self, audio_path: str, reference_text: str, use_cache: bool = True) -> tuple[dict, bool]: ...
|
||||
async def combine_voice_prompts(self, audio_paths: list[str], ref_texts: list[str]) -> tuple[np.ndarray, str]: ...
|
||||
async def generate(self, text: str, voice_prompt: dict, language: str = "en", seed: int | None = None, instruct: str | None = None) -> tuple[np.ndarray, int]: ...
|
||||
def unload_model(self) -> None: ...
|
||||
def is_loaded(self) -> bool: ...
|
||||
def _get_model_path(self, model_size: str) -> str: ...
|
||||
```
|
||||
|
||||
**Key decisions per engine:**
|
||||
|
||||
| Decision | Options | Examples |
|
||||
|----------|---------|---------|
|
||||
| **Voice prompt storage** | Pre-computed tensors vs deferred file paths | Qwen stores tensor dicts; Chatterbox stores paths |
|
||||
| **Caching** | Use voice prompt cache or skip it | LuxTTS caches with prefix; Chatterbox skips caching |
|
||||
| **Device selection** | CUDA / MPS / CPU | Chatterbox forces CPU on macOS (MPS bugs) |
|
||||
| **Model download** | Library handles it vs manual `snapshot_download` | Turbo uses manual download to bypass `token=True` bug |
|
||||
| **Sample rate** | Engine-specific | LuxTTS outputs 48kHz, everything else is 24kHz |
|
||||
|
||||
### 1.2 Voice Prompt Patterns
|
||||
|
||||
**Pattern A: Pre-computed tensors** (Qwen, LuxTTS)
|
||||
```python
|
||||
encoded = model.encode_prompt(audio_path)
|
||||
return encoded, False # (prompt_dict, was_cached)
|
||||
```
|
||||
|
||||
**Pattern B: Deferred file paths** (Chatterbox, MLX)
|
||||
```python
|
||||
return {"ref_audio": audio_path, "ref_text": reference_text}, False
|
||||
```
|
||||
|
||||
**Pattern C: Hybrid** (possible for new engines)
|
||||
```python
|
||||
embedding = model.extract_speaker(audio_path)
|
||||
return {"embedding": embedding, "ref_audio": audio_path}, False
|
||||
```
|
||||
|
||||
If caching, prefix your cache keys:
|
||||
```python
|
||||
cache_key = "yourengine_" + get_cache_key(audio_path, reference_text)
|
||||
```
|
||||
|
||||
### 1.3 Register the Engine
|
||||
|
||||
In `backend/backends/__init__.py`:
|
||||
|
||||
**Add a `ModelConfig` entry:**
|
||||
|
||||
```python
|
||||
ModelConfig(
|
||||
model_name="your-engine",
|
||||
display_name="Your Engine",
|
||||
engine="your_engine",
|
||||
hf_repo_id="org/model-repo",
|
||||
size_mb=3200,
|
||||
needs_trim=False, # set True if output needs trim_tts_output()
|
||||
languages=["en", "fr", "de"],
|
||||
),
|
||||
```
|
||||
|
||||
**Add to `TTS_ENGINES` dict:**
|
||||
|
||||
```python
|
||||
TTS_ENGINES = {
|
||||
...
|
||||
"your_engine": "Your Engine",
|
||||
}
|
||||
```
|
||||
|
||||
**Add factory branch:**
|
||||
|
||||
```python
|
||||
elif engine == "your_engine":
|
||||
from .your_backend import YourBackend
|
||||
backend = YourBackend()
|
||||
```
|
||||
|
||||
### 1.4 Update Request Models
|
||||
|
||||
In `backend/models.py`:
|
||||
- Add engine name to `GenerationRequest.engine` regex pattern
|
||||
- Add any new language codes to the language regex
|
||||
|
||||
## Phase 2: Route and Service Integration
|
||||
|
||||
With the model config registry, route and service layers have **zero per-engine dispatch points**. All endpoints use registry helpers like `get_model_config()`, `load_engine_model()`, `engine_needs_trim()`, `check_model_loaded()`, etc.
|
||||
|
||||
**You don't need to touch any route or service files** unless your engine needs custom behavior in the generate pipeline.
|
||||
|
||||
### Post-Processing
|
||||
|
||||
If your model produces trailing silence, set `needs_trim=True` on your `ModelConfig`. The generation service applies `trim_tts_output()` automatically.
|
||||
|
||||
## Phase 3: Frontend Integration
|
||||
|
||||
### 3.1 TypeScript Types
|
||||
|
||||
In `app/src/lib/api/types.ts`:
|
||||
- Add to the `engine` union type on `GenerationRequest`
|
||||
|
||||
### 3.2 Language Maps
|
||||
|
||||
In `app/src/lib/constants/languages.ts`:
|
||||
- Add entry to `ENGINE_LANGUAGES` record
|
||||
- Add any new language codes to `ALL_LANGUAGES` if needed
|
||||
|
||||
### 3.3 Engine/Model Selector
|
||||
|
||||
In `app/src/components/Generation/EngineModelSelector.tsx`:
|
||||
- Add entry to `ENGINE_OPTIONS` and `ENGINE_DESCRIPTIONS`
|
||||
- Add to `ENGLISH_ONLY_ENGINES` if applicable
|
||||
|
||||
### 3.4 Form Hook
|
||||
|
||||
In `app/src/lib/hooks/useGenerationForm.ts`:
|
||||
- Add to Zod schema enum for `engine`
|
||||
- Add engine-to-model-name mapping
|
||||
- Update payload construction for engine-specific fields
|
||||
|
||||
### 3.5 Model Management
|
||||
|
||||
In `app/src/components/ServerSettings/ModelManagement.tsx`:
|
||||
- Add description to `MODEL_DESCRIPTIONS` record
|
||||
|
||||
## Phase 4: Dependencies
|
||||
|
||||
### 4.1 Python Dependencies
|
||||
|
||||
Add to `backend/requirements.txt`. Watch for:
|
||||
|
||||
**Pinned dependency conflicts** — If the model package pins old versions, install with `--no-deps`:
|
||||
```bash
|
||||
pip install --no-deps chatterbox-tts
|
||||
```
|
||||
|
||||
Then list sub-dependencies manually in `requirements.txt`.
|
||||
|
||||
**Non-PyPI packages:**
|
||||
```
|
||||
linacodec @ git+https://github.com/user/repo.git
|
||||
```
|
||||
|
||||
**Custom package indexes:**
|
||||
```
|
||||
--find-links https://k2-fsa.github.io/icefall/piper_phonemize.html
|
||||
```
|
||||
|
||||
### 4.2 Identifying Hidden Sub-Dependencies
|
||||
|
||||
1. Install the package normally in a throwaway venv
|
||||
2. Run `pip show <package>` to get its `Requires:` list
|
||||
3. Cross-reference against existing requirements.txt
|
||||
4. Test that the engine loads and generates
|
||||
|
||||
## Phase 5: PyInstaller Bundling
|
||||
|
||||
This is where most of the pain lives. Common issues:
|
||||
|
||||
| Issue | Symptom | Fix |
|
||||
|-------|---------|-----|
|
||||
| `inspect.getsource()` at import | "could not get source code" | `--collect-all <package>` |
|
||||
| Data files (yaml, .pth.tar) | FileNotFoundError at runtime | `--collect-all <package>` |
|
||||
| Native data paths (espeak-ng) | Library looks at `/usr/share/...` | Set env var in frozen builds |
|
||||
| `importlib.metadata` lookups | "No package metadata found" | `--copy-metadata <package>` |
|
||||
| Dynamic imports | ModuleNotFoundError | `--hidden-import <module>` |
|
||||
|
||||
### Testing Frozen Builds
|
||||
|
||||
You can't skip this. Models that work in `python -m uvicorn` will break in the PyInstaller binary.
|
||||
|
||||
1. Build: `just build`
|
||||
2. Run and try download + load + generate
|
||||
3. Check stderr for the actual error
|
||||
4. Fix, rebuild, repeat
|
||||
|
||||
## Phase 6: Common Upstream Workarounds
|
||||
|
||||
### torch.load device mismatch
|
||||
```python
|
||||
_original_torch_load = torch.load
|
||||
def _patched_torch_load(*args, **kwargs):
|
||||
kwargs.setdefault("map_location", "cpu")
|
||||
return _original_torch_load(*args, **kwargs)
|
||||
torch.load = _patched_torch_load
|
||||
```
|
||||
|
||||
### Float64/Float32 dtype mismatch
|
||||
```python
|
||||
original_fn = SomeClass.some_method
|
||||
def patched_fn(self, *args, **kwargs):
|
||||
result = original_fn(self, *args, **kwargs)
|
||||
return result.float()
|
||||
SomeClass.some_method = patched_fn
|
||||
```
|
||||
|
||||
### HuggingFace token bug
|
||||
```python
|
||||
from huggingface_hub import snapshot_download
|
||||
local_path = snapshot_download(repo_id=REPO, token=None)
|
||||
model = ModelClass.from_local(local_path, device=device)
|
||||
```
|
||||
|
||||
### MPS tensor issues
|
||||
Skip MPS entirely if operators aren't supported:
|
||||
```python
|
||||
def _get_device(self):
|
||||
if torch.cuda.is_available():
|
||||
return "cuda"
|
||||
return "cpu" # Skip MPS
|
||||
```
|
||||
|
||||
## Upcoming Engines
|
||||
|
||||
Based on the current model landscape, these are candidates for future integration:
|
||||
|
||||
| Model | Languages | Size | Key Features | Status |
|
||||
|-------|-----------|------|--------------|--------|
|
||||
| **CosyVoice2-0.5B** | Multilingual | ~500MB | Instruct support (`inference_instruct2()`) | Ready |
|
||||
| **Fish Speech** | 50+ | Medium | Word-level control via inline text | Ready |
|
||||
| **Kokoro-82M** | English | 82M | CPU realtime, Apache 2.0 | Ready |
|
||||
| **XTTS-v2** | 17+ | Medium | Zero-shot cloning | Ready |
|
||||
| **HumeAI TADA** | EN (1B), Multi (3B) | Medium | 700s+ coherent audio, synced transcripts | Needs vetting |
|
||||
| **MOSS-TTS** | Multilingual | Medium | Text-to-voice design, multi-speaker dialogue | Needs vetting |
|
||||
| **Pocket TTS** | English | ~100M | CPU-first, >1× realtime | Needs vetting |
|
||||
|
||||
The multi-engine architecture is now in place, making new model integration straightforward (~1 day for a well-documented model with a PyPI package).
|
||||
+19
-16
@@ -49,14 +49,16 @@ class ProfileSample(Base):
|
||||
|
||||
Profiles are stored in the data directory:
|
||||
|
||||
```
|
||||
data/
|
||||
└── profiles/
|
||||
└── {profile_id}/
|
||||
├── {sample_id_1}.wav
|
||||
├── {sample_id_2}.wav
|
||||
└── ...
|
||||
```
|
||||
<Files>
|
||||
<Folder name="data" defaultOpen>
|
||||
<Folder name="profiles">
|
||||
<Folder name="{profile_id}">
|
||||
<File name="{sample_id_1}.wav" />
|
||||
<File name="{sample_id_2}.wav" />
|
||||
</Folder>
|
||||
</Folder>
|
||||
</Folder>
|
||||
</Files>
|
||||
|
||||
## Core Functions
|
||||
|
||||
@@ -158,14 +160,15 @@ Reference audio is validated before being accepted:
|
||||
|
||||
Profiles can be exported as ZIP archives for sharing:
|
||||
|
||||
```
|
||||
profile_export.zip
|
||||
├── profile.json # Metadata
|
||||
├── samples/
|
||||
│ ├── sample_1.wav
|
||||
│ └── sample_1.json # Reference text
|
||||
└── ...
|
||||
```
|
||||
<Files>
|
||||
<Folder name="profile_export.zip" defaultOpen>
|
||||
<File name="profile.json" />
|
||||
<Folder name="samples">
|
||||
<File name="sample_1.wav" />
|
||||
<File name="sample_1.json" />
|
||||
</Folder>
|
||||
</Folder>
|
||||
</Files>
|
||||
|
||||
## API Endpoints
|
||||
|
||||
@@ -0,0 +1,36 @@
|
||||
---
|
||||
title: "Voicebox Documentation"
|
||||
description: "Voicebox is a local-first voice cloning studio -- a free and open-source alternative to ElevenLabs."
|
||||
---
|
||||
|
||||
Voicebox is a **local-first voice cloning studio** -- a free and open-source alternative to ElevenLabs. Clone voices from a few seconds of audio, generate speech in 23 languages across 4 TTS engines, apply post-processing effects, and compose multi-voice projects with a timeline editor.
|
||||
|
||||

|
||||
|
||||
- **Complete privacy** -- models and voice data stay on your machine
|
||||
- **4 TTS engines** -- Qwen3-TTS, LuxTTS, Chatterbox Multilingual, and Chatterbox Turbo
|
||||
- **23 languages** -- from English to Arabic, Japanese, Hindi, Swahili, and more
|
||||
- **Post-processing effects** -- pitch shift, reverb, delay, chorus, compression, and filters
|
||||
- **Expressive speech** -- paralinguistic tags like `[laugh]`, `[sigh]`, `[gasp]` via Chatterbox Turbo
|
||||
- **Unlimited length** -- auto-chunking with crossfade for scripts, articles, and chapters
|
||||
- **Stories editor** -- multi-track timeline for conversations, podcasts, and narratives
|
||||
- **API-first** -- REST API for integrating voice synthesis into your own projects
|
||||
- **Native performance** -- built with Tauri (Rust), not Electron
|
||||
- **Runs everywhere** -- macOS (MLX/Metal), Windows (CUDA), Linux, AMD ROCm, Intel Arc, Docker
|
||||
|
||||
## Download
|
||||
|
||||
| Platform | Download |
|
||||
|----------|----------|
|
||||
| macOS (Apple Silicon) | [Download DMG](https://voicebox.sh/download/mac-arm) |
|
||||
| macOS (Intel) | [Download DMG](https://voicebox.sh/download/mac-intel) |
|
||||
| Windows | [Download MSI](https://voicebox.sh/download/windows) |
|
||||
| Docker | `docker compose up` |
|
||||
|
||||
[View all releases](https://github.com/jamiepine/voicebox/releases/latest)
|
||||
|
||||
## Get Started
|
||||
|
||||
- [Installation](/docs/overview/installation) -- download and install Voicebox
|
||||
- [Quick Start](/docs/overview/quick-start) -- get up and running in 5 minutes
|
||||
- [API Reference](/docs/api-reference) -- integrate voice synthesis into your apps
|
||||
@@ -0,0 +1,4 @@
|
||||
{
|
||||
"title": "Voicebox Documentation",
|
||||
"pages": ["overview", "api-reference", "developer"]
|
||||
}
|
||||
+20
-20
@@ -28,32 +28,32 @@ Voice profiles are the foundation of voice cloning in Voicebox. This guide cover
|
||||
|
||||
### Ideal Sample Characteristics
|
||||
|
||||
<CardGroup cols={2}>
|
||||
<Card title="Duration" icon="clock">
|
||||
<Cards>
|
||||
<Card title="Duration">
|
||||
**10-30 seconds**
|
||||
|
||||
Too short: Poor quality
|
||||
Too long: Unnecessary
|
||||
</Card>
|
||||
<Card title="Clarity" icon="volume">
|
||||
<Card title="Clarity">
|
||||
**Clear speech**
|
||||
|
||||
No background noise
|
||||
No music or overlapping voices
|
||||
</Card>
|
||||
<Card title="Quality" icon="sparkles">
|
||||
<Card title="Quality">
|
||||
**High fidelity**
|
||||
|
||||
44.1kHz or 48kHz sample rate
|
||||
Minimal compression
|
||||
</Card>
|
||||
<Card title="Content" icon="microphone">
|
||||
<Card title="Content">
|
||||
**Natural speech**
|
||||
|
||||
Conversational tone
|
||||
Complete sentences
|
||||
</Card>
|
||||
</CardGroup>
|
||||
</Cards>
|
||||
|
||||
### File Formats
|
||||
|
||||
@@ -63,9 +63,9 @@ Supported formats:
|
||||
- **M4A** - Acceptable
|
||||
- **FLAC** - Lossless alternative
|
||||
|
||||
<Tip>
|
||||
<Callout type="info">
|
||||
Use WAV for best results. Avoid heavily compressed formats.
|
||||
</Tip>
|
||||
</Callout>
|
||||
|
||||
## Recording Tips
|
||||
|
||||
@@ -108,20 +108,20 @@ Adding multiple samples can significantly improve quality:
|
||||
|
||||
### Why Multiple Samples?
|
||||
|
||||
<CardGroup cols={2}>
|
||||
<Card title="Robustness" icon="shield">
|
||||
<Cards>
|
||||
<Card title="Robustness">
|
||||
Model learns a more complete representation
|
||||
</Card>
|
||||
<Card title="Versatility" icon="palette">
|
||||
<Card title="Versatility">
|
||||
Handles different speaking styles better
|
||||
</Card>
|
||||
<Card title="Quality" icon="star">
|
||||
<Card title="Quality">
|
||||
Reduces artifacts and improves naturalness
|
||||
</Card>
|
||||
<Card title="Consistency" icon="check">
|
||||
<Card title="Consistency">
|
||||
More reliable across different texts
|
||||
</Card>
|
||||
</CardGroup>
|
||||
</Cards>
|
||||
|
||||
### Sample Variety
|
||||
|
||||
@@ -144,9 +144,9 @@ Consider adding samples with:
|
||||
- Phone call quality (if needed)
|
||||
- Room acoustics
|
||||
|
||||
<Warning>
|
||||
<Callout type="warn">
|
||||
All samples should be from the **same speaker**. Mixing voices will produce poor results.
|
||||
</Warning>
|
||||
</Callout>
|
||||
|
||||
## Processing Existing Audio
|
||||
|
||||
@@ -286,11 +286,11 @@ The emotional tone of samples affects generation:
|
||||
|
||||
## Next Steps
|
||||
|
||||
<CardGroup cols={2}>
|
||||
<Card title="Generate Speech" icon="waveform" href="/guides/generating-speech">
|
||||
<Cards>
|
||||
<Card title="Generate Speech" href="/overview/generating-speech">
|
||||
Use your profile to generate speech
|
||||
</Card>
|
||||
<Card title="Build Stories" icon="film" href="/guides/building-stories">
|
||||
<Card title="Build Stories" href="/overview/building-stories">
|
||||
Create multi-voice narratives
|
||||
</Card>
|
||||
</CardGroup>
|
||||
</Cards>
|
||||
+2
-2
@@ -43,9 +43,9 @@ Use formatting to suggest emphasis:
|
||||
- Bold for strong emphasis: "This is **very** important"
|
||||
```
|
||||
|
||||
<Note>
|
||||
<Callout type="info">
|
||||
The model interprets these hints but results may vary.
|
||||
</Note>
|
||||
</Callout>
|
||||
|
||||
## Advanced Features
|
||||
|
||||
+12
-12
@@ -9,20 +9,20 @@ Voicebox keeps a complete history of all generated audio, making it easy to find
|
||||
|
||||
## Features
|
||||
|
||||
<CardGroup cols={2}>
|
||||
<Card title="Full History" icon="clock">
|
||||
<Cards>
|
||||
<Card title="Full History" icon={<svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24" fill="none" stroke="currentColor" strokeWidth="2" strokeLinecap="round" strokeLinejoin="round"><circle cx="12" cy="12" r="10"/><polyline points="12 6 12 12 16 14"/></svg>}>
|
||||
Every generation is automatically saved
|
||||
</Card>
|
||||
<Card title="Search & Filter" icon="search">
|
||||
<Card title="Search & Filter" icon={<svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24" fill="none" stroke="currentColor" strokeWidth="2" strokeLinecap="round" strokeLinejoin="round"><circle cx="11" cy="11" r="8"/><path d="m21 21-4.3-4.3"/></svg>}>
|
||||
Find by text, voice, or date
|
||||
</Card>
|
||||
<Card title="Re-generate" icon="rotate">
|
||||
<Card title="Re-generate" icon={<svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24" fill="none" stroke="currentColor" strokeWidth="2" strokeLinecap="round" strokeLinejoin="round"><path d="M3 12a9 9 0 0 1 9-9 9.75 9.75 0 0 1 6.74 2.74L21 8"/><path d="M21 3v5h-5"/><path d="M21 12a9 9 0 0 1-9 9 9.75 9.75 0 0 1-6.74-2.74L3 16"/><path d="M8 16H3v5"/></svg>}>
|
||||
Regenerate any past generation with one click
|
||||
</Card>
|
||||
<Card title="Export" icon="download">
|
||||
<Card title="Export" icon={<svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24" fill="none" stroke="currentColor" strokeWidth="2" strokeLinecap="round" strokeLinejoin="round"><path d="M21 15v4a2 2 0 0 1-2 2H5a2 2 0 0 1-2-2v-4"/><polyline points="7 10 12 15 17 10"/><line x1="12" y1="15" x2="12" y2="3"/></svg>}>
|
||||
Download individual or batch exports
|
||||
</Card>
|
||||
</CardGroup>
|
||||
</Cards>
|
||||
|
||||
## Viewing History
|
||||
|
||||
@@ -54,20 +54,20 @@ Drag generations to the Stories Editor timeline.
|
||||
|
||||
## Search & Filter
|
||||
|
||||
<Tabs>
|
||||
<Tab title="By Text">
|
||||
<Tabs items={["By Text", "By Voice", "By Date"]}>
|
||||
<Tab value="By Text">
|
||||
Search for specific text content
|
||||
```
|
||||
"Hello world"
|
||||
```
|
||||
</Tab>
|
||||
<Tab title="By Voice">
|
||||
<Tab value="By Voice">
|
||||
Filter by voice profile
|
||||
```
|
||||
Select from dropdown
|
||||
```
|
||||
</Tab>
|
||||
<Tab title="By Date">
|
||||
<Tab value="By Date">
|
||||
Filter by date range
|
||||
```
|
||||
Last 7 days, Last 30 days, Custom range
|
||||
@@ -83,6 +83,6 @@ History is stored locally:
|
||||
- **Windows**: `%APPDATA%/com.voicebox.app/data/`
|
||||
- **Linux**: `~/.config/com.voicebox.app/data/`
|
||||
|
||||
<Warning>
|
||||
<Callout type="warn">
|
||||
Deleting the data directory will remove all history. Export important files first.
|
||||
</Warning>
|
||||
</Callout>
|
||||
@@ -7,19 +7,19 @@ description: "Download and install Voicebox on macOS, Windows, or Linux"
|
||||
|
||||
Voicebox is available for macOS and Windows, with Linux builds coming soon.
|
||||
|
||||
<CardGroup cols={2}>
|
||||
<Card title="macOS" icon="apple">
|
||||
<Cards>
|
||||
<Card title="macOS" icon={<svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24" fill="none" stroke="currentColor" strokeWidth="2" strokeLinecap="round" strokeLinejoin="round"><path d="M12 2c-1.5 0-2.8.4-3.9 1.1A5.5 5.5 0 0 0 4 2.5C2.5 2.5 1 4 1 6c0 3.5 2.5 6 5 7.5C5 16 4 18 4 20c0 1.5.5 2.5 1.5 3C6.5 23.5 8 24 9.5 24c2 0 3.5-.5 5-2 1.5 1.5 3 2 5 2 1.5 0 3-.5 4-1 1-.5 1.5-1.5 1.5-3 0-2-1-4-2-6.5 2.5-1.5 5-4 5-7.5 0-2-1.5-3.5-3-3.5-.9 0-2.1.4-3.1 1.1A6.5 6.5 0 0 0 12 2Z"/></svg>}>
|
||||
Download for Apple Silicon or Intel Macs
|
||||
</Card>
|
||||
<Card title="Windows" icon="windows">
|
||||
<Card title="Windows" icon={<svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24" fill="none" stroke="currentColor" strokeWidth="2" strokeLinecap="round" strokeLinejoin="round"><rect x="2" y="3" width="20" height="14" rx="2" ry="2"/><line x1="8" y1="21" x2="16" y2="21"/><line x1="12" y1="17" x2="12" y2="21"/></svg>}>
|
||||
Download MSI installer or Setup executable
|
||||
</Card>
|
||||
</CardGroup>
|
||||
</Cards>
|
||||
|
||||
### macOS
|
||||
|
||||
<Tabs>
|
||||
<Tab title="Apple Silicon">
|
||||
<Tabs items={["Apple Silicon", "Intel"]}>
|
||||
<Tab value="Apple Silicon">
|
||||
Download: [voicebox_aarch64.app.tar.gz](https://github.com/jamiepine/voicebox/releases/latest/download/voicebox_aarch64.app.tar.gz)
|
||||
|
||||
```bash
|
||||
@@ -30,7 +30,7 @@ Voicebox is available for macOS and Windows, with Linux builds coming soon.
|
||||
mv Voicebox.app /Applications/
|
||||
```
|
||||
</Tab>
|
||||
<Tab title="Intel">
|
||||
<Tab value="Intel">
|
||||
Download: [voicebox_x64.app.tar.gz](https://github.com/jamiepine/voicebox/releases/latest/download/voicebox_x64.app.tar.gz)
|
||||
|
||||
```bash
|
||||
@@ -45,13 +45,13 @@ Voicebox is available for macOS and Windows, with Linux builds coming soon.
|
||||
|
||||
### Windows
|
||||
|
||||
<Tabs>
|
||||
<Tab title="MSI Installer">
|
||||
<Tabs items={["MSI Installer", "Setup Executable"]}>
|
||||
<Tab value="MSI Installer">
|
||||
Download: [voicebox_x64_en-US.msi](https://github.com/jamiepine/voicebox/releases/latest/download/voicebox_x64_en-US.msi)
|
||||
|
||||
Double-click the MSI file and follow the installation wizard.
|
||||
</Tab>
|
||||
<Tab title="Setup Executable">
|
||||
<Tab value="Setup Executable">
|
||||
Download: [voicebox_x64-setup.exe](https://github.com/jamiepine/voicebox/releases/latest/download/voicebox_x64-setup.exe)
|
||||
|
||||
Run the executable and follow the installation wizard.
|
||||
@@ -60,9 +60,9 @@ Voicebox is available for macOS and Windows, with Linux builds coming soon.
|
||||
|
||||
### Linux
|
||||
|
||||
<Note>
|
||||
<Callout type="info">
|
||||
Linux builds are coming soon. Currently blocked by GitHub runner disk space limitations.
|
||||
</Note>
|
||||
</Callout>
|
||||
|
||||
## First Launch
|
||||
|
||||
@@ -76,9 +76,9 @@ When you launch Voicebox for the first time:
|
||||
|
||||
3. **Backend Server** — The bundled Python server starts automatically
|
||||
|
||||
<Tip>
|
||||
<Callout type="info">
|
||||
First generation will be slower due to model downloads. Subsequent runs use cached models.
|
||||
</Tip>
|
||||
</Callout>
|
||||
|
||||
## System Requirements
|
||||
|
||||
@@ -95,9 +95,9 @@ When you launch Voicebox for the first time:
|
||||
- **GPU:** CUDA-capable NVIDIA GPU (for faster generation)
|
||||
- **Storage:** 10GB+ free space
|
||||
|
||||
<Note>
|
||||
<Callout type="info">
|
||||
CPU inference is supported but significantly slower than GPU. A CUDA-capable GPU is highly recommended for real-time workflows.
|
||||
</Note>
|
||||
</Callout>
|
||||
|
||||
## Verification
|
||||
|
||||
@@ -108,12 +108,12 @@ After installation, verify everything works:
|
||||
3. Navigate to **Profiles** and create a test profile
|
||||
4. Generate a short audio clip to verify the TTS engine works
|
||||
|
||||
<Check>
|
||||
<Callout type="success">
|
||||
If you see a green status indicator and can generate audio, you're all set!
|
||||
</Check>
|
||||
</Callout>
|
||||
|
||||
## Next Steps
|
||||
|
||||
<Card title="Quick Start Guide" icon="rocket" href="/overview/quick-start">
|
||||
<Card title="Quick Start Guide" href="/overview/quick-start">
|
||||
Create your first voice profile and generate speech
|
||||
</Card>
|
||||
@@ -0,0 +1,64 @@
|
||||
---
|
||||
title: "Introduction"
|
||||
description: "Voicebox is a local-first voice cloning studio -- a free and open-source alternative to ElevenLabs."
|
||||
---
|
||||
|
||||
## What is Voicebox?
|
||||
|
||||
Voicebox is a **local-first voice cloning studio** -- a free and open-source alternative to ElevenLabs. Clone voices from a few seconds of audio, generate speech in 23 languages across 4 TTS engines, apply post-processing effects, and compose multi-voice projects with a timeline editor.
|
||||
|
||||
- **Complete privacy** -- models and voice data stay on your machine
|
||||
- **4 TTS engines** -- Qwen3-TTS, LuxTTS, Chatterbox Multilingual, and Chatterbox Turbo
|
||||
- **23 languages** -- from English to Arabic, Japanese, Hindi, Swahili, and more
|
||||
- **Post-processing effects** -- pitch shift, reverb, delay, chorus, compression, and filters
|
||||
- **Expressive speech** -- paralinguistic tags like `[laugh]`, `[sigh]`, `[gasp]` via Chatterbox Turbo
|
||||
- **Unlimited length** -- auto-chunking with crossfade for scripts, articles, and chapters
|
||||
- **Stories editor** -- multi-track timeline for conversations, podcasts, and narratives
|
||||
- **API-first** -- REST API for integrating voice synthesis into your own projects
|
||||
- **Native performance** -- built with Tauri (Rust), not Electron
|
||||
- **Runs everywhere** -- macOS (MLX/Metal), Windows (CUDA), Linux, AMD ROCm, Intel Arc, Docker
|
||||
|
||||
## TTS Engines
|
||||
|
||||
Four engines with different strengths, switchable per-generation:
|
||||
|
||||
| Engine | Languages | Strengths |
|
||||
|--------|-----------|-----------|
|
||||
| **Qwen3-TTS** (0.6B / 1.7B) | 10 | High-quality multilingual cloning, delivery instructions |
|
||||
| **LuxTTS** | English | Lightweight (~1GB VRAM), 48kHz output, 150x realtime on CPU |
|
||||
| **Chatterbox Multilingual** | 23 | Broadest language coverage |
|
||||
| **Chatterbox Turbo** | English | Fast 350M model with paralinguistic emotion/sound tags |
|
||||
|
||||
## GPU Support
|
||||
|
||||
| Platform | Backend | Notes |
|
||||
|----------|---------|-------|
|
||||
| macOS (Apple Silicon) | MLX (Metal) | 4-5x faster via Neural Engine |
|
||||
| Windows / Linux (NVIDIA) | PyTorch (CUDA) | Auto-downloads CUDA binary from within the app |
|
||||
| Linux (AMD) | PyTorch (ROCm) | Auto-configures HSA_OVERRIDE_GFX_VERSION |
|
||||
| Windows (any GPU) | DirectML | Universal Windows GPU support |
|
||||
| Intel Arc | IPEX/XPU | Intel discrete GPU acceleration |
|
||||
| Any | CPU | Works everywhere, just slower |
|
||||
|
||||
## Use Cases
|
||||
|
||||
- **Game development** -- generate dynamic dialogue for characters
|
||||
- **Content creation** -- produce podcasts and video voiceovers
|
||||
- **Accessibility** -- build text-to-speech tools for users who need them
|
||||
- **Voice assistants** -- create custom voice interfaces
|
||||
- **Production pipelines** -- automate voiceover workflows via the REST API
|
||||
|
||||
## Tech Stack
|
||||
|
||||
| Layer | Technology |
|
||||
|-------|------------|
|
||||
| Desktop App | Tauri (Rust) |
|
||||
| Frontend | React, TypeScript, Tailwind CSS |
|
||||
| State | Zustand, React Query |
|
||||
| Backend | FastAPI (Python) |
|
||||
| TTS Engines | Qwen3-TTS, LuxTTS, Chatterbox, Chatterbox Turbo |
|
||||
| Effects | Pedalboard (Spotify) |
|
||||
| Transcription | Whisper / Whisper Turbo (PyTorch or MLX) |
|
||||
| Inference | MLX (Apple Silicon) / PyTorch (CUDA/ROCm/XPU/CPU) |
|
||||
| Database | SQLite |
|
||||
| Audio | WaveSurfer.js, librosa |
|
||||
@@ -0,0 +1,18 @@
|
||||
{
|
||||
"title": "Overview",
|
||||
"defaultOpen": true,
|
||||
"pages": [
|
||||
"introduction",
|
||||
"installation",
|
||||
"quick-start",
|
||||
"voice-cloning",
|
||||
"stories-editor",
|
||||
"recording-transcription",
|
||||
"generation-history",
|
||||
"remote-mode",
|
||||
"creating-voice-profiles",
|
||||
"generating-speech",
|
||||
"building-stories",
|
||||
"troubleshooting"
|
||||
]
|
||||
}
|
||||
@@ -46,9 +46,9 @@ Voice profiles are the foundation of Voicebox. Each profile contains voice sampl
|
||||
</Step>
|
||||
</Steps>
|
||||
|
||||
<Tip>
|
||||
<Callout type="info">
|
||||
For best results, use clean audio with minimal background noise and consistent speaking tone.
|
||||
</Tip>
|
||||
</Callout>
|
||||
|
||||
## Step 2: Generate Speech
|
||||
|
||||
@@ -74,9 +74,9 @@ Now let's use your new voice profile to generate speech.
|
||||
<Step title="Generate">
|
||||
Click **Generate** and wait a few seconds
|
||||
|
||||
<Note>
|
||||
<Callout type="info">
|
||||
First generation may take longer due to model initialization. Subsequent generations will be faster.
|
||||
</Note>
|
||||
</Callout>
|
||||
</Step>
|
||||
|
||||
<Step title="Play & Download">
|
||||
@@ -114,20 +114,20 @@ The Stories Editor lets you create multi-voice narratives with a timeline-based
|
||||
|
||||
## What's Next?
|
||||
|
||||
<CardGroup cols={2}>
|
||||
<Card title="Voice Cloning Guide" icon="microphone" href="/guides/creating-voice-profiles">
|
||||
<Cards>
|
||||
<Card title="Voice Cloning Guide" href="/overview/creating-voice-profiles">
|
||||
Learn advanced techniques for high-quality voice cloning
|
||||
</Card>
|
||||
<Card title="API Integration" icon="code" href="/api/overview">
|
||||
<Card title="API Integration" href="/api-reference">
|
||||
Integrate Voicebox into your own applications
|
||||
</Card>
|
||||
<Card title="Stories Editor" icon="film" href="/overview/stories-editor">
|
||||
<Card title="Stories Editor" href="/overview/stories-editor">
|
||||
Master the multi-track timeline editor
|
||||
</Card>
|
||||
<Card title="Remote Mode" icon="server" href="/overview/remote-mode">
|
||||
<Card title="Remote Mode" href="/overview/remote-mode">
|
||||
Connect to a GPU server for faster generation
|
||||
</Card>
|
||||
</CardGroup>
|
||||
</Cards>
|
||||
|
||||
## Tips for Success
|
||||
|
||||
@@ -149,6 +149,6 @@ The Stories Editor lets you create multi-voice narratives with a timeline-based
|
||||
- **Server won't start:** Check if port 17493 is available
|
||||
- **Poor audio quality:** Try adding more voice samples
|
||||
- **Slow generation:** Verify GPU acceleration is enabled
|
||||
- See the full [Troubleshooting Guide](/guides/troubleshooting) for more
|
||||
- See the full [Troubleshooting Guide](/overview/troubleshooting) for more
|
||||
</Accordion>
|
||||
</AccordionGroup>
|
||||
+2
-2
@@ -59,6 +59,6 @@ Automatic speech-to-text powered by OpenAI's Whisper model.
|
||||
</Step>
|
||||
</Steps>
|
||||
|
||||
<Tip>
|
||||
<Callout type="info">
|
||||
Transcription is useful for creating voice samples from existing audio or generating subtitles.
|
||||
</Tip>
|
||||
</Callout>
|
||||
@@ -41,9 +41,9 @@ In Remote Mode, the Voicebox desktop app (running on your local machine) communi
|
||||
uvicorn main:app --host 0.0.0.0 --port 17493
|
||||
```
|
||||
|
||||
<Warning>
|
||||
<Callout type="warn">
|
||||
This exposes the server to your network. Use a firewall or VPN for security.
|
||||
</Warning>
|
||||
</Callout>
|
||||
</Step>
|
||||
|
||||
<Step title="Open Firewall">
|
||||
@@ -108,9 +108,9 @@ In Remote Mode, the Voicebox desktop app (running on your local machine) communi
|
||||
|
||||
## Security Considerations
|
||||
|
||||
<Warning>
|
||||
<Callout type="warn">
|
||||
The API currently has no authentication. Only use on trusted networks or with a VPN.
|
||||
</Warning>
|
||||
</Callout>
|
||||
|
||||
**Best Practices:**
|
||||
- Use a VPN (WireGuard, Tailscale) instead of exposing to the internet
|
||||
@@ -129,9 +129,9 @@ Expected performance on various GPUs:
|
||||
| RTX 3060 | ~5-7s per 10 words |
|
||||
| CPU (12-core) | ~20-30s per 10 words |
|
||||
|
||||
<Tip>
|
||||
<Callout type="info">
|
||||
A GPU with 8GB+ VRAM is recommended for best performance.
|
||||
</Tip>
|
||||
</Callout>
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
@@ -9,20 +9,20 @@ The Stories Editor is a DAW-like timeline interface for creating multi-voice nar
|
||||
|
||||
## Features
|
||||
|
||||
<CardGroup cols={2}>
|
||||
<Card title="Multi-Track Timeline" icon="timeline">
|
||||
<Cards>
|
||||
<Card title="Multi-Track Timeline">
|
||||
Arrange multiple voice tracks in parallel
|
||||
</Card>
|
||||
<Card title="Inline Editing" icon="scissors">
|
||||
<Card title="Inline Editing">
|
||||
Trim and split clips directly in the timeline
|
||||
</Card>
|
||||
<Card title="Auto-Playback" icon="play">
|
||||
<Card title="Auto-Playback">
|
||||
Preview with synchronized playhead
|
||||
</Card>
|
||||
<Card title="Voice Mixing" icon="users">
|
||||
<Card title="Voice Mixing">
|
||||
Build conversations with multiple speakers
|
||||
</Card>
|
||||
</CardGroup>
|
||||
</Cards>
|
||||
|
||||
## Creating a Story
|
||||
|
||||
@@ -25,9 +25,9 @@ Windows SmartScreen may warn that the app is unrecognized.
|
||||
- Click "More info"
|
||||
- Click "Run anyway"
|
||||
|
||||
<Note>
|
||||
<Callout type="info">
|
||||
This is expected for unsigned applications. We're working on code signing for future releases.
|
||||
</Note>
|
||||
</Callout>
|
||||
|
||||
## Server Issues
|
||||
|
||||
@@ -163,9 +163,9 @@ This is expected behavior. The first generation downloads the Qwen3-TTS model (~
|
||||
|
||||
Settings → Generation → Use CPU instead of GPU
|
||||
|
||||
<Warning>
|
||||
<Callout type="warn">
|
||||
CPU generation is 5-10x slower but uses system RAM instead of VRAM.
|
||||
</Warning>
|
||||
</Callout>
|
||||
</Accordion>
|
||||
|
||||
<Accordion title="Reduce Batch Size">
|
||||
@@ -322,9 +322,9 @@ bun run tauri build
|
||||
|
||||
**Solutions:**
|
||||
|
||||
<Warning>
|
||||
<Callout type="warn">
|
||||
This will delete all your voice profiles and generation history. Export important profiles first if possible.
|
||||
</Warning>
|
||||
</Callout>
|
||||
|
||||
```bash
|
||||
# macOS
|
||||
@@ -28,20 +28,20 @@ Voicebox uses **Qwen3-TTS** from Alibaba to achieve near-perfect voice cloning f
|
||||
|
||||
### Sample Quality
|
||||
|
||||
<CardGroup cols={2}>
|
||||
<Card title="Do" icon="check">
|
||||
<Cards>
|
||||
<Card title="Do">
|
||||
- Use 10-30 seconds of audio
|
||||
- Clear, consistent speaking
|
||||
- Minimal background noise
|
||||
- Natural speaking pace
|
||||
</Card>
|
||||
<Card title="Don't" icon="xmark">
|
||||
<Card title="Don't">
|
||||
- Very short clips (< 5 seconds)
|
||||
- Heavy background noise
|
||||
- Music or overlapping voices
|
||||
- Heavily processed audio
|
||||
</Card>
|
||||
</CardGroup>
|
||||
</Cards>
|
||||
|
||||
### Multiple Samples
|
||||
|
||||
@@ -51,9 +51,9 @@ Adding multiple samples from the same speaker can improve quality:
|
||||
- Different emotions (happy, serious)
|
||||
- Different recording conditions
|
||||
|
||||
<Tip>
|
||||
<Callout type="info">
|
||||
The model will learn a more robust representation from diverse samples.
|
||||
</Tip>
|
||||
</Callout>
|
||||
|
||||
## Supported Languages
|
||||
|
||||
@@ -65,9 +65,9 @@ More languages coming soon.
|
||||
|
||||
## Limitations
|
||||
|
||||
<Warning>
|
||||
<Callout type="warn">
|
||||
Voice cloning should only be used with consent. Ensure you have permission to clone someone's voice.
|
||||
</Warning>
|
||||
</Callout>
|
||||
|
||||
- Quality depends on sample clarity
|
||||
- Works best with consistent speaking tone
|
||||
@@ -1,15 +0,0 @@
|
||||
/* Anchor hover styles */
|
||||
.nav-anchor:hover {
|
||||
@apply text-[#BF9E40];
|
||||
}
|
||||
|
||||
/* Icon wrapper on hover */
|
||||
.nav-anchor:hover div {
|
||||
background: #BF9E40 !important;
|
||||
filter: brightness(1) !important;
|
||||
}
|
||||
|
||||
/* Icon SVG on hover */
|
||||
.nav-anchor:hover svg {
|
||||
@apply bg-white !important;
|
||||
}
|
||||
@@ -1,84 +0,0 @@
|
||||
---
|
||||
title: "Auto-Updater"
|
||||
description: "Configure and use the Tauri auto-updater"
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
Voicebox uses Tauri's built-in auto-updater to deliver updates to users automatically.
|
||||
|
||||
## Quick Reference
|
||||
|
||||
For detailed setup instructions, see the existing documentation:
|
||||
|
||||
- [AUTOUPDATER_QUICKSTART.md](https://github.com/jamiepine/voicebox/blob/main/docs/AUTOUPDATER_QUICKSTART.md)
|
||||
- [AUTOUPDATER.md](https://github.com/jamiepine/voicebox/blob/main/docs/AUTOUPDATER.md)
|
||||
|
||||
## How It Works
|
||||
|
||||
The auto-updater follows a secure update process:
|
||||
|
||||
1. **Check for Updates** - The Voicebox app periodically checks GitHub Releases for new versions
|
||||
2. **Download Update** - If a new version is found, the update package is downloaded
|
||||
3. **Verify Signature** - The downloaded package is cryptographically verified using the public key
|
||||
4. **Install** - After verification, the update is installed
|
||||
5. **Restart** - The app restarts with the new version
|
||||
|
||||
## Configuration
|
||||
|
||||
Updates are configured in `tauri/src-tauri/tauri.conf.json`:
|
||||
|
||||
```json
|
||||
{
|
||||
"updater": {
|
||||
"active": true,
|
||||
"endpoints": [
|
||||
"https://github.com/jamiepine/voicebox/releases/latest/download/latest.json"
|
||||
],
|
||||
"dialog": true,
|
||||
"pubkey": "YOUR_PUBLIC_KEY"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Generating Keys
|
||||
|
||||
```bash
|
||||
# Generate signing keys
|
||||
bun run generate:keys
|
||||
|
||||
# Keys saved to ~/.tauri/voicebox.key
|
||||
```
|
||||
|
||||
<Warning>
|
||||
Keep your private key secure! Never commit it to the repository.
|
||||
</Warning>
|
||||
|
||||
## Release Process
|
||||
|
||||
1. **Bump version** using bumpversion
|
||||
2. **Push tag** to trigger CI/CD
|
||||
3. **GitHub Actions** builds and signs releases
|
||||
4. **Users** receive update notification
|
||||
|
||||
## User Experience
|
||||
|
||||
When an update is available:
|
||||
|
||||
1. User sees a notification dialog
|
||||
2. User clicks "Update"
|
||||
3. Update downloads in background
|
||||
4. App restarts with new version
|
||||
|
||||
## For Developers
|
||||
|
||||
See the full documentation files for:
|
||||
|
||||
- Setting up signing keys
|
||||
- Configuring GitHub releases
|
||||
- Testing updates locally
|
||||
- Troubleshooting update failures
|
||||
|
||||
<Card title="View Full Docs" href="https://github.com/jamiepine/voicebox/tree/main/docs">
|
||||
Access AUTOUPDATER.md and AUTOUPDATER_QUICKSTART.md in the repository
|
||||
</Card>
|
||||
@@ -1,270 +0,0 @@
|
||||
---
|
||||
title: "Building"
|
||||
description: "Build Voicebox for production"
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
Voicebox uses a multi-step build process to create platform-specific installers.
|
||||
|
||||
## Quick Build
|
||||
|
||||
```bash
|
||||
# Build for your current platform (automatically builds server binary first)
|
||||
make build
|
||||
|
||||
# Or manually
|
||||
bun run build
|
||||
```
|
||||
|
||||
This automatically:
|
||||
1. Builds the Python server binary (`bun run build:server`)
|
||||
2. Builds the Tauri app (`cd tauri && bun run tauri build`)
|
||||
|
||||
## Build Process
|
||||
|
||||
The build process consists of two steps, but `bun run build` handles both automatically:
|
||||
|
||||
### 1. Server Binary Build (Automatic)
|
||||
|
||||
The Python backend is compiled into a standalone executable using PyInstaller. This happens automatically when you run `bun run build`.
|
||||
|
||||
**Platform-specific binaries:**
|
||||
- macOS (Apple Silicon): `voicebox-server-aarch64-apple-darwin` (includes MLX backend)
|
||||
- macOS (Intel): `voicebox-server-x86_64-apple-darwin` (PyTorch backend)
|
||||
- Windows: `voicebox-server-x86_64-pc-windows-msvc.exe` (PyTorch backend)
|
||||
- Linux: `voicebox-server-x86_64-unknown-linux-gnu` (PyTorch backend)
|
||||
|
||||
<Note>
|
||||
The build script automatically detects your platform and includes the appropriate backend (MLX for Apple Silicon, PyTorch for others).
|
||||
</Note>
|
||||
|
||||
**Manual build (if needed):**
|
||||
```bash
|
||||
bun run build:server
|
||||
```
|
||||
|
||||
### 2. Tauri App Build (Automatic)
|
||||
|
||||
The Tauri app build is also handled automatically, which:
|
||||
1. Builds the React frontend (Vite)
|
||||
2. Compiles the Rust backend
|
||||
3. Bundles the server binary as a sidecar
|
||||
4. Creates platform-specific installers
|
||||
|
||||
**Manual build (if needed):**
|
||||
```bash
|
||||
cd tauri && bun run tauri build
|
||||
```
|
||||
|
||||
### 3. Output
|
||||
|
||||
Installers are created in `tauri/src-tauri/target/release/bundle/`:
|
||||
|
||||
**macOS:**
|
||||
- `dmg/` - Disk image installer
|
||||
- `macos/` - App bundle
|
||||
|
||||
**Windows:**
|
||||
- `msi/` - MSI installer
|
||||
- `nsis/` - NSIS installer
|
||||
|
||||
**Linux:**
|
||||
- `deb/` - Debian package
|
||||
- `appimage/` - AppImage
|
||||
|
||||
## Advanced Options
|
||||
|
||||
### Building for Specific Platform
|
||||
|
||||
```bash
|
||||
# Build for macOS (Apple Silicon)
|
||||
bun run tauri build -- --target aarch64-apple-darwin
|
||||
|
||||
# Build for macOS (Intel)
|
||||
bun run tauri build -- --target x86_64-apple-darwin
|
||||
|
||||
# Build for Windows
|
||||
bun run tauri build -- --target x86_64-pc-windows-msvc
|
||||
|
||||
# Build for Linux
|
||||
bun run tauri build -- --target x86_64-unknown-linux-gnu
|
||||
```
|
||||
|
||||
### Using Local Qwen3-TTS
|
||||
|
||||
If you're developing Qwen3-TTS locally:
|
||||
|
||||
```bash
|
||||
export QWEN_TTS_PATH=~/path/to/Qwen3-TTS
|
||||
bun run build:server # Build server binary only
|
||||
# or
|
||||
bun run build # Build everything
|
||||
```
|
||||
|
||||
This makes PyInstaller use your local version instead of the pip package.
|
||||
|
||||
### Debug Build
|
||||
|
||||
```bash
|
||||
cd tauri
|
||||
bun run tauri build --debug
|
||||
```
|
||||
|
||||
Creates a debug build with symbols and logging.
|
||||
|
||||
## Build Configuration
|
||||
|
||||
### Tauri Config
|
||||
|
||||
Edit `tauri/src-tauri/tauri.conf.json`:
|
||||
|
||||
```json
|
||||
{
|
||||
"bundle": {
|
||||
"identifier": "com.voicebox.app",
|
||||
"icon": [
|
||||
"icons/32x32.png",
|
||||
"icons/128x128.png",
|
||||
"icons/icon.icns",
|
||||
"icons/icon.ico"
|
||||
]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Sidecar Configuration
|
||||
|
||||
The Python server is bundled as a sidecar:
|
||||
|
||||
```json
|
||||
{
|
||||
"tauri": {
|
||||
"bundle": {
|
||||
"externalBin": [
|
||||
"binaries/voicebox-server"
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Code Signing
|
||||
|
||||
### macOS
|
||||
|
||||
To sign the app for distribution:
|
||||
|
||||
```bash
|
||||
# Set signing identity
|
||||
export APPLE_SIGNING_IDENTITY="Developer ID Application: Your Name"
|
||||
|
||||
# Build with signing
|
||||
bun run tauri build
|
||||
```
|
||||
|
||||
For notarization:
|
||||
|
||||
```bash
|
||||
# Set credentials
|
||||
export APPLE_ID="[email protected]"
|
||||
export APPLE_PASSWORD="app-specific-password"
|
||||
|
||||
# Build and notarize
|
||||
bun run tauri build
|
||||
```
|
||||
|
||||
### Windows
|
||||
|
||||
For Windows code signing:
|
||||
|
||||
```bash
|
||||
# Set certificate
|
||||
export WINDOWS_CERTIFICATE_PATH="/path/to/cert.pfx"
|
||||
export WINDOWS_CERTIFICATE_PASSWORD="password"
|
||||
|
||||
# Build with signing
|
||||
bun run tauri build
|
||||
```
|
||||
|
||||
## Release Process
|
||||
|
||||
The full release process is automated:
|
||||
|
||||
```bash
|
||||
# 1. Bump version
|
||||
bumpversion patch # or minor/major
|
||||
|
||||
# 2. Build all platforms (CI/CD handles this)
|
||||
git push --tags
|
||||
|
||||
# 3. GitHub Actions creates releases
|
||||
```
|
||||
|
||||
See [CONTRIBUTING.md](/development/contributing) for the full release workflow.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
<AccordionGroup>
|
||||
<Accordion title="Server Binary Build Fails">
|
||||
**Common issues:**
|
||||
- Missing Python dependencies: `pip install -r requirements.txt`
|
||||
- PyInstaller not found: `pip install pyinstaller`
|
||||
- Qwen3-TTS not installed: `pip install git+https://github.com/QwenLM/Qwen3-TTS.git`
|
||||
|
||||
**Solution:**
|
||||
```bash
|
||||
cd backend
|
||||
source venv/bin/activate
|
||||
pip install -r requirements.txt
|
||||
pip install pyinstaller
|
||||
```
|
||||
</Accordion>
|
||||
|
||||
<Accordion title="Tauri Build Fails">
|
||||
**Common issues:**
|
||||
- Rust not installed: `curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh`
|
||||
- Server binary missing: Usually auto-built, but can run manually: `./scripts/build-server.sh`
|
||||
- Node modules outdated: `bun install`
|
||||
|
||||
**Solution:**
|
||||
```bash
|
||||
# Clean and rebuild
|
||||
cd tauri/src-tauri
|
||||
cargo clean
|
||||
cd ../..
|
||||
bun run build # Automatically builds server binary first
|
||||
```
|
||||
</Accordion>
|
||||
|
||||
<Accordion title="App Won't Launch After Build">
|
||||
**Check:**
|
||||
- Server binary has execute permissions
|
||||
- All dependencies are bundled
|
||||
- Check logs in the app's data directory
|
||||
|
||||
**macOS:**
|
||||
```bash
|
||||
tail -f ~/Library/Application\ Support/com.voicebox.app/logs/server.log
|
||||
```
|
||||
|
||||
**Windows:**
|
||||
```bash
|
||||
type %APPDATA%\com.voicebox.app\logs\server.log
|
||||
```
|
||||
</Accordion>
|
||||
</AccordionGroup>
|
||||
|
||||
## CI/CD
|
||||
|
||||
GitHub Actions automatically builds releases when tags are pushed:
|
||||
|
||||
```yaml
|
||||
# .github/workflows/release.yml
|
||||
on:
|
||||
push:
|
||||
tags:
|
||||
- 'v*'
|
||||
```
|
||||
|
||||
See the [repository](https://github.com/jamiepine/voicebox) for the full CI/CD configuration.
|
||||
@@ -1,239 +0,0 @@
|
||||
---
|
||||
title: "Development Setup"
|
||||
description: "Set up your local development environment for Voicebox"
|
||||
---
|
||||
|
||||
## Prerequisites
|
||||
|
||||
Before you begin, ensure you have the following installed:
|
||||
|
||||
<CardGroup cols={3}>
|
||||
<Card title="Bun" icon="package">
|
||||
[Download Bun](https://bun.sh)
|
||||
```bash
|
||||
curl -fsSL https://bun.sh/install | bash
|
||||
```
|
||||
</Card>
|
||||
<Card title="Python 3.11+" icon="python">
|
||||
[Download Python](https://python.org)
|
||||
```bash
|
||||
python --version
|
||||
```
|
||||
</Card>
|
||||
<Card title="Rust" icon="rust">
|
||||
[Install Rust](https://rustup.rs)
|
||||
```bash
|
||||
rustc --version
|
||||
```
|
||||
</Card>
|
||||
</CardGroup>
|
||||
|
||||
## Clone the Repository
|
||||
|
||||
```bash
|
||||
git clone https://github.com/jamiepine/voicebox.git
|
||||
cd voicebox
|
||||
```
|
||||
|
||||
## Quick Setup (Recommended)
|
||||
|
||||
The easiest way to get started is using the Makefile:
|
||||
|
||||
```bash
|
||||
# Setup everything
|
||||
make setup
|
||||
|
||||
# Start development
|
||||
make dev
|
||||
```
|
||||
|
||||
<Note>
|
||||
The Makefile is available on macOS and Linux. Windows users should follow the manual setup below.
|
||||
</Note>
|
||||
|
||||
## Manual Setup
|
||||
|
||||
### 1. Install JavaScript Dependencies
|
||||
|
||||
```bash
|
||||
bun install
|
||||
```
|
||||
|
||||
This installs dependencies for:
|
||||
- `app/` - Shared React frontend
|
||||
- `tauri/` - Tauri desktop wrapper
|
||||
- `web/` - Web deployment wrapper
|
||||
|
||||
### 2. Set Up Python Backend
|
||||
|
||||
```bash
|
||||
cd backend
|
||||
|
||||
# Create virtual environment
|
||||
python -m venv venv
|
||||
|
||||
# Activate virtual environment
|
||||
source venv/bin/activate # macOS/Linux
|
||||
# or
|
||||
venv\Scripts\activate # Windows
|
||||
|
||||
# Install Python dependencies
|
||||
pip install -r requirements.txt
|
||||
|
||||
# Install MLX dependencies (Apple Silicon only - for faster inference)
|
||||
# On Apple Silicon, this enables native Metal acceleration
|
||||
if [[ $(uname -m) == "arm64" ]]; then
|
||||
pip install -r requirements-mlx.txt
|
||||
fi
|
||||
|
||||
# Install Qwen3-TTS
|
||||
pip install git+https://github.com/QwenLM/Qwen3-TTS.git
|
||||
```
|
||||
|
||||
## Running in Development
|
||||
|
||||
Development requires **two terminals**: one for the Python backend, one for the Tauri app.
|
||||
|
||||
<Tabs>
|
||||
<Tab title="Terminal 1: Backend">
|
||||
Start the Python server first:
|
||||
|
||||
```bash
|
||||
cd backend
|
||||
source venv/bin/activate # Activate venv
|
||||
bun run dev:server
|
||||
```
|
||||
|
||||
Or manually:
|
||||
```bash
|
||||
uvicorn main:app --reload --port 17493
|
||||
```
|
||||
|
||||
Backend will be available at `http://localhost:17493`
|
||||
</Tab>
|
||||
|
||||
<Tab title="Terminal 2: Desktop App">
|
||||
Then start the Tauri app:
|
||||
|
||||
```bash
|
||||
bun run dev
|
||||
```
|
||||
|
||||
This will:
|
||||
- Create a placeholder sidecar binary
|
||||
- Start Vite dev server on port 5173
|
||||
- Launch Tauri window
|
||||
- Enable hot reload
|
||||
</Tab>
|
||||
</Tabs>
|
||||
|
||||
<Info>
|
||||
In dev mode, the app connects to your manually-started Python server. The bundled server binary is only used in production builds.
|
||||
</Info>
|
||||
|
||||
### Optional: Web App
|
||||
|
||||
```bash
|
||||
bun run dev:web
|
||||
```
|
||||
|
||||
Web app will be available at `http://localhost:5174`
|
||||
|
||||
## Model Downloads
|
||||
|
||||
Models are automatically downloaded from HuggingFace Hub on first use:
|
||||
|
||||
- **Whisper** (transcription): Auto-downloads on first transcription
|
||||
- **Qwen3-TTS** (voice cloning): Auto-downloads on first generation (~2-4GB)
|
||||
|
||||
<Warning>
|
||||
First-time usage will be slower due to model downloads, but subsequent runs will use cached models.
|
||||
</Warning>
|
||||
|
||||
## Project Structure
|
||||
|
||||
```
|
||||
voicebox/
|
||||
├── app/ # Shared React frontend
|
||||
│ └── src/
|
||||
│ ├── components/ # UI components
|
||||
│ ├── lib/ # Utilities and API client
|
||||
│ └── hooks/ # React hooks
|
||||
├── backend/ # Python FastAPI server
|
||||
│ ├── main.py # API routes
|
||||
│ ├── tts.py # Voice synthesis
|
||||
│ └── database.py # SQLite operations
|
||||
├── tauri/ # Desktop app wrapper
|
||||
│ └── src-tauri/ # Rust backend
|
||||
├── web/ # Web deployment
|
||||
├── landing/ # Marketing website
|
||||
└── scripts/ # Build & release scripts
|
||||
```
|
||||
|
||||
## Available Make Commands
|
||||
|
||||
Run `make help` to see all available commands:
|
||||
|
||||
```bash
|
||||
make setup # Install all dependencies
|
||||
make dev # Start development servers
|
||||
make dev-web # Start web development server
|
||||
make build # Build desktop app
|
||||
make build-web # Build web app
|
||||
make clean # Clean build artifacts
|
||||
make test # Run tests
|
||||
```
|
||||
|
||||
## Generate OpenAPI Client
|
||||
|
||||
After starting the backend server, generate the TypeScript API client:
|
||||
|
||||
```bash
|
||||
./scripts/generate-api.sh
|
||||
# or
|
||||
bun run generate:api
|
||||
```
|
||||
|
||||
This downloads the OpenAPI schema and generates the TypeScript client in `app/src/lib/api/`
|
||||
|
||||
## Next Steps
|
||||
|
||||
<CardGroup cols={2}>
|
||||
<Card title="Architecture" icon="diagram-project" href="/development/architecture">
|
||||
Understand the system architecture
|
||||
</Card>
|
||||
<Card title="Contributing" icon="code-pull-request" href="/development/contributing">
|
||||
Read the contribution guidelines
|
||||
</Card>
|
||||
<Card title="Building" icon="hammer" href="/development/building">
|
||||
Learn how to build production releases
|
||||
</Card>
|
||||
<Card title="API Reference" icon="code" href="/api/overview">
|
||||
Explore the REST API
|
||||
</Card>
|
||||
</CardGroup>
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
<AccordionGroup>
|
||||
<Accordion title="Backend won't start">
|
||||
- Check Python version (must be 3.11+)
|
||||
- Ensure virtual environment is activated
|
||||
- Verify all dependencies are installed: `pip install -r requirements.txt`
|
||||
- Check if port 17493 is available
|
||||
</Accordion>
|
||||
|
||||
<Accordion title="Tauri build fails">
|
||||
- Ensure Rust is installed: `rustc --version`
|
||||
- Clean the build: `cd tauri/src-tauri && cargo clean`
|
||||
- Try rebuilding: `bun run dev`
|
||||
</Accordion>
|
||||
|
||||
<Accordion title="OpenAPI client generation fails">
|
||||
- Ensure backend is running: `curl http://localhost:17493/openapi.json`
|
||||
- Check network connectivity
|
||||
- Verify the backend is accessible at localhost:17493
|
||||
</Accordion>
|
||||
</AccordionGroup>
|
||||
|
||||
See the full [Troubleshooting Guide](/guides/troubleshooting) for more issues and solutions.
|
||||
@@ -0,0 +1 @@
|
||||
export { twMerge as cn } from 'tailwind-merge';
|
||||
@@ -0,0 +1,9 @@
|
||||
import type { BaseLayoutProps } from 'fumadocs-ui/layouts/shared';
|
||||
|
||||
export function baseOptions(): BaseLayoutProps {
|
||||
return {
|
||||
nav: {
|
||||
title: 'Voicebox',
|
||||
},
|
||||
};
|
||||
}
|
||||
@@ -0,0 +1,5 @@
|
||||
import { createOpenAPI } from 'fumadocs-openapi/server';
|
||||
|
||||
export const openapi = createOpenAPI({
|
||||
input: ['./openapi.json'],
|
||||
});
|
||||
@@ -0,0 +1,27 @@
|
||||
import { docs } from '@/.source';
|
||||
import { type InferPageType, loader } from 'fumadocs-core/source';
|
||||
import { lucideIconsPlugin } from 'fumadocs-core/source/lucide-icons';
|
||||
|
||||
// See https://fumadocs.dev/docs/headless/source-api for more info
|
||||
export const source = loader({
|
||||
baseUrl: '/docs',
|
||||
source: docs.toFumadocsSource(),
|
||||
plugins: [lucideIconsPlugin()],
|
||||
});
|
||||
|
||||
export function getPageImage(page: InferPageType<typeof source>) {
|
||||
const segments = [...page.slugs, 'image.png'];
|
||||
|
||||
return {
|
||||
segments,
|
||||
url: `/og/docs/${segments.join('/')}`,
|
||||
};
|
||||
}
|
||||
|
||||
export async function getLLMText(page: InferPageType<typeof source>) {
|
||||
const processed = await page.data.getText('processed');
|
||||
|
||||
return `# ${page.data.title}
|
||||
|
||||
${processed}`;
|
||||
}
|
||||
@@ -0,0 +1,53 @@
|
||||
import { Callout } from 'fumadocs-ui/components/callout';
|
||||
import { Card, Cards } from 'fumadocs-ui/components/card';
|
||||
import { File, Files, Folder } from 'fumadocs-ui/components/files';
|
||||
import { Step, Steps } from 'fumadocs-ui/components/steps';
|
||||
import { Tab, Tabs } from 'fumadocs-ui/components/tabs';
|
||||
import defaultMdxComponents from 'fumadocs-ui/mdx';
|
||||
import type { MDXComponents } from 'mdx/types';
|
||||
import type { ReactNode } from 'react';
|
||||
import { APIPage } from '@/components/api-page';
|
||||
|
||||
// Simple accordion using native HTML details/summary
|
||||
function AccordionGroup({ children }: { children: ReactNode }) {
|
||||
return <div className="my-6 space-y-2">{children}</div>;
|
||||
}
|
||||
|
||||
function Accordion({ title, children }: { title: string; children: ReactNode }) {
|
||||
return (
|
||||
<details className="group border rounded-lg p-4">
|
||||
<summary className="cursor-pointer font-semibold list-none">
|
||||
<span className="group-open:rotate-90 transition-transform inline-block mr-2">▶</span>
|
||||
{title}
|
||||
</summary>
|
||||
<div className="mt-4 pl-6">{children}</div>
|
||||
</details>
|
||||
);
|
||||
}
|
||||
|
||||
export function getMDXComponents(components?: MDXComponents): MDXComponents {
|
||||
return {
|
||||
...defaultMdxComponents,
|
||||
// Layout components
|
||||
Card,
|
||||
Cards,
|
||||
// Files
|
||||
Files,
|
||||
Folder,
|
||||
File,
|
||||
// Callouts
|
||||
Callout,
|
||||
// Tabs
|
||||
Tabs,
|
||||
Tab,
|
||||
// Steps
|
||||
Steps,
|
||||
Step,
|
||||
// Accordion (native HTML-based)
|
||||
AccordionGroup,
|
||||
Accordion,
|
||||
// OpenAPI component
|
||||
APIPage,
|
||||
...components,
|
||||
};
|
||||
}
|
||||
-102
@@ -1,102 +0,0 @@
|
||||
{
|
||||
"$schema": "https://mintlify.com/schema.json",
|
||||
"name": "Voicebox",
|
||||
"logo": {
|
||||
"light": "/logo/icon-light.png",
|
||||
"dark": "/logo/icon-dark.png"
|
||||
},
|
||||
"favicon": "/favicon.png",
|
||||
"colors": {
|
||||
"primary": "#BF9E40",
|
||||
"light": "#D4B560",
|
||||
"dark": "#A68A35"
|
||||
},
|
||||
"styles": {
|
||||
"css": ["/custom.css"]
|
||||
},
|
||||
"anchors": [
|
||||
{
|
||||
"name": "Overview",
|
||||
"icon": "book-open",
|
||||
"url": "overview"
|
||||
},
|
||||
{
|
||||
"name": "API",
|
||||
"icon": "code",
|
||||
"url": "api"
|
||||
},
|
||||
{
|
||||
"name": "Developer",
|
||||
"icon": "book-open-cover",
|
||||
"url": "developer"
|
||||
},
|
||||
{
|
||||
"name": "GitHub",
|
||||
"icon": "github",
|
||||
"url": "https://github.com/jamiepine/voicebox"
|
||||
}
|
||||
],
|
||||
"navigation": [
|
||||
{
|
||||
"group": "Getting Started",
|
||||
"icon": "rocket",
|
||||
"pages": ["overview/introduction", "overview/installation", "overview/quick-start"]
|
||||
},
|
||||
{
|
||||
"group": "Features",
|
||||
"icon": "sparkles",
|
||||
"pages": [
|
||||
"overview/voice-cloning",
|
||||
"overview/stories-editor",
|
||||
"overview/recording-transcription",
|
||||
"overview/generation-history",
|
||||
"overview/remote-mode"
|
||||
]
|
||||
},
|
||||
{
|
||||
"group": "User Guides",
|
||||
"icon": "compass",
|
||||
"pages": [
|
||||
"overview/creating-voice-profiles",
|
||||
"overview/generating-speech",
|
||||
"overview/building-stories",
|
||||
"overview/troubleshooting"
|
||||
]
|
||||
},
|
||||
{
|
||||
"group": "Development",
|
||||
"icon": "wrench",
|
||||
"pages": [
|
||||
"developer/setup",
|
||||
"developer/architecture",
|
||||
"developer/contributing",
|
||||
"developer/building",
|
||||
"developer/autoupdater"
|
||||
]
|
||||
},
|
||||
{
|
||||
"group": "API Reference",
|
||||
"icon": "code",
|
||||
"pages": [
|
||||
"api/overview",
|
||||
"api/authentication",
|
||||
"api/voice-profiles",
|
||||
"api/generation",
|
||||
"api/recordings"
|
||||
]
|
||||
},
|
||||
{
|
||||
"group": "Architecture",
|
||||
"icon": "book-open-cover",
|
||||
"pages": [
|
||||
"developer/voice-profiles",
|
||||
"developer/tts-generation",
|
||||
"developer/history",
|
||||
"developer/stories",
|
||||
"developer/transcription",
|
||||
"developer/audio-channels",
|
||||
"developer/model-management"
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,25 @@
|
||||
import { createMDX } from 'fumadocs-mdx/next';
|
||||
|
||||
const withMDX = createMDX();
|
||||
|
||||
/** @type {import('next').NextConfig} */
|
||||
const config = {
|
||||
reactStrictMode: true,
|
||||
async rewrites() {
|
||||
return [
|
||||
{
|
||||
source: '/docs/:path*.mdx',
|
||||
destination: '/llms.mdx/docs/:path*',
|
||||
},
|
||||
];
|
||||
},
|
||||
webpack: (config) => {
|
||||
config.experiments = {
|
||||
...config.experiments,
|
||||
topLevelAwait: true,
|
||||
};
|
||||
return config;
|
||||
},
|
||||
};
|
||||
|
||||
export default withMDX(config);
|
||||
@@ -1,11 +0,0 @@
|
||||
[phases.setup]
|
||||
nixPkgs = ["nodejs_20", "bun"]
|
||||
|
||||
[phases.install]
|
||||
cmds = ["bun install"]
|
||||
|
||||
[phases.build]
|
||||
cmds = ["bun run build"]
|
||||
|
||||
[start]
|
||||
cmd = "bun run start"
|
||||
File diff suppressed because one or more lines are too long
@@ -1,58 +0,0 @@
|
||||
---
|
||||
title: "Introduction"
|
||||
description: "Welcome to Voicebox - the open-source voice synthesis studio"
|
||||
---
|
||||
|
||||
## What is Voicebox?
|
||||
|
||||
Voicebox is a **local-first voice cloning studio** with DAW-like features for professional voice synthesis. Think of it as a **local, free and open-source alternative to ElevenLabs** — download models, clone voices, and generate speech entirely on your machine.
|
||||
|
||||
<Frame>
|
||||
<img src="/images/app-screenshot-1.webp" alt="Voicebox App Screenshot" />
|
||||
</Frame>
|
||||
|
||||
Unlike cloud services that lock your voice data behind subscriptions, Voicebox gives you:
|
||||
|
||||
- **Complete privacy** — models and voice data stay on your machine
|
||||
- **Professional tools** — multi-track timeline editor, audio trimming, conversation mixing
|
||||
- **Model flexibility** — currently powered by Qwen3-TTS, with support for XTTS, Bark, and other models coming soon
|
||||
- **API-first** — use the desktop app or integrate voice synthesis into your own projects
|
||||
- **Native performance** — built with Tauri (Rust), not Electron
|
||||
|
||||
Download a voice model, clone any voice from a few seconds of audio, and compose multi-voice projects with studio-grade editing tools. No Python install required, no cloud dependency, no limits.
|
||||
|
||||
## Key Features
|
||||
|
||||
<CardGroup cols={2}>
|
||||
<Card title="Voice Cloning" icon="microphone">
|
||||
Instant cloning from just a few seconds of audio with Qwen3-TTS
|
||||
</Card>
|
||||
<Card title="Stories Editor" icon="film">
|
||||
Multi-track timeline for creating conversations and narratives
|
||||
</Card>
|
||||
<Card title="Full API" icon="code">
|
||||
REST API for integrating voice synthesis into your apps
|
||||
</Card>
|
||||
<Card title="Local-First" icon="shield">
|
||||
Everything runs on your machine - complete privacy
|
||||
</Card>
|
||||
</CardGroup>
|
||||
|
||||
## Use Cases
|
||||
|
||||
- **Game Development** — Generate dynamic dialogue for characters
|
||||
- **Content Creation** — Produce podcasts and video voiceovers
|
||||
- **Accessibility** — Build text-to-speech tools
|
||||
- **Voice Assistants** — Create custom voice interfaces
|
||||
- **Production Pipelines** — Automate voiceover workflows
|
||||
|
||||
## Next Steps
|
||||
|
||||
<CardGroup cols={2}>
|
||||
<Card title="Installation" icon="download" href="/overview/installation">
|
||||
Download and install Voicebox on your machine
|
||||
</Card>
|
||||
<Card title="Quick Start" icon="rocket" href="/overview/quick-start">
|
||||
Get up and running in 5 minutes
|
||||
</Card>
|
||||
</CardGroup>
|
||||
+28
-10
@@ -1,17 +1,35 @@
|
||||
{
|
||||
"name": "voicebox-docs",
|
||||
"version": "0.1.0",
|
||||
"name": "example-next-mdx",
|
||||
"version": "0.0.0",
|
||||
"private": true,
|
||||
"scripts": {
|
||||
"dev": "mintlify dev",
|
||||
"build": "mintlify build",
|
||||
"start": "mintlify serve",
|
||||
"install:mintlify": "bun add -g mintlify"
|
||||
"build": "fumadocs-mdx && next build",
|
||||
"dev": "fumadocs-mdx && next dev",
|
||||
"start": "next start",
|
||||
"postinstall": "fumadocs-mdx"
|
||||
},
|
||||
"dependencies": {
|
||||
"@radix-ui/react-popover": "^1.1.15",
|
||||
"class-variance-authority": "^0.7.1",
|
||||
"fumadocs-core": "^16.4.11",
|
||||
"fumadocs-mdx": "13",
|
||||
"fumadocs-openapi": "^10.2.7",
|
||||
"fumadocs-ui": "^16.4.11",
|
||||
"lucide-react": "^0.546.0",
|
||||
"next": "^16.1.6",
|
||||
"react": "^19.2.0",
|
||||
"react-dom": "^19.2.0",
|
||||
"shiki": "^3.22.0",
|
||||
"tailwind-merge": "^3.5.0"
|
||||
},
|
||||
"devDependencies": {
|
||||
"mintlify": "latest"
|
||||
},
|
||||
"engines": {
|
||||
"bun": ">=1.0.0"
|
||||
"@tailwindcss/postcss": "^4.1.15",
|
||||
"@types/mdx": "^2.0.13",
|
||||
"@types/node": "^24.9.1",
|
||||
"@types/react": "^19.2.2",
|
||||
"@types/react-dom": "^19.2.2",
|
||||
"postcss": "^8.5.6",
|
||||
"tailwindcss": "^4.1.15",
|
||||
"typescript": "^5.9.3"
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,363 +0,0 @@
|
||||
# Adding a TTS Engine to Voicebox
|
||||
|
||||
Guide for adding new TTS model backends. Based on the implementation of LuxTTS (#254), Chatterbox Multilingual (#257), Chatterbox Turbo (#258), and the PyInstaller fixes in v0.2.3.
|
||||
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
Adding an engine touches ~10 files across 4 layers. The backend protocol work is straightforward — the real time sink is dependency hell, upstream library bugs, and PyInstaller bundling.
|
||||
|
||||
The backend is split into layers: `routes/` (thin HTTP handlers), `services/` (business logic), `backends/` (engine implementations), and `utils/` (shared utilities). New engines only need to touch `backends/` and `models.py` on the backend side — the route and service layers use a model config registry that handles dispatch automatically.
|
||||
|
||||
---
|
||||
|
||||
## Phase 1: Backend Implementation
|
||||
|
||||
### 1.1 Create the backend file
|
||||
|
||||
`backend/backends/<engine>_backend.py` (~200-300 lines)
|
||||
|
||||
Implement the `TTSBackend` protocol from `backend/backends/__init__.py`:
|
||||
|
||||
```python
|
||||
class YourBackend:
|
||||
"""Must satisfy the TTSBackend protocol."""
|
||||
|
||||
async def load_model(self, model_size: str = "default") -> None: ...
|
||||
async def create_voice_prompt(self, audio_path: str, reference_text: str, use_cache: bool = True) -> tuple[dict, bool]: ...
|
||||
async def combine_voice_prompts(self, audio_paths: list[str], ref_texts: list[str]) -> tuple[np.ndarray, str]: ...
|
||||
async def generate(self, text: str, voice_prompt: dict, language: str = "en", seed: int | None = None, instruct: str | None = None) -> tuple[np.ndarray, int]: ...
|
||||
def unload_model(self) -> None: ...
|
||||
def is_loaded(self) -> bool: ...
|
||||
def _get_model_path(self, model_size: str) -> str: ...
|
||||
```
|
||||
|
||||
Key decisions per engine:
|
||||
|
||||
| Decision | Options | Examples |
|
||||
|----------|---------|---------|
|
||||
| **Voice prompt storage** | Pre-computed tensors vs deferred file paths | Qwen PyTorch stores tensor dicts; Chatterbox stores `{"ref_audio": path, "ref_text": text}` |
|
||||
| **Caching** | Use voice prompt cache or skip it | LuxTTS caches with `luxtts_` prefix; Chatterbox skips caching entirely |
|
||||
| **Device selection** | CUDA / MPS / CPU | Chatterbox forces CPU on macOS (MPS tensor bugs); LuxTTS supports MPS |
|
||||
| **Model download** | Library handles it vs manual `snapshot_download` | Turbo uses manual download to bypass upstream `token=True` bug |
|
||||
| **Sample rate** | Engine-specific | LuxTTS outputs 48kHz, everything else is 24kHz |
|
||||
|
||||
### 1.2 Voice prompt patterns
|
||||
|
||||
There are three patterns in use. Pick the one that fits your model:
|
||||
|
||||
**Pattern A: Pre-computed tensors** (Qwen PyTorch, LuxTTS)
|
||||
```python
|
||||
# create_voice_prompt returns opaque dict of tensors
|
||||
# Cached via torch.save(), reused across generations
|
||||
encoded = model.encode_prompt(audio_path)
|
||||
return encoded, False # (prompt_dict, was_cached)
|
||||
```
|
||||
|
||||
**Pattern B: Deferred file paths** (Chatterbox, MLX)
|
||||
```python
|
||||
# Just store paths, process at generation time
|
||||
return {"ref_audio": audio_path, "ref_text": reference_text}, False
|
||||
```
|
||||
|
||||
**Pattern C: Hybrid** (possible for new engines)
|
||||
```python
|
||||
# Pre-compute speaker embeddings, store alongside paths
|
||||
embedding = model.extract_speaker(audio_path)
|
||||
return {"embedding": embedding, "ref_audio": audio_path}, False
|
||||
```
|
||||
|
||||
If caching, prefix your cache keys to avoid collisions with other engines using the same reference audio:
|
||||
```python
|
||||
cache_key = "yourengine_" + get_cache_key(audio_path, reference_text)
|
||||
```
|
||||
|
||||
### 1.3 Register the engine
|
||||
|
||||
In `backend/backends/__init__.py`, three things:
|
||||
|
||||
**1. Add a `ModelConfig` entry** in `_get_non_qwen_tts_configs()`:
|
||||
|
||||
```python
|
||||
ModelConfig(
|
||||
model_name="your-engine",
|
||||
display_name="Your Engine",
|
||||
engine="your_engine",
|
||||
hf_repo_id="org/model-repo",
|
||||
size_mb=3200,
|
||||
needs_trim=False, # set True if output needs trim_tts_output()
|
||||
languages=["en", "fr", "de"],
|
||||
),
|
||||
```
|
||||
|
||||
This single entry replaces what used to be 6+ scattered dicts in `main.py`. The registry helpers (`get_model_config()`, `check_model_loaded()`, `engine_needs_trim()`, etc.) all derive from this config automatically.
|
||||
|
||||
**2. Add to `TTS_ENGINES` dict:**
|
||||
|
||||
```python
|
||||
TTS_ENGINES = {
|
||||
...
|
||||
"your_engine": "Your Engine",
|
||||
}
|
||||
```
|
||||
|
||||
**3. Add an elif branch in `get_tts_backend_for_engine()`:**
|
||||
|
||||
```python
|
||||
elif engine == "your_engine":
|
||||
from .your_backend import YourBackend
|
||||
backend = YourBackend()
|
||||
```
|
||||
|
||||
The import is deferred so platform-specific deps aren't loaded until the engine is first requested.
|
||||
|
||||
### 1.4 Update request models
|
||||
|
||||
In `backend/models.py`:
|
||||
|
||||
- Add engine name to `GenerationRequest.engine` regex pattern
|
||||
- Add any new language codes to the language regex on both `GenerationRequest` and `VoiceProfileCreate`
|
||||
|
||||
---
|
||||
|
||||
## Phase 2: Route and Service Integration
|
||||
|
||||
With the model config registry, the route and service layers have **zero per-engine dispatch points**. All endpoints use registry helpers like `get_model_config()`, `load_engine_model()`, `engine_needs_trim()`, `check_model_loaded()`, etc.
|
||||
|
||||
**You don't need to touch any route or service files** unless your engine needs custom behavior in the generate pipeline (e.g. a new post-processing step beyond `trim_tts_output`).
|
||||
|
||||
### 2.1 What the registry handles automatically
|
||||
|
||||
| Route file | Registry function used |
|
||||
|------------|----------------------|
|
||||
| `routes/generations.py` | `load_engine_model(engine, size)` + `engine_needs_trim(engine)` |
|
||||
| `routes/models.py` | `get_all_model_configs()` + `check_model_loaded(config)` |
|
||||
| `routes/models.py` | `get_model_config(name)` + `get_model_load_func(config)` |
|
||||
| `services/generation.py` | `get_tts_backend_for_engine()` + `ensure_model_cached_or_raise()` |
|
||||
|
||||
### 2.2 Post-processing
|
||||
|
||||
If your model produces trailing silence or hallucinated audio, set `needs_trim=True` on your `ModelConfig`. The generation service checks `engine_needs_trim(engine)` and applies `trim_tts_output()` automatically.
|
||||
|
||||
---
|
||||
|
||||
## Phase 3: Frontend Integration
|
||||
|
||||
### 3.1 TypeScript types
|
||||
|
||||
In `app/src/lib/api/types.ts`:
|
||||
- Add to the `engine` union type on `GenerationRequest`
|
||||
|
||||
### 3.2 Language maps
|
||||
|
||||
In `app/src/lib/constants/languages.ts`:
|
||||
- Add entry to `ENGINE_LANGUAGES` record
|
||||
- Add any new language codes to `ALL_LANGUAGES` if needed
|
||||
|
||||
### 3.3 Engine/model selector (shared component)
|
||||
|
||||
The model selector is a shared component — update one file:
|
||||
|
||||
- `app/src/components/Generation/EngineModelSelector.tsx`
|
||||
|
||||
Add an entry to `ENGINE_OPTIONS` and `ENGINE_DESCRIPTIONS`. If the engine is English-only, add it to `ENGLISH_ONLY_ENGINES`. The `handleEngineChange()` function handles language validation automatically (resets to first available language if the current one isn't supported).
|
||||
|
||||
Both `GenerationForm.tsx` and `FloatingGenerateBox.tsx` use `<EngineModelSelector>` — no changes needed in either.
|
||||
|
||||
Handle engine-specific UI conditionals in the form components if needed:
|
||||
- Hide instruct field for engines that don't support it
|
||||
- Show engine-specific controls (e.g. `ParalinguisticInput` for Turbo)
|
||||
|
||||
### 3.4 Form hook
|
||||
|
||||
In `app/src/lib/hooks/useGenerationForm.ts`:
|
||||
- Add to Zod schema enum for `engine`
|
||||
- Add engine-to-model-name mapping (e.g. `"your_engine"` → `"your-engine"`)
|
||||
- Update payload construction to conditionally include engine-specific fields
|
||||
|
||||
### 3.5 Model management
|
||||
|
||||
In `app/src/components/ServerSettings/ModelManagement.tsx`:
|
||||
- Add description to `MODEL_DESCRIPTIONS` record
|
||||
- The model list auto-renders from `/models/status` data
|
||||
|
||||
---
|
||||
|
||||
## Phase 4: Dependencies
|
||||
|
||||
### 4.1 Python dependencies
|
||||
|
||||
Add to `backend/requirements.txt`. Watch for:
|
||||
|
||||
**Pinned dependency conflicts** — If the model package pins old versions of numpy, torch, or transformers, install with `--no-deps` and list sub-dependencies manually. This is what Chatterbox requires:
|
||||
```
|
||||
# In justfile (NOT requirements.txt):
|
||||
pip install --no-deps chatterbox-tts
|
||||
|
||||
# In requirements.txt — list the transitive deps:
|
||||
conformer
|
||||
diffusers
|
||||
omegaconf
|
||||
# ... etc
|
||||
```
|
||||
|
||||
**Non-PyPI packages** — Some deps only exist as git repos:
|
||||
```
|
||||
linacodec @ git+https://github.com/user/repo.git
|
||||
Zipvoice @ git+https://github.com/user/repo.git
|
||||
```
|
||||
|
||||
**Custom package indexes** — Some packages need `--find-links`:
|
||||
```
|
||||
--find-links https://k2-fsa.github.io/icefall/piper_phonemize.html
|
||||
```
|
||||
|
||||
### 4.2 Identifying hidden sub-dependencies
|
||||
|
||||
When using `--no-deps`, you need to manually figure out what the package actually imports. There's no shortcut:
|
||||
|
||||
1. Install the package normally in a throwaway venv
|
||||
2. Run `pip show <package>` to get its `Requires:` list
|
||||
3. Cross-reference against what's already in our requirements.txt
|
||||
4. Test that the engine loads and generates without import errors
|
||||
|
||||
---
|
||||
|
||||
## Phase 5: PyInstaller Bundling
|
||||
|
||||
This is where most of the pain lives. If your model's Python package or its dependencies use any of the following at runtime, PyInstaller won't bundle them automatically:
|
||||
|
||||
### 5.1 Common PyInstaller issues
|
||||
|
||||
| Issue | Symptom | Fix |
|
||||
|-------|---------|-----|
|
||||
| **`inspect.getsource()` at import time** | "could not get source code" | `--collect-all <package>` (bundles `.py` source files, not just bytecode) |
|
||||
| **Data files (yaml, .pth.tar, lang dicts)** | FileNotFoundError at runtime | `--collect-all <package>` or `--collect-data <package>` |
|
||||
| **Native data paths (espeak-ng, etc.)** | Library looks at `/usr/share/...` | Set env var in frozen builds: `os.environ["ESPEAK_DATA_PATH"] = bundled_path` |
|
||||
| **`importlib.metadata` lookups** | "No package metadata found" | `--copy-metadata <package>` |
|
||||
| **Dynamic imports** | ModuleNotFoundError | `--hidden-import <module>` |
|
||||
| **`typeguard` / `@typechecked`** | Calls `inspect.getsource()` on decorated functions | `--collect-all` for the decorated package |
|
||||
|
||||
### 5.2 Testing frozen builds
|
||||
|
||||
You can't skip this. Models that work in `python -m uvicorn` will break in the PyInstaller binary. The flow:
|
||||
|
||||
1. Build the binary: `just build` or the PyInstaller spec
|
||||
2. Run it and try to download + load + generate with the new engine
|
||||
3. Check stderr for the actual error (macOS/Linux: stdout/stderr go to Tauri sidecar logs)
|
||||
4. Fix, rebuild, repeat
|
||||
|
||||
### 5.3 Real examples from v0.2.3
|
||||
|
||||
These were all models that worked perfectly in dev:
|
||||
|
||||
- **LuxTTS**: `typeguard`'s `@typechecked` calls `inspect.getsource()` at import → needed `--collect-all inflect`. `piper_phonemize` bundles `espeak-ng-data/` → needed `--collect-all piper_phonemize` + `ESPEAK_DATA_PATH` env var
|
||||
- **Chatterbox**: `resemble-perth` bundles a pretrained watermark model (`.pth.tar`, `hparams.yaml`) → needed `--collect-all perth`
|
||||
- **Both**: `huggingface_hub` silently disables tqdm based on logger level → progress bars showed 0% in frozen builds until we force-enabled the internal counter
|
||||
|
||||
---
|
||||
|
||||
## Phase 6: Common Upstream Workarounds
|
||||
|
||||
Almost every model library has bugs you'll need to work around. Here's the catalog:
|
||||
|
||||
### 6.1 torch.load device mismatch
|
||||
|
||||
If model weights were saved on CUDA but you're loading on CPU/MPS:
|
||||
```python
|
||||
_original_torch_load = torch.load
|
||||
def _patched_torch_load(*args, **kwargs):
|
||||
kwargs.setdefault("map_location", "cpu")
|
||||
return _original_torch_load(*args, **kwargs)
|
||||
torch.load = _patched_torch_load
|
||||
```
|
||||
Used by both Chatterbox backends. Use a threading lock if patching globally.
|
||||
|
||||
### 6.2 Float64/Float32 dtype mismatch
|
||||
|
||||
`librosa` returns float64, model weights are float32. Patch the offending methods:
|
||||
```python
|
||||
original_fn = SomeClass.some_method
|
||||
def patched_fn(self, *args, **kwargs):
|
||||
result = original_fn(self, *args, **kwargs)
|
||||
return result.float() # float64 → float32
|
||||
SomeClass.some_method = patched_fn
|
||||
```
|
||||
Used by Chatterbox for `S3Tokenizer.log_mel_spectrogram` and `VoiceEncoder.forward`.
|
||||
|
||||
### 6.3 Transformers attention implementation
|
||||
|
||||
If the model uses `output_attentions=True` with transformers >= 4.36:
|
||||
```python
|
||||
for module in model.modules():
|
||||
if hasattr(module, '_attn_implementation'):
|
||||
module._attn_implementation = "eager"
|
||||
```
|
||||
SDPA (the new default) doesn't support `output_attentions`. Force eager attention.
|
||||
|
||||
### 6.4 HuggingFace token bug
|
||||
|
||||
Some models' `from_pretrained()` passes `token=True` which requires a stored HF token even for public repos:
|
||||
```python
|
||||
from huggingface_hub import snapshot_download
|
||||
local_path = snapshot_download(repo_id=REPO, token=None)
|
||||
model = ModelClass.from_local(local_path, device=device)
|
||||
```
|
||||
Used by Chatterbox Turbo.
|
||||
|
||||
### 6.5 MPS tensor issues
|
||||
|
||||
MPS (Apple Silicon GPU) has incomplete operator coverage. If generation crashes on MPS:
|
||||
```python
|
||||
def _get_device(self):
|
||||
if torch.cuda.is_available():
|
||||
return "cuda"
|
||||
return "cpu" # Skip MPS entirely
|
||||
```
|
||||
Used by both Chatterbox backends. LuxTTS works fine on MPS.
|
||||
|
||||
### 6.6 HuggingFace progress tracking
|
||||
|
||||
To get download progress bars in the UI, wrap model loading with `HFProgressTracker`:
|
||||
```python
|
||||
from ..utils.hf_progress import HFProgressTracker
|
||||
tracker = HFProgressTracker(model_name, progress_manager)
|
||||
with tracker.patch_download():
|
||||
model = ModelClass.from_pretrained(repo_id)
|
||||
```
|
||||
The tracker monkey-patches tqdm to intercept HuggingFace's internal progress bars. Must be set up BEFORE importing the model library if it imports HF at module level.
|
||||
|
||||
---
|
||||
|
||||
## Checklist
|
||||
|
||||
### Backend
|
||||
- [ ] `backend/backends/<engine>_backend.py` — implements TTSBackend protocol
|
||||
- [ ] `backend/backends/__init__.py` — `ModelConfig` entry + `TTS_ENGINES` + `get_tts_backend_for_engine()` elif
|
||||
- [ ] `backend/models.py` — engine name in regex, any new language codes
|
||||
- [ ] `backend/requirements.txt` — dependencies added (check for `--no-deps` needs)
|
||||
- [ ] `justfile` — `--no-deps` install step if needed
|
||||
|
||||
### Routes and services
|
||||
No changes needed — the model config registry handles all dispatch automatically.
|
||||
|
||||
### Frontend
|
||||
- [ ] `app/src/lib/api/types.ts` — engine union type
|
||||
- [ ] `app/src/lib/constants/languages.ts` — `ENGINE_LANGUAGES` entry
|
||||
- [ ] `app/src/components/Generation/EngineModelSelector.tsx` — `ENGINE_OPTIONS` + `ENGINE_DESCRIPTIONS` + `ENGLISH_ONLY_ENGINES`
|
||||
- [ ] `app/src/lib/hooks/useGenerationForm.ts` — Zod schema + model mapping
|
||||
- [ ] `app/src/components/ServerSettings/ModelManagement.tsx` — model description
|
||||
|
||||
### Production
|
||||
- [ ] PyInstaller spec — `--collect-all`, `--hidden-import`, `--copy-metadata` as needed
|
||||
- [ ] Test in frozen binary — download, load, generate all work
|
||||
- [ ] Download progress — `HFProgressTracker` wired up, progress shows in UI
|
||||
|
||||
### Upstream workarounds (check which apply)
|
||||
- [ ] torch.load device mapping (CUDA weights on CPU)
|
||||
- [ ] Float64→Float32 patches (librosa interaction)
|
||||
- [ ] Eager attention forcing (transformers >= 4.36)
|
||||
- [ ] HF token bypass (snapshot_download + from_local)
|
||||
- [ ] MPS skip (if operators not supported)
|
||||
- [ ] espeak-ng / native data path env vars
|
||||
@@ -1,581 +0,0 @@
|
||||
# CUDA Backend Swap via Binary Replacement
|
||||
|
||||
> Status: Plan | Target: v0.2.0 | Created: 2026-03-12
|
||||
|
||||
## Problem
|
||||
|
||||
The CUDA PyTorch backend binary is ~2.4 GB. GitHub Releases has a 2 GB asset limit. The current release ships CPU-only PyTorch on Windows and Intel Mac — NVIDIA GPU users get no acceleration from official releases. This is the #1 reported issue category (19 open issues).
|
||||
|
||||
Users who want GPU today must clone the repo and run from source. That's not acceptable for a desktop app targeting non-technical users.
|
||||
|
||||
## Solution
|
||||
|
||||
Ship two backend binaries: a default CPU build (~150 MB) bundled with the app, and a downloadable CUDA build (~2.4 GB) hosted externally. When the user downloads the CUDA build, the app kills the current backend process, swaps in the CUDA binary, and relaunches — a backend-only restart. The frontend stays running, all UI state is preserved.
|
||||
|
||||
No subprocesses. No HTTP protocol between processes. No port allocation. No provider manager. The backend is still one monolithic process — just a different binary.
|
||||
|
||||
## Architecture
|
||||
|
||||
### What Exists Today
|
||||
|
||||
```
|
||||
Tauri App
|
||||
├── React Frontend (in-process webview)
|
||||
└── voicebox-server (sidecar subprocess on :17493)
|
||||
└── One PyInstaller binary: CPU PyTorch or MLX
|
||||
```
|
||||
|
||||
**Sidecar lifecycle** (`tauri/src-tauri/src/main.rs`):
|
||||
- `start_server` command spawns `voicebox-server` sidecar (line 181)
|
||||
- Binary located at `tauri/src-tauri/binaries/voicebox-server-{platform-triple}`
|
||||
- Tauri resolves the sidecar name via `externalBin` in `tauri.conf.json` (line 16)
|
||||
- Waits up to 120s for "Uvicorn running" in stdout/stderr (line 286)
|
||||
- `stop_server` kills the process tree (line 466)
|
||||
|
||||
**Frontend reconnection** (`app/src/lib/hooks/useServer.ts`):
|
||||
- Health check polls `GET /health` every 30 seconds
|
||||
- React Query cache retains data for 10 minutes after disconnect
|
||||
- All UI state (Zustand stores, form data, open tabs) survives disconnection
|
||||
- No active reconnect logic — just keeps polling until server responds
|
||||
|
||||
This means a backend restart is mostly invisible to the frontend: it sees a few seconds of failed health checks, then the server comes back. The only risk is in-flight operations (generation, transcription) failing mid-request.
|
||||
|
||||
### What Changes
|
||||
|
||||
```
|
||||
Tauri App
|
||||
├── React Frontend (in-process webview)
|
||||
└── voicebox-server (sidecar subprocess on :17493)
|
||||
└── One of:
|
||||
├── voicebox-server-cpu (bundled, ~150 MB)
|
||||
└── voicebox-server-cuda (downloaded, ~2.4 GB)
|
||||
```
|
||||
|
||||
The CUDA binary is functionally identical to the CPU binary. Same FastAPI app, same endpoints, same code. The only difference is PyTorch is compiled with CUDA 12.1 support and the binary includes CUDA runtime libraries.
|
||||
|
||||
The user downloads it once. On every subsequent app launch, Tauri checks which binary variant exists and spawns the appropriate one.
|
||||
|
||||
## Implementation Plan
|
||||
|
||||
### Phase 1: Build Infrastructure
|
||||
|
||||
Build the CUDA binary in CI separately from the main release.
|
||||
|
||||
#### 1a. CUDA PyInstaller Build
|
||||
|
||||
Add a `build_binary_cuda.py` or parameterize the existing `build_binary.py`:
|
||||
|
||||
```python
|
||||
# backend/build_binary.py — add flag
|
||||
def build_server(cuda=False):
|
||||
args = [
|
||||
'server.py',
|
||||
'--onefile',
|
||||
'--name', f'voicebox-server-{"cuda" if cuda else "cpu"}',
|
||||
]
|
||||
|
||||
if cuda:
|
||||
args.extend([
|
||||
'--hidden-import', 'torch.cuda',
|
||||
'--hidden-import', 'torch.backends.cudnn',
|
||||
])
|
||||
# ... rest of existing build
|
||||
```
|
||||
|
||||
The `--onefile` flag is already used, which produces a single executable. This is important — `--onedir` would complicate the swap (replacing a directory vs a file).
|
||||
|
||||
#### 1b. CI Workflow for CUDA Binary
|
||||
|
||||
New workflow: `.github/workflows/build-cuda.yml`
|
||||
|
||||
```yaml
|
||||
name: Build CUDA Provider
|
||||
on:
|
||||
workflow_dispatch:
|
||||
push:
|
||||
tags: ["v*"]
|
||||
|
||||
jobs:
|
||||
build-cuda:
|
||||
runs-on: windows-latest # CUDA is Windows/Linux only
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/setup-python@v5
|
||||
with: { python-version: "3.12" }
|
||||
- name: Install dependencies
|
||||
run: |
|
||||
pip install pyinstaller
|
||||
pip install -r backend/requirements.txt
|
||||
pip install torch --index-url https://download.pytorch.org/whl/cu121 --force-reinstall
|
||||
- name: Build CUDA binary
|
||||
run: python backend/build_binary.py --cuda
|
||||
- name: Split binary for GitHub Releases
|
||||
run: |
|
||||
python scripts/split_binary.py backend/dist/voicebox-server-cuda.exe \
|
||||
--chunk-size 1900MB \
|
||||
--output release-assets/
|
||||
- name: Upload to R2
|
||||
# Full binary to R2 (no size limit)
|
||||
run: |
|
||||
aws s3 cp backend/dist/voicebox-server-cuda.exe \
|
||||
s3://voicebox-downloads/cuda/v${{ github.ref_name }}/voicebox-server-cuda.exe \
|
||||
--endpoint-url ${{ secrets.R2_ENDPOINT }}
|
||||
- name: Upload split parts to GitHub Release
|
||||
# Split parts as GitHub Release assets (each <2 GB)
|
||||
uses: softprops/action-gh-release@v1
|
||||
with:
|
||||
files: release-assets/*
|
||||
```
|
||||
|
||||
Two distribution paths for redundancy:
|
||||
- **Cloudflare R2**: Full binary, direct download, no size limit.
|
||||
- **GitHub Releases**: Split into <2 GB chunks as fallback.
|
||||
|
||||
#### 1c. Binary Splitting Script
|
||||
|
||||
```python
|
||||
# scripts/split_binary.py
|
||||
"""Split a large binary into chunks for GitHub Releases."""
|
||||
import hashlib
|
||||
import argparse
|
||||
from pathlib import Path
|
||||
|
||||
def split(input_path: Path, chunk_size: int, output_dir: Path):
|
||||
output_dir.mkdir(parents=True, exist_ok=True)
|
||||
data = input_path.read_bytes()
|
||||
|
||||
# Write SHA-256 of the complete file
|
||||
sha256 = hashlib.sha256(data).hexdigest()
|
||||
(output_dir / f"{input_path.stem}.sha256").write_text(
|
||||
f"{sha256} {input_path.name}\n"
|
||||
)
|
||||
|
||||
# Split into chunks
|
||||
parts = []
|
||||
for i in range(0, len(data), chunk_size):
|
||||
part_name = f"{input_path.stem}.part{len(parts):02d}{input_path.suffix}"
|
||||
part_path = output_dir / part_name
|
||||
part_path.write_bytes(data[i:i + chunk_size])
|
||||
parts.append(part_name)
|
||||
|
||||
# Write manifest
|
||||
(output_dir / f"{input_path.stem}.manifest").write_text(
|
||||
"\n".join(parts) + "\n"
|
||||
)
|
||||
|
||||
print(f"Split into {len(parts)} parts, SHA-256: {sha256}")
|
||||
```
|
||||
|
||||
### Phase 2: Download & Assemble in App
|
||||
|
||||
#### 2a. Backend Download Endpoint
|
||||
|
||||
Add to `backend/main.py`:
|
||||
|
||||
```python
|
||||
@app.post("/backend/download-cuda")
|
||||
async def download_cuda_backend():
|
||||
"""Download the CUDA backend binary."""
|
||||
# Returns immediately, runs download in background
|
||||
task = asyncio.create_task(_download_cuda_binary())
|
||||
task.add_done_callback(lambda t: logger.error(f"CUDA download failed: {t.exception()}") if t.exception() else None)
|
||||
return {"status": "downloading"}
|
||||
|
||||
@app.get("/backend/cuda-status")
|
||||
async def cuda_status():
|
||||
"""Check if CUDA binary is available."""
|
||||
cuda_path = _get_cuda_binary_path()
|
||||
return {
|
||||
"available": cuda_path is not None and cuda_path.exists(),
|
||||
"active": _is_cuda_active(),
|
||||
"download_progress": progress_manager.get_progress("cuda-backend"),
|
||||
}
|
||||
```
|
||||
|
||||
#### 2b. Download + Assemble + Verify Logic
|
||||
|
||||
New file: `backend/cuda_download.py`
|
||||
|
||||
Core logic:
|
||||
|
||||
```python
|
||||
import hashlib
|
||||
from pathlib import Path
|
||||
from backend.config import get_data_dir
|
||||
from backend.utils.progress import get_progress_manager
|
||||
|
||||
CUDA_DOWNLOAD_URL = "https://downloads.voicebox.sh/cuda/{version}/voicebox-server-cuda{ext}"
|
||||
CUDA_CHECKSUMS = {
|
||||
# Populated per release
|
||||
"0.2.0-windows": "sha256:abc123...",
|
||||
"0.2.0-linux": "sha256:def456...",
|
||||
}
|
||||
|
||||
def get_cuda_binary_dir() -> Path:
|
||||
"""Where CUDA binaries live. Inside the app's data directory."""
|
||||
return get_data_dir() / "backends"
|
||||
|
||||
def get_cuda_binary_path() -> Path | None:
|
||||
"""Return path to CUDA binary if it exists and is verified."""
|
||||
d = get_cuda_binary_dir()
|
||||
for name in ["voicebox-server-cuda.exe", "voicebox-server-cuda"]:
|
||||
p = d / name
|
||||
if p.exists():
|
||||
return p
|
||||
return None
|
||||
|
||||
async def download_cuda_binary(version: str):
|
||||
"""Download, assemble (if split), and verify the CUDA binary."""
|
||||
progress = get_progress_manager()
|
||||
dest_dir = get_cuda_binary_dir()
|
||||
dest_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
ext = ".exe" if sys.platform == "win32" else ""
|
||||
url = CUDA_DOWNLOAD_URL.format(version=version, ext=ext)
|
||||
|
||||
# Download with progress tracking
|
||||
temp_path = dest_dir / f"voicebox-server-cuda{ext}.download"
|
||||
async with httpx.AsyncClient(follow_redirects=True) as client:
|
||||
async with client.stream("GET", url) as response:
|
||||
total = int(response.headers.get("content-length", 0))
|
||||
downloaded = 0
|
||||
with open(temp_path, "wb") as f:
|
||||
async for chunk in response.aiter_bytes(chunk_size=1024 * 1024):
|
||||
f.write(chunk)
|
||||
downloaded += len(chunk)
|
||||
progress.update("cuda-backend", downloaded, total)
|
||||
|
||||
# Verify checksum
|
||||
sha256 = hashlib.sha256(temp_path.read_bytes()).hexdigest()
|
||||
expected = CUDA_CHECKSUMS.get(f"{version}-{sys.platform}")
|
||||
if expected and not expected.endswith(sha256):
|
||||
temp_path.unlink()
|
||||
raise ValueError(f"Checksum mismatch: expected {expected}, got sha256:{sha256}")
|
||||
|
||||
# Atomic move into place
|
||||
final_path = dest_dir / f"voicebox-server-cuda{ext}"
|
||||
temp_path.rename(final_path)
|
||||
|
||||
# Make executable on Unix
|
||||
if sys.platform != "win32":
|
||||
final_path.chmod(0o755)
|
||||
|
||||
progress.complete("cuda-backend")
|
||||
```
|
||||
|
||||
Key points:
|
||||
- Downloads to a `.download` temp file, verifies checksum, then atomically renames. No partial binaries left on crash.
|
||||
- Progress tracked via the existing `ProgressManager` so the frontend SSE system works unchanged.
|
||||
- CUDA binary lives in the **app data directory** (`data/backends/`), not alongside the app bundle. This avoids code-signing issues on macOS (though CUDA isn't relevant on macOS) and survives app updates.
|
||||
|
||||
#### 2c. Reassembly from Split Parts (GitHub Releases Fallback)
|
||||
|
||||
If the R2 download fails, fall back to downloading split parts from GitHub Releases:
|
||||
|
||||
```python
|
||||
async def download_cuda_from_github(version: str):
|
||||
"""Fallback: download split parts from GitHub Releases, reassemble."""
|
||||
base_url = f"https://github.com/jamiepine/voicebox/releases/download/v{version}"
|
||||
|
||||
# Get manifest
|
||||
manifest_url = f"{base_url}/voicebox-server-cuda.manifest"
|
||||
async with httpx.AsyncClient(follow_redirects=True) as client:
|
||||
manifest = (await client.get(manifest_url)).text
|
||||
parts = [p.strip() for p in manifest.strip().splitlines()]
|
||||
|
||||
# Download checksum
|
||||
sha256_url = f"{base_url}/voicebox-server-cuda.sha256"
|
||||
expected_sha = (await client.get(sha256_url)).text.split()[0]
|
||||
|
||||
# Download parts
|
||||
dest_dir = get_cuda_binary_dir()
|
||||
dest_dir.mkdir(parents=True, exist_ok=True)
|
||||
temp_path = dest_dir / "voicebox-server-cuda.exe.download"
|
||||
|
||||
total_downloaded = 0
|
||||
with open(temp_path, "wb") as f:
|
||||
for i, part_name in enumerate(parts):
|
||||
part_url = f"{base_url}/{part_name}"
|
||||
async with client.stream("GET", part_url) as response:
|
||||
async for chunk in response.aiter_bytes(chunk_size=1024 * 1024):
|
||||
f.write(chunk)
|
||||
total_downloaded += len(chunk)
|
||||
get_progress_manager().update(
|
||||
"cuda-backend", total_downloaded, None,
|
||||
message=f"Downloading part {i+1}/{len(parts)}"
|
||||
)
|
||||
|
||||
# Verify reassembled file
|
||||
sha256 = hashlib.sha256(temp_path.read_bytes()).hexdigest()
|
||||
if sha256 != expected_sha:
|
||||
temp_path.unlink()
|
||||
raise ValueError(f"Checksum mismatch after reassembly")
|
||||
|
||||
final_path = dest_dir / "voicebox-server-cuda.exe"
|
||||
temp_path.rename(final_path)
|
||||
get_progress_manager().complete("cuda-backend")
|
||||
```
|
||||
|
||||
### Phase 3: Backend Restart (The Swap)
|
||||
|
||||
This is the core of the feature: kill the CPU backend, launch the CUDA backend, frontend reconnects automatically.
|
||||
|
||||
#### 3a. New Tauri Command: `restart_server`
|
||||
|
||||
Add to `tauri/src-tauri/src/main.rs`:
|
||||
|
||||
```rust
|
||||
#[command]
|
||||
async fn restart_server(
|
||||
app: tauri::AppHandle,
|
||||
state: State<'_, ServerState>,
|
||||
use_cuda: Option<bool>,
|
||||
) -> Result<String, String> {
|
||||
println!("restart_server: use_cuda={:?}", use_cuda);
|
||||
|
||||
// 1. Stop the current server
|
||||
stop_server(state.clone()).await?;
|
||||
|
||||
// 2. Brief wait for port release
|
||||
tokio::time::sleep(tokio::time::Duration::from_millis(500)).await;
|
||||
|
||||
// 3. Start with the appropriate binary
|
||||
// The start_server logic needs to check for CUDA binary
|
||||
start_server(app, state, None).await
|
||||
}
|
||||
```
|
||||
|
||||
#### 3b. Modify `start_server` to Prefer CUDA Binary
|
||||
|
||||
The existing `start_server` uses `app.shell().sidecar("voicebox-server")` which resolves via Tauri's `externalBin` config. For the CUDA binary (which lives in the data directory, not the app bundle), we need an alternative launch path.
|
||||
|
||||
Modify `start_server` in `main.rs`:
|
||||
|
||||
```rust
|
||||
// After the existing sidecar logic, before spawning:
|
||||
|
||||
// Check for CUDA binary in data directory
|
||||
let cuda_binary = data_dir.join("backends")
|
||||
.join(if cfg!(windows) { "voicebox-server-cuda.exe" } else { "voicebox-server-cuda" });
|
||||
|
||||
let (mut rx, child) = if cuda_binary.exists() {
|
||||
println!("Found CUDA backend binary at {:?}", cuda_binary);
|
||||
|
||||
// Launch CUDA binary directly (not as Tauri sidecar)
|
||||
let mut cmd = app.shell().command(cuda_binary.to_str().unwrap());
|
||||
cmd = cmd.args([
|
||||
"--data-dir",
|
||||
data_dir.to_str().ok_or("Invalid data dir path")?,
|
||||
"--port",
|
||||
&SERVER_PORT.to_string(),
|
||||
]);
|
||||
if remote.unwrap_or(false) {
|
||||
cmd = cmd.args(["--host", "0.0.0.0"]);
|
||||
}
|
||||
cmd.spawn().map_err(|e| format!("Failed to spawn CUDA backend: {}", e))?
|
||||
} else {
|
||||
// Existing sidecar launch (CPU binary bundled with app)
|
||||
sidecar.spawn().map_err(|e| format!("Failed to spawn: {}", e))?
|
||||
};
|
||||
```
|
||||
|
||||
Key decisions:
|
||||
- CUDA binary is launched via `app.shell().command()` (arbitrary path), not `app.shell().sidecar()` (bundled path). Tauri's sidecar system only resolves binaries within the app bundle.
|
||||
- The CUDA binary gets the same args (`--data-dir`, `--port`) as the CPU binary. It's the same `server.py` entry point.
|
||||
- Preference: if CUDA binary exists, use it. Otherwise fall back to bundled CPU. No user configuration needed.
|
||||
|
||||
#### 3c. Frontend: Trigger Restart After Download
|
||||
|
||||
Add to the platform lifecycle interface (`app/src/platform/types.ts`):
|
||||
|
||||
```typescript
|
||||
interface PlatformLifecycle {
|
||||
startServer(remote?: boolean): Promise<string>;
|
||||
stopServer(): Promise<void>;
|
||||
restartServer(useCuda?: boolean): Promise<string>; // new
|
||||
// ...
|
||||
}
|
||||
```
|
||||
|
||||
Implement in `tauri/src/platform/lifecycle.ts`:
|
||||
|
||||
```typescript
|
||||
async restartServer(useCuda?: boolean): Promise<string> {
|
||||
const result = await invoke<string>('restart_server', { useCuda });
|
||||
this.onServerReady?.();
|
||||
return result;
|
||||
}
|
||||
```
|
||||
|
||||
#### 3d. Frontend: GPU Settings UI
|
||||
|
||||
Add a section to the Server Settings page (or Model Management). Minimal UI:
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────┐
|
||||
│ GPU Acceleration │
|
||||
│ │
|
||||
│ Status: CPU only (no CUDA backend) │
|
||||
│ │
|
||||
│ [Download CUDA Backend (2.4 GB)] │
|
||||
│ │
|
||||
│ Requires an NVIDIA GPU with 4+ GB VRAM. │
|
||||
│ The app will restart its backend process │
|
||||
│ after download. Your work is preserved. │
|
||||
└─────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
After download:
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────┐
|
||||
│ GPU Acceleration │
|
||||
│ │
|
||||
│ Status: ✓ CUDA backend active (RTX 4090) │
|
||||
│ │
|
||||
│ [Switch to CPU] [Delete CUDA Backend] │
|
||||
└─────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
#### 3e. Frontend: Reconnection During Restart
|
||||
|
||||
The current health poll interval is 30 seconds — too slow for a restart UX. During a restart, temporarily increase polling:
|
||||
|
||||
```typescript
|
||||
// In the component that triggers restart:
|
||||
const restart = async () => {
|
||||
setRestarting(true);
|
||||
try {
|
||||
await platform.lifecycle.restartServer(true);
|
||||
} catch (e) {
|
||||
// Frontend will show "reconnecting" state
|
||||
}
|
||||
// Aggressively poll until health check succeeds
|
||||
const interval = setInterval(async () => {
|
||||
try {
|
||||
await apiClient.getHealth();
|
||||
clearInterval(interval);
|
||||
setRestarting(false);
|
||||
queryClient.invalidateQueries(); // Refresh all data
|
||||
} catch {}
|
||||
}, 1000); // Poll every 1s during restart
|
||||
// Safety timeout
|
||||
setTimeout(() => clearInterval(interval), 30000);
|
||||
};
|
||||
```
|
||||
|
||||
### Phase 4: Auto-Detection on Startup
|
||||
|
||||
No user action needed on subsequent launches. The preference logic in `start_server` (Phase 3b) handles this:
|
||||
|
||||
1. App launches → `start_server` called
|
||||
2. Check `data/backends/voicebox-server-cuda{.exe}`
|
||||
3. If exists → launch CUDA binary
|
||||
4. If not → launch bundled CPU binary
|
||||
|
||||
The user downloads CUDA once, and every future app launch (including after updates) uses it automatically. The CUDA binary lives in the data directory, not the app bundle, so app updates don't overwrite it.
|
||||
|
||||
### Phase 5: Handling Version Mismatches
|
||||
|
||||
When the app updates but the CUDA binary is from an older version, the API might be incompatible. Handle this by:
|
||||
|
||||
1. Add `--version` flag to `server.py`:
|
||||
|
||||
```python
|
||||
parser.add_argument("--version", action="store_true")
|
||||
# If invoked with --version, print version and exit
|
||||
if args.version:
|
||||
from backend import __version__
|
||||
print(f"voicebox-server {__version__}")
|
||||
sys.exit(0)
|
||||
```
|
||||
|
||||
2. In `start_server` (Rust), before launching the CUDA binary:
|
||||
|
||||
```rust
|
||||
// Quick version check
|
||||
let version_output = std::process::Command::new(cuda_binary.to_str().unwrap())
|
||||
.arg("--version")
|
||||
.output();
|
||||
|
||||
match version_output {
|
||||
Ok(output) => {
|
||||
let version = String::from_utf8_lossy(&output.stdout);
|
||||
let app_version = env!("CARGO_PKG_VERSION");
|
||||
if !version.contains(app_version) {
|
||||
println!("CUDA binary version mismatch (app: {}, cuda: {}), falling back to CPU",
|
||||
app_version, version.trim());
|
||||
// Fall through to CPU sidecar launch
|
||||
}
|
||||
}
|
||||
Err(_) => {
|
||||
println!("Failed to check CUDA binary version, falling back to CPU");
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
3. Frontend shows a notification: "Your GPU backend needs an update. [Download latest] or [Use CPU for now]"
|
||||
|
||||
## Files Changed
|
||||
|
||||
### New Files
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `backend/cuda_download.py` | Download, reassemble, verify CUDA binary |
|
||||
| `scripts/split_binary.py` | Split binary into <2 GB chunks for GitHub Releases |
|
||||
| `.github/workflows/build-cuda.yml` | CI: build + upload CUDA binary |
|
||||
|
||||
### Modified Files
|
||||
|
||||
| File | Change |
|
||||
|------|--------|
|
||||
| `tauri/src-tauri/src/main.rs` | Add `restart_server` command, modify `start_server` to check for CUDA binary in data dir |
|
||||
| `backend/server.py` | Add `--version` flag |
|
||||
| `backend/main.py` | Add `/backend/download-cuda`, `/backend/cuda-status`, `/backend/progress/cuda-backend` endpoints |
|
||||
| `backend/build_binary.py` | Accept `--cuda` flag to build CUDA variant |
|
||||
| `app/src/platform/types.ts` | Add `restartServer` to lifecycle interface |
|
||||
| `tauri/src/platform/lifecycle.ts` | Implement `restartServer` |
|
||||
| `app/src/components/ServerSettings/` | New GPU acceleration section |
|
||||
| `.github/workflows/release.yml` | Trigger CUDA build workflow on tag |
|
||||
|
||||
### NOT Changed
|
||||
|
||||
| File | Why |
|
||||
|------|-----|
|
||||
| `backend/backends/__init__.py` | No changes to the TTSBackend singleton or factory. CUDA binary runs the same code. |
|
||||
| `backend/backends/pytorch_backend.py` | Already detects CUDA at runtime (line 28-49). No changes needed. |
|
||||
| `app/src/lib/api/client.ts` | API is identical between CPU and CUDA backends. |
|
||||
| `app/src/lib/hooks/useGenerationForm.ts` | Generation flow is unchanged. |
|
||||
|
||||
## What This Doesn't Solve
|
||||
|
||||
- **Multi-model support** — This is purely about GPU acceleration. LuxTTS, Chatterbox, etc. need the in-process model registry, which is an independent workstream.
|
||||
- **AMD GPU support** — DirectML/ROCm needs a different PyTorch build. Same pattern applies (another binary variant) but deferred.
|
||||
- **Linux CUDA** — Same approach works, just another CI matrix entry. Can be added in the same release or shortly after.
|
||||
- **Remote server mode** — Users who want to run TTS on a different machine still need the external provider architecture. Separate concern.
|
||||
|
||||
## What This DOES Solve
|
||||
|
||||
- **19 "GPU not detected" issues** — Users download the CUDA backend, restart, GPU works.
|
||||
- **2 GB GitHub Release limit** — Binary splitting + R2 hosting.
|
||||
- **Update burden** — App updates don't re-download the 2.4 GB CUDA binary. It persists in the data directory.
|
||||
- **First-run experience** — App works immediately on CPU. GPU is an optional enhancement, not a setup blocker.
|
||||
|
||||
## Rollout Plan
|
||||
|
||||
1. Build and test CUDA binary locally on Windows with an NVIDIA GPU.
|
||||
2. Set up R2 bucket at `downloads.voicebox.sh/cuda/`.
|
||||
3. Ship the backend restart + download UI in v0.2.0.
|
||||
4. Announce: "GPU acceleration is here — one click in Settings."
|
||||
|
||||
## Risks
|
||||
|
||||
| Risk | Mitigation |
|
||||
|------|-----------|
|
||||
| CUDA binary doesn't work on some GPU/driver combos | `/health` endpoint reports GPU info. Fallback to CPU if CUDA init fails. Clear error message. |
|
||||
| Antivirus flags downloaded binary (Windows) | Code-sign the CUDA binary in CI. Document AV exceptions. |
|
||||
| Data dir CUDA binary survives app uninstall | Document in uninstall notes. Not a real problem — it's just a file. |
|
||||
| Version mismatch after app update | Version check on startup (Phase 5). Auto-fallback to CPU. Prompt to re-download. |
|
||||
| R2 downtime | GitHub Releases split-binary fallback. |
|
||||
| Download interrupted | Temp file with `.download` extension. Atomic rename on completion. Resume not implemented in v1 — restart download from scratch. |
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user