Enhance MLX and PyTorch Backend Integration

- Added support for MLX backend on Apple Silicon, enabling optimized performance for TTS and STT tasks.
- Implemented platform detection to dynamically select between MLX and PyTorch based on the runtime environment.
- Updated build process to include MLX-specific dependencies and configurations for macOS.
- Refactored backend code to improve model loading and inference logic, accommodating backend-specific requirements.
- Enhanced documentation to clarify backend selection and performance benefits for different platforms.
- Streamlined installation instructions and troubleshooting guidance for MLX-related issues.
This commit is contained in:
Jamie Pine
2026-01-29 23:11:48 -08:00
parent 081f45e680
commit 94487f32a5
17 changed files with 200 additions and 82 deletions
+5 -1
View File
@@ -121,8 +121,12 @@ bun run dev
### Production
```bash
# Build everything (server binary + Tauri app)
bun run build
# Or build separately:
# 1. Build server binary (PyInstaller)
./scripts/build-server.sh
bun run build:server
# 2. Build Tauri app (includes server)
cd tauri && bun run tauri build
+33 -26
View File
@@ -10,46 +10,52 @@ Voicebox uses a multi-step build process to create platform-specific installers.
## Quick Build
```bash
# Build for your current platform
# Build for your current platform (automatically builds server binary first)
make build
# Or manually
cd tauri && bun run tauri build
bun run build
```
## Build Steps
This automatically:
1. Builds the Python server binary (`bun run build:server`)
2. Builds the Tauri app (`cd tauri && bun run tauri build`)
### 1. Build Server Binary
## Build Process
The Python backend must be compiled into a standalone executable first:
The build process consists of two steps, but `bun run build` handles both automatically:
```bash
./scripts/build-server.sh
```
### 1. Server Binary Build (Automatic)
This uses PyInstaller to create a binary in `tauri/src-tauri/binaries/`.
The Python backend is compiled into a standalone executable using PyInstaller. This happens automatically when you run `bun run build`.
**Platform-specific binaries:**
- macOS: `voicebox-server-aarch64-apple-darwin` or `voicebox-server-x86_64-apple-darwin`
- Windows: `voicebox-server-x86_64-pc-windows-msvc.exe`
- Linux: `voicebox-server-x86_64-unknown-linux-gnu`
- macOS (Apple Silicon): `voicebox-server-aarch64-apple-darwin` (includes MLX backend)
- macOS (Intel): `voicebox-server-x86_64-apple-darwin` (PyTorch backend)
- Windows: `voicebox-server-x86_64-pc-windows-msvc.exe` (PyTorch backend)
- Linux: `voicebox-server-x86_64-unknown-linux-gnu` (PyTorch backend)
<Note>
The build script automatically detects your platform and creates the appropriate binary.
The build script automatically detects your platform and includes the appropriate backend (MLX for Apple Silicon, PyTorch for others).
</Note>
### 2. Build Tauri App
**Manual build (if needed):**
```bash
cd tauri
bun run tauri build
bun run build:server
```
This will:
1. Build the React frontend (Vite)
2. Compile the Rust backend
3. Bundle the server binary as a sidecar
4. Create platform-specific installers
### 2. Tauri App Build (Automatic)
The Tauri app build is also handled automatically, which:
1. Builds the React frontend (Vite)
2. Compiles the Rust backend
3. Bundles the server binary as a sidecar
4. Creates platform-specific installers
**Manual build (if needed):**
```bash
cd tauri && bun run tauri build
```
### 3. Output
@@ -91,7 +97,9 @@ If you're developing Qwen3-TTS locally:
```bash
export QWEN_TTS_PATH=~/path/to/Qwen3-TTS
./scripts/build-server.sh
bun run build:server # Build server binary only
# or
bun run build # Build everything
```
This makes PyInstaller use your local version instead of the pip package.
@@ -216,7 +224,7 @@ See [CONTRIBUTING.md](/development/contributing) for the full release workflow.
<Accordion title="Tauri Build Fails">
**Common issues:**
- Rust not installed: `curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh`
- Server binary missing: Run `./scripts/build-server.sh` first
- Server binary missing: Usually auto-built, but can run manually: `./scripts/build-server.sh`
- Node modules outdated: `bun install`
**Solution:**
@@ -225,8 +233,7 @@ See [CONTRIBUTING.md](/development/contributing) for the full release workflow.
cd tauri/src-tauri
cargo clean
cd ../..
./scripts/build-server.sh
bun run tauri build
bun run build # Automatically builds server binary first
```
</Accordion>
+6 -9
View File
@@ -80,19 +80,16 @@ venv\Scripts\activate # Windows
# Install Python dependencies
pip install -r requirements.txt
# Install MLX dependencies (Apple Silicon only - for faster inference)
# On Apple Silicon, this enables native Metal acceleration
if [[ $(uname -m) == "arm64" ]]; then
pip install -r requirements-mlx.txt
fi
# Install Qwen3-TTS
pip install git+https://github.com/QwenLM/Qwen3-TTS.git
```
### 3. Initialize Database
```bash
cd backend
python -c "from database import init_db; init_db()"
```
This creates the SQLite database at `data/voicebox.db`.
## Running in Development
Development requires **two terminals**: one for the Python backend, one for the Tauri app.