Problem: The server binary with CUDA support was 2.9GB, causing:
- MSI installer failures in CI (WiX can't handle 3GB files)
- Massive downloads for all users (even those without GPUs)
- Poor user experience
Solution: Build two separate server binaries:
- voicebox-server.exe (CPU-only, ~295MB) - ships with installer
- voicebox-server-cuda.exe (CUDA, ~2.9GB) - optional download
Changes:
- backend/build_binary.py: Added 'variant' parameter for CPU/CUDA builds
- backend/build_cpu.bat: Script to build CPU-only binary
- backend/build_cuda.bat: Script to build CUDA binary
- backend/build_both.bat: Script to build both binaries
- backend/build_cpu.sh: Unix build script for CPU binary
- .github/workflows/release.yml: Build both variants, upload CUDA separately
- tauri/vite.config.ts: Externalize Tauri plugins to fix build
- docs/dual-server-binaries.md: Complete documentation
Results:
- Installer size reduced from 3GB to ~500MB (6x smaller)
- CI builds now succeed (WiX can handle 500MB)
- GPU users can opt-in to download CUDA support
- Better bandwidth usage for CPU-only users
Next steps:
- Frontend implementation to detect GPU and download CUDA binary
- Settings UI to toggle between CPU/CUDA modes
Co-Authored-By: Claude Sonnet 4.5 (1M context) <[email protected]>
- Added a step to install PyTorch with CUDA for Windows in the release workflow.
- Updated model references in backend/main.py to use openai/whisper models instead of mlx-community for the MLX backend.
- Added support for MLX backend on Apple Silicon, enabling optimized performance for TTS and STT tasks.
- Updated release workflow to include MLX-specific dependencies and configurations for macOS platforms.
- Refactored backend code to dynamically select between MLX and PyTorch based on the runtime environment.
- Enhanced model loading and inference logic to accommodate backend-specific requirements, including updated model IDs and hidden imports.
- Improved health check and model status reporting to reflect the active backend type.
- Streamlined caching mechanisms to support both backend types, ensuring compatibility and performance.
- Introduced steps to install the Apple API key and codesigning certificate for macOS platforms.
- Enhanced the release workflow to support secure handling of Apple credentials for code signing.
- Updated environment variables to include necessary Apple signing information for Tauri builds.
- Added a new UpdateNotification component to inform users about available updates and facilitate installation.
- Integrated useAutoUpdater hook for managing update checks and installations.
- Updated package.json with new build and generate scripts for release preparation and key generation.
- Enhanced tauri configuration to support autoupdater with signing keys and endpoints.
- Created scripts for preparing signed releases and updating icons, ensuring a streamlined workflow for asset management.
- Added detailed documentation for autoupdater setup and icon update workflow.
llvmlite only supports LLVM up to version 20, but brew install llvm
installs version 21. Update macOS runners to install llvm@20 specifically.
Co-Authored-By: Claude Sonnet 4.5 (1M context) <[email protected]>
- Install llvm-dev on Ubuntu for llvmlite compilation
- Install LLVM via Homebrew on macOS and configure PATH
- Exclude matplotlib, IPython, notebook, pytest, tensorboard from bundle to reduce size
- Keep --onefile mode with module exclusions to avoid 4GB limit
