harryandcapy-ai-staging[bot]
ab9a19790c
Four related fixes uncovered while getting Docker running on a Linux host without AVX-512 and with an NVIDIA GPU: - pedalboard>=0.9.21 ships a Linux wheel with AVX-512 instructions baked into its native extension, which SIGILLs (exit 132) on import on any CPU without AVX-512 support. Pin below the regression until upstream fixes it (spotify/pedalboard#454). - The rocminfo probe in app.py ran unconditionally on every build variant, always failing and logging on non-ROCm systems. Gate it on /dev/kfd actually being present. - sox (required by qwen-tts's X-vector extractor) and a C compiler (required by Triton to JIT-compile its CUDA driver shim on first use) were missing from the runtime image, causing hard failures once those code paths were actually exercised. - Named volumes and bind mounts (HF cache, app data, generated audio) are created root-owned by Docker on first use, but the app runs as the unprivileged voicebox user. chown the mount points in the entrypoint before dropping privileges so this self-heals on every start regardless of host UID.
Voicebox Backend
FastAPI server powering voice cloning, speech generation, and audio processing. Runs locally as a Tauri sidecar or standalone via python -m backend.main.
Running
# Via justfile (recommended)
just dev:server
# Standalone
python -m backend.main --host 127.0.0.1 --port 17493
# With custom data directory
python -m backend.main --data-dir /path/to/data
The server auto-initializes the SQLite database on first startup. Models are downloaded from HuggingFace on first use.
Architecture
backend/
app.py # FastAPI app factory, CORS, lifecycle events
main.py # Entry point (imports app, runs uvicorn)
config.py # Data directory paths and configuration
models.py # Pydantic request/response schemas
server.py # Tauri sidecar launcher, parent-pid watchdog
routes/ # Thin HTTP handlers — validation, delegation, response formatting
services/ # Business logic, CRUD, orchestration
backends/ # TTS/STT engine implementations (MLX, PyTorch, etc.)
database/ # ORM models, session management, migrations, seed data
utils/ # Shared utilities (audio, effects, caching, progress tracking)
Request flow
HTTP request
-> routes/ (validate input, parse params)
-> services/ (business logic, database queries, orchestration)
-> backends/ (TTS/STT inference)
-> utils/ (audio processing, effects, caching)
Route handlers are intentionally thin. They validate input, delegate to a service function, and format the response. All business logic lives in services/.
Key modules
services/generation.py -- Single run_generation() function that handles all three generation modes (generate, retry, regenerate). Manages model loading, voice prompt creation, chunked inference, normalization, effects, and version persistence.
services/task_queue.py -- Serial generation queue. Ensures only one GPU inference runs at a time. Background tasks are tracked to prevent garbage collection.
backends/__init__.py -- Protocol definitions (TTSBackend, STTBackend), model config registry, and factory functions. Adding a new engine means implementing the protocol and registering a config entry.
backends/base.py -- Shared utilities used across all engine implementations: HuggingFace cache checks, device detection, voice prompt combination, progress tracking.
database/ -- SQLAlchemy ORM models with a re-exporting __init__.py for backward compatibility. Migrations run automatically on startup.
Backend selection
The server detects the best inference backend at startup:
| Platform | Backend | Acceleration |
|---|---|---|
| macOS (Apple Silicon) | MLX | Metal / Neural Engine |
| Windows / Linux (NVIDIA) | PyTorch | CUDA |
| Linux (AMD) | PyTorch | ROCm |
| Intel Arc | PyTorch | IPEX / XPU |
| Windows (any GPU) | PyTorch | DirectML |
| Any | PyTorch | CPU fallback |
Detection is handled by utils/platform_detect.py. Both backends implement the same TTSBackend protocol, so the API layer is engine-agnostic.
API
90 endpoints organized by domain. Full interactive documentation available at http://localhost:17493/docs when the server is running.
| Domain | Prefix | Description |
|---|---|---|
| Health | /, /health |
Server status, GPU info, filesystem checks |
| Profiles | /profiles |
Voice profile CRUD, samples, avatars, import/export |
| Channels | /channels |
Audio channel management and voice assignment |
| Generation | /generate |
TTS generation, retry, regenerate, status SSE |
| History | /history |
Generation history, search, favorites, export |
| Transcription | /transcribe |
Whisper-based audio-to-text |
| Stories | /stories |
Multi-track timeline editor, audio export |
| Effects | /effects |
Effect presets, preview, version management |
| Audio | /audio, /samples |
Audio file serving |
| Models | /models |
Load, unload, download, migrate, status |
| Tasks | /tasks, /cache |
Active task tracking, cache management |
| CUDA | /backend/cuda-* |
CUDA binary download and management |
Quick examples
# Generate speech
curl -X POST http://localhost:17493/generate \
-H "Content-Type: application/json" \
-d '{"text": "Hello world", "profile_id": "...", "language": "en"}'
# List profiles
curl http://localhost:17493/profiles
# Stream generation status (SSE)
curl http://localhost:17493/generate/{id}/status
Data directory
{data_dir}/
voicebox.db # SQLite database
profiles/{id}/ # Voice samples per profile
generations/ # Generated audio files
cache/ # Voice prompt cache (memory + disk)
backends/ # Downloaded CUDA binary (if applicable)
Default location is the OS-specific app data directory. Override with --data-dir or the VOICEBOX_DATA_DIR environment variable.
Code quality
Linting and formatting are enforced by ruff, configured in pyproject.toml. See STYLE_GUIDE.md for conventions.
just check-python # lint + format check
just fix-python # auto-fix lint issues + reformat
just test # run pytest
Dependencies
Runtime dependencies are in requirements.txt. macOS-only MLX dependencies are in requirements-mlx.txt. Dev tools (ruff, pytest) are installed automatically by just setup-python.