a383ff6863
torch is compiled against numpy 1.x. numpy 2.x changed the ABI version returned by PyArray_GetNDArrayCVersion() (0x01000009 → 0x02000000), so torch's is_numpy_available() always returns False and torch.from_numpy() raises RuntimeError. This causes TTS generation to fail with: ValueError: Unable to create tensor, you should probably activate padding with 'padding=True' Two fixes: 1. Pin numpy<2.0 in requirements.txt so new builds bundle a compatible numpy version. (The existing comment already flagged this intention but the upper bound was never added.) 2. Add a PyInstaller runtime hook (pyi_rth_numpy_compat.py) that installs a ctypes memmove fallback for torch.from_numpy() at startup. Runtime hooks run after FrozenImporter is registered so frozen torch is importable. The fallback catches RuntimeError from the C-level ABI check and copies the numpy array into a new tensor via raw memory copy, bypassing the check entirely. This is a belt-and-suspenders fix that works regardless of the bundled numpy version. Co-authored-by: aimaaaimaa <[email protected]> Co-authored-by: Claude Sonnet 4.6 <[email protected]>
Voicebox Backend
FastAPI server powering voice cloning, speech generation, and audio processing. Runs locally as a Tauri sidecar or standalone via python -m backend.main.
Running
# Via justfile (recommended)
just dev:server
# Standalone
python -m backend.main --host 127.0.0.1 --port 17493
# With custom data directory
python -m backend.main --data-dir /path/to/data
The server auto-initializes the SQLite database on first startup. Models are downloaded from HuggingFace on first use.
Architecture
backend/
app.py # FastAPI app factory, CORS, lifecycle events
main.py # Entry point (imports app, runs uvicorn)
config.py # Data directory paths and configuration
models.py # Pydantic request/response schemas
server.py # Tauri sidecar launcher, parent-pid watchdog
routes/ # Thin HTTP handlers — validation, delegation, response formatting
services/ # Business logic, CRUD, orchestration
backends/ # TTS/STT engine implementations (MLX, PyTorch, etc.)
database/ # ORM models, session management, migrations, seed data
utils/ # Shared utilities (audio, effects, caching, progress tracking)
Request flow
HTTP request
-> routes/ (validate input, parse params)
-> services/ (business logic, database queries, orchestration)
-> backends/ (TTS/STT inference)
-> utils/ (audio processing, effects, caching)
Route handlers are intentionally thin. They validate input, delegate to a service function, and format the response. All business logic lives in services/.
Key modules
services/generation.py -- Single run_generation() function that handles all three generation modes (generate, retry, regenerate). Manages model loading, voice prompt creation, chunked inference, normalization, effects, and version persistence.
services/task_queue.py -- Serial generation queue. Ensures only one GPU inference runs at a time. Background tasks are tracked to prevent garbage collection.
backends/__init__.py -- Protocol definitions (TTSBackend, STTBackend), model config registry, and factory functions. Adding a new engine means implementing the protocol and registering a config entry.
backends/base.py -- Shared utilities used across all engine implementations: HuggingFace cache checks, device detection, voice prompt combination, progress tracking.
database/ -- SQLAlchemy ORM models with a re-exporting __init__.py for backward compatibility. Migrations run automatically on startup.
Backend selection
The server detects the best inference backend at startup:
| Platform | Backend | Acceleration |
|---|---|---|
| macOS (Apple Silicon) | MLX | Metal / Neural Engine |
| Windows / Linux (NVIDIA) | PyTorch | CUDA |
| Linux (AMD) | PyTorch | ROCm |
| Intel Arc | PyTorch | IPEX / XPU |
| Windows (any GPU) | PyTorch | DirectML |
| Any | PyTorch | CPU fallback |
Detection is handled by utils/platform_detect.py. Both backends implement the same TTSBackend protocol, so the API layer is engine-agnostic.
API
90 endpoints organized by domain. Full interactive documentation available at http://localhost:17493/docs when the server is running.
| Domain | Prefix | Description |
|---|---|---|
| Health | /, /health |
Server status, GPU info, filesystem checks |
| Profiles | /profiles |
Voice profile CRUD, samples, avatars, import/export |
| Channels | /channels |
Audio channel management and voice assignment |
| Generation | /generate |
TTS generation, retry, regenerate, status SSE |
| History | /history |
Generation history, search, favorites, export |
| Transcription | /transcribe |
Whisper-based audio-to-text |
| Stories | /stories |
Multi-track timeline editor, audio export |
| Effects | /effects |
Effect presets, preview, version management |
| Audio | /audio, /samples |
Audio file serving |
| Models | /models |
Load, unload, download, migrate, status |
| Tasks | /tasks, /cache |
Active task tracking, cache management |
| CUDA | /backend/cuda-* |
CUDA binary download and management |
Quick examples
# Generate speech
curl -X POST http://localhost:17493/generate \
-H "Content-Type: application/json" \
-d '{"text": "Hello world", "profile_id": "...", "language": "en"}'
# List profiles
curl http://localhost:17493/profiles
# Stream generation status (SSE)
curl http://localhost:17493/generate/{id}/status
Data directory
{data_dir}/
voicebox.db # SQLite database
profiles/{id}/ # Voice samples per profile
generations/ # Generated audio files
cache/ # Voice prompt cache (memory + disk)
backends/ # Downloaded CUDA binary (if applicable)
Default location is the OS-specific app data directory. Override with --data-dir or the VOICEBOX_DATA_DIR environment variable.
Code quality
Linting and formatting are enforced by ruff, configured in pyproject.toml. See STYLE_GUIDE.md for conventions.
just check-python # lint + format check
just fix-python # auto-fix lint issues + reformat
just test # run pytest
Dependencies
Runtime dependencies are in requirements.txt. macOS-only MLX dependencies are in requirements-mlx.txt. Dev tools (ruff, pytest) are installed automatically by just setup-python.