Labyricorn/voicebox
1
0
Fork 0
mirror of https://github.com/jamiepine/voicebox.git synced 2026-09-16 21:30:39 -07:00
Code Issues Packages Projects Releases Wiki Activity

rewrite backend README, remove completed refactor plan, update style guide

Browse Source 
Replace the outdated backend README (473 lines of stale API docs and
pre-refactor file tree) with a concise architecture document covering
module structure, request flow, backend selection, API domain overview,
and development commands.

Delete REFACTOR_PLAN.md -- all phases are complete.

Update STYLE_GUIDE.md to remove refactor plan references and replace
the verbose target layout with the current actual structure.
This commit is contained in:
James Pine
2026-03-16 02:18:34 -07:00
parent 7c1ea0a1e1
commit 7b0fbfb567
3 changed files with 122 additions and 693 deletions
Download Patch File Download Diff File Expand all files Collapse all files
backend/README.md
+107 -445
View File
@@ -1,473 +1,135 @@
# voicebox Backend
# Voicebox Backend
Production-quality FastAPI backend for Qwen3-TTS voice cloning.
FastAPI server powering voice cloning, speech generation, and audio processing. Runs locally as a Tauri sidecar or standalone via `python -m backend.main`.
## Features
## Running
- ✅ **Voice Profile Management** - Create, update, delete voice profiles with multi-sample support
- ✅ **Voice Cloning** - Generate speech using voice profiles with caching
- ✅ **Generation History** - Full history tracking with search and filtering
- ✅ **Transcription** - Whisper-based audio transcription
- ✅ **Multi-Sample Profiles** - Combine multiple reference samples for better quality
- ✅ **Voice Prompt Caching** - Dual memory + disk caching for fast generation
- ✅ **Audio Validation** - Automatic validation of reference audio quality
- ✅ **Model Management** - Lazy loading and VRAM management
```bash
# Via justfile (recommended)
just dev:server
# Standalone
python -m backend.main --host 127.0.0.1 --port 17493
# With custom data directory
python -m backend.main --data-dir /path/to/data
```
The server auto-initializes the SQLite database on first startup. Models are downloaded from HuggingFace on first use.
## Architecture
```
backend/
├── main.py # FastAPI app with all routes
├── server.py # PyInstaller entry point, CLI arg parsing
├── models.py # Pydantic request/response models
├── config.py # Data directory configuration
├── database.py # SQLAlchemy ORM models + migrations
├── platform_detect.py # Platform detection for backend selection
├── tts.py # TTS backend facade
├── transcribe.py # STT backend facade
├── profiles.py # Voice profile CRUD
├── history.py # Generation history CRUD
├── channels.py # Audio channel management
├── stories.py # Story/timeline management + audio export
├── effects.py # Effect preset CRUD
├── versions.py # Generation version management
├── export_import.py # ZIP export/import for profiles and generations
├── backends/ # Backend implementations
│ ├── __init__.py # Protocols, model config registry, factory functions
│ ├── mlx_backend.py # MLX backend (Apple Silicon)
│ ├── pytorch_backend.py # PyTorch backend (Windows/Linux/Intel)
│ ├── chatterbox_backend.py # Chatterbox Multilingual TTS
│ ├── chatterbox_turbo_backend.py # Chatterbox Turbo TTS
│ └── luxtts_backend.py # LuxTTS backend
└── utils/
├── audio.py # Audio load/save/normalize/validate/trim
├── cache.py # Voice prompt caching (memory + disk)
├── effects.py # Audio effects engine (pedalboard)
├── chunked_tts.py # Text chunking + audio concatenation
├── progress.py # SSE progress tracking
├── tasks.py # Active task tracking
├── hf_progress.py # HuggingFace download progress tracking
├── hf_offline_patch.py # HuggingFace offline mode patch (MLX)
└── images.py # Avatar image processing
app.py # FastAPI app factory, CORS, lifecycle events
main.py # Entry point (imports app, runs uvicorn)
config.py # Data directory paths and configuration
models.py # Pydantic request/response schemas
server.py # Tauri sidecar launcher, parent-pid watchdog
routes/ # Thin HTTP handlers — validation, delegation, response formatting
services/ # Business logic, CRUD, orchestration
backends/ # TTS/STT engine implementations (MLX, PyTorch, etc.)
database/ # ORM models, session management, migrations, seed data
utils/ # Shared utilities (audio, effects, caching, progress tracking)
```
### Backend Selection
Voicebox automatically selects the best backend based on platform:
- **Apple Silicon (M1/M2/M3)**: Uses MLX backend with native Metal acceleration (4-5x faster)
- **Windows/Linux/Intel Mac**: Uses PyTorch backend (CUDA GPU if available, CPU fallback)
The backend is detected at runtime via `platform_detect.py`. Both backends implement the same interface, so the API remains consistent across platforms.
## API Endpoints
### Health & Info
#### `GET /`
Root endpoint with version info.
#### `GET /health`
Health check with model status.
**Response:**
```json
{
"status": "healthy",
"model_loaded": true,
"gpu_available": true,
"gpu_type": "Metal (Apple Silicon via MLX)",
"backend_type": "mlx",
"vram_used_mb": null
}
```
**Backend Types:**
- `"mlx"` - MLX backend (Apple Silicon with Metal acceleration)
- `"pytorch"` - PyTorch backend (Windows/Linux/Intel Mac)
### Voice Profiles
**Note:** The database is automatically initialized when the server starts. No manual setup required.
#### `POST /profiles`
Create a new voice profile.
**Request:**
```json
{
"name": "My Voice",
"description": "Optional description",
"language": "en"
}
```
**Response:**
```json
{
"id": "uuid",
"name": "My Voice",
"description": "Optional description",
"language": "en",
"created_at": "2024-01-01T00:00:00Z",
"updated_at": "2024-01-01T00:00:00Z"
}
```
#### `GET /profiles`
List all voice profiles.
#### `GET /profiles/{profile_id}`
Get a specific profile.
#### `PUT /profiles/{profile_id}`
Update a profile.
#### `DELETE /profiles/{profile_id}`
Delete a profile and all associated samples.
#### `POST /profiles/{profile_id}/samples`
Add a sample to a profile.
**Form Data:**
- `file`: Audio file (WAV, MP3, etc.)
- `reference_text`: Transcript of the audio
**Response:**
```json
{
"id": "sample-uuid",
"profile_id": "profile-uuid",
"audio_path": "/path/to/sample.wav",
"reference_text": "This is my voice"
}
```
#### `GET /profiles/{profile_id}/samples`
List all samples for a profile.
#### `DELETE /profiles/samples/{sample_id}`
Delete a specific sample.
### Generation
#### `POST /generate`
Generate speech from text using a voice profile.
**Request:**
```json
{
"profile_id": "uuid",
"text": "Hello, this is a test.",
"language": "en",
"seed": 42
}
```
**Response:**
```json
{
"id": "generation-uuid",
"profile_id": "profile-uuid",
"text": "Hello, this is a test.",
"language": "en",
"audio_path": "/path/to/audio.wav",
"duration": 2.5,
"seed": 42,
"created_at": "2024-01-01T00:00:00Z"
}
```
### History
#### `GET /history`
List generation history with optional filters.
**Query Parameters:**
- `profile_id` (optional): Filter by profile
- `search` (optional): Search in text content
- `limit` (default: 50): Results per page
- `offset` (default: 0): Pagination offset
#### `GET /history/{generation_id}`
Get a specific generation.
#### `DELETE /history/{generation_id}`
Delete a generation.
#### `GET /history/stats`
Get generation statistics.
**Response:**
```json
{
"total_generations": 100,
"total_duration_seconds": 250.5,
"generations_by_profile": {
"profile-uuid-1": 50,
"profile-uuid-2": 50
}
}
```
### Audio Files
#### `GET /audio/{generation_id}`
Download generated audio file.
Returns WAV file with appropriate headers.
### Transcription
#### `POST /transcribe`
Transcribe audio file to text.
**Form Data:**
- `file`: Audio file
- `language` (optional): Language hint (en or zh)
**Response:**
```json
{
"text": "Transcribed text here",
"duration": 5.5
}
```
### Model Management
#### `POST /models/load`
Manually load TTS model.
**Query Parameters:**
- `model_size`: Model size (1.7B or 0.6B)
#### `POST /models/unload`
Unload TTS model to free memory.
## Database Schema
### profiles
- `id`: UUID primary key
- `name`: Profile name (unique)
- `description`: Optional description
- `language`: Language code (en/zh)
- `created_at`: Creation timestamp
- `updated_at`: Last update timestamp
### profile_samples
- `id`: UUID primary key
- `profile_id`: Foreign key to profiles
- `audio_path`: Path to audio file
- `reference_text`: Transcript
### generations
- `id`: UUID primary key
- `profile_id`: Foreign key to profiles
- `text`: Generated text
- `language`: Language code
- `audio_path`: Path to audio file
- `duration`: Duration in seconds
- `seed`: Random seed (optional)
- `created_at`: Creation timestamp
### projects
- `id`: UUID primary key
- `name`: Project name
- `data`: JSON data
- `created_at`: Creation timestamp
- `updated_at`: Last update timestamp
## File Structure
### Request flow
```
data/
├── profiles/
│ └── {profile_id}/
│ ├── {sample_id}.wav
│ └── ...
├── generations/
│ └── {generation_id}.wav
├── cache/
│ └── {hash}.prompt
├── projects/
│ └── {project_id}.json
└── voicebox.db
HTTP request
-> routes/ (validate input, parse params)
-> services/ (business logic, database queries, orchestration)
-> backends/ (TTS/STT inference)
-> utils/ (audio processing, effects, caching)
```
## Setup
Route handlers are intentionally thin. They validate input, delegate to a service function, and format the response. All business logic lives in `services/`.
### 1. Install Dependencies
```bash
pip install -r requirements.txt
```
**Note:** On Apple Silicon, also install MLX dependencies for faster inference:
```bash
pip install -r requirements-mlx.txt
```
### 2. Download Models (Automatic)
The Qwen3-TTS models are automatically downloaded from HuggingFace Hub on first use, similar to how Whisper models work.
**No manual download required!** The models will be cached locally after the first download.
Available models:
- **1.7B** (recommended): `Qwen/Qwen3-TTS-12Hz-1.7B-Base` (~4GB)
- **0.6B** (faster): `Qwen/Qwen3-TTS-12Hz-0.6B-Base` (~2GB)
**Note:** The first generation will take longer as the model downloads. Subsequent generations will use the cached model.
#### Manual Download (Optional)
If you prefer to download models manually or have limited internet during runtime:
```bash
# Install huggingface-cli
pip install huggingface_hub
# Download 1.7B model
huggingface-cli download Qwen/Qwen3-TTS-12Hz-1.7B-Base
# Or use Python
python -c "from huggingface_hub import snapshot_download; snapshot_download('Qwen/Qwen3-TTS-12Hz-1.7B-Base')"
```
Models are cached in `~/.cache/huggingface/hub/` by default.
### 4. Run Server
```bash
# Development (local only)
python -m backend.main
# Production (allow remote access)
python -m backend.main --host 0.0.0.0 --port 8000
```
## Usage Examples
The desktop app, web client, and current development workflow use `http://localhost:17493` by default.
If you launch the backend manually with a different host or port, substitute that address in the examples below.
### Creating a Voice Profile
```bash
# 1. Create profile
curl -X POST http://localhost:17493/profiles \
-H "Content-Type: application/json" \
-d '{"name": "My Voice", "language": "en"}'
# Response: {"id": "abc-123", ...}
# 2. Add sample
curl -X POST http://localhost:17493/profiles/abc-123/samples \
-F "[email protected]" \
-F "reference_text=This is my voice sample"
```
### Generating Speech
### Key modules
**services/generation.py** -- Single `run_generation()` function that handles all three generation modes (generate, retry, regenerate). Manages model loading, voice prompt creation, chunked inference, normalization, effects, and version persistence.
**services/task_queue.py** -- Serial generation queue. Ensures only one GPU inference runs at a time. Background tasks are tracked to prevent garbage collection.
**backends/__init__.py** -- Protocol definitions (`TTSBackend`, `STTBackend`), model config registry, and factory functions. Adding a new engine means implementing the protocol and registering a config entry.
**backends/base.py** -- Shared utilities used across all engine implementations: HuggingFace cache checks, device detection, voice prompt combination, progress tracking.
**database/** -- SQLAlchemy ORM models with a re-exporting `__init__.py` for backward compatibility. Migrations run automatically on startup.
### Backend selection
The server detects the best inference backend at startup:
| Platform | Backend | Acceleration |
|----------|---------|-------------|
| macOS (Apple Silicon) | MLX | Metal / Neural Engine |
| Windows / Linux (NVIDIA) | PyTorch | CUDA |
| Linux (AMD) | PyTorch | ROCm |
| Intel Arc | PyTorch | IPEX / XPU |
| Windows (any GPU) | PyTorch | DirectML |
| Any | PyTorch | CPU fallback |
Detection is handled by `utils/platform_detect.py`. Both backends implement the same `TTSBackend` protocol, so the API layer is engine-agnostic.
## API
90 endpoints organized by domain. Full interactive documentation available at `http://localhost:17493/docs` when the server is running.
| Domain | Prefix | Description |
|--------|--------|-------------|
| Health | `/`, `/health` | Server status, GPU info, filesystem checks |
| Profiles | `/profiles` | Voice profile CRUD, samples, avatars, import/export |
| Channels | `/channels` | Audio channel management and voice assignment |
| Generation | `/generate` | TTS generation, retry, regenerate, status SSE |
| History | `/history` | Generation history, search, favorites, export |
| Transcription | `/transcribe` | Whisper-based audio-to-text |
| Stories | `/stories` | Multi-track timeline editor, audio export |
| Effects | `/effects` | Effect presets, preview, version management |
| Audio | `/audio`, `/samples` | Audio file serving |
| Models | `/models` | Load, unload, download, migrate, status |
| Tasks | `/tasks`, `/cache` | Active task tracking, cache management |
| CUDA | `/backend/cuda-*` | CUDA binary download and management |
### Quick examples
```bash
# Generate speech
curl -X POST http://localhost:17493/generate \
-H "Content-Type: application/json" \
-d '{
"profile_id": "abc-123",
"text": "Hello, this is a test.",
"language": "en",
"seed": 42
}'
-d '{"text": "Hello world", "profile_id": "...", "language": "en"}'
# Response: {"id": "gen-456", "audio_path": "/path/to/audio.wav", ...}
# List profiles
curl http://localhost:17493/profiles
# Download audio
curl http://localhost:17493/audio/gen-456 -o output.wav
# Stream generation status (SSE)
curl http://localhost:17493/generate/{id}/status
```
### Transcribing Audio
## Data directory
```
{data_dir}/
voicebox.db # SQLite database
profiles/{id}/ # Voice samples per profile
generations/ # Generated audio files
cache/ # Voice prompt cache (memory + disk)
backends/ # Downloaded CUDA binary (if applicable)
```
Default location is the OS-specific app data directory. Override with `--data-dir` or the `VOICEBOX_DATA_DIR` environment variable.
## Code quality
Linting and formatting are enforced by [ruff](https://docs.astral.sh/ruff/), configured in `pyproject.toml`. See `STYLE_GUIDE.md` for conventions.
```bash
curl -X POST http://localhost:17493/transcribe \
-F "[email protected]" \
-F "language=en"
# Response: {"text": "Transcribed text", "duration": 5.5}
just check-python # lint + format check
just fix-python # auto-fix lint issues + reformat
just test # run pytest
```
## Advanced Features
## Dependencies
### Multi-Sample Profiles
Add multiple samples to a profile for better quality:
```bash
# Add first sample
curl -X POST http://localhost:17493/profiles/abc-123/samples \
-F "[email protected]" \
-F "reference_text=First sample"
# Add second sample
curl -X POST http://localhost:17493/profiles/abc-123/samples \
-F "[email protected]" \
-F "reference_text=Second sample"
# Generation will automatically combine all samples
```
### Voice Prompt Caching
Voice prompts are automatically cached for faster generation:
- First generation: ~5-10 seconds (creates prompt)
- Subsequent generations: ~1-2 seconds (uses cached prompt)
Cache is stored in `data/cache/` and persists across server restarts.
### VRAM Management
Models are lazy-loaded and can be manually unloaded:
```bash
# Unload TTS model
curl -X POST http://localhost:17493/models/unload
# Load specific model size
curl -X POST "http://localhost:17493/models/load?model_size=0.6B"
```
## Error Handling
All endpoints return proper HTTP status codes:
- `200 OK`: Success
- `400 Bad Request`: Invalid input
- `404 Not Found`: Resource not found
- `500 Internal Server Error`: Server error
Error responses include details:
```json
{
"detail": "Profile not found"
}
```
## Performance Tips
1. **Use multi-sample profiles** - Better quality than single sample
2. **Let caching work** - Voice prompts are cached automatically
3. **Use 0.6B model on CPU** - Faster than 1.7B with acceptable quality
4. **Use 1.7B model on GPU** - Best quality, still fast
5. **Unload Whisper after transcription** - Frees VRAM for TTS
## TODO
- [ ] WebSocket support for generation progress
- [ ] Batch generation endpoint
- [ ] Voice design (text-to-voice)
- [ ] Authentication & rate limiting
## License
See main project LICENSE.
Runtime dependencies are in `requirements.txt`. macOS-only MLX dependencies are in `requirements-mlx.txt`. Dev tools (ruff, pytest) are installed automatically by `just setup-python`.
backend/REFACTOR_PLAN.md
-177
View File
@@ -1,177 +0,0 @@
# Backend Refactor Plan
## Current State
`main.py` is still a ~2,800-line god file with 72 routes, 3x duplicated generation orchestration, fake async CRUD modules, and scattered constants. The backend dedup is done — adding new engines is now trivial.
---
## Phase 1: Dead Code & Low-Hanging Fruit ✓
Deleted `studio.py`, `migrate_add_instruct.py`, `utils/validation.py`. Removed duplicate `_profile_to_response`, duplicate `import asyncio`, pointless wrapper functions. Consolidated `LANGUAGE_CODE_TO_NAME` and `WHISPER_HF_REPOS` into `backends/__init__.py`. Updated README.
---
## Phase 2: Backend Deduplication ✓
Created `backends/base.py` with shared utilities:
- `is_model_cached()` — parameterized HF cache check (replaced 7 copies)
- `get_torch_device()` — parameterized device detection (replaced 5 copies)
- `combine_voice_prompts()` — load + normalize + concatenate (replaced 5 copies)
- `model_load_progress()` — context manager for progress tracking lifecycle (replaced 7 copies)
- `patch_chatterbox_f32()` — shared dtype monkey-patches (replaced 2 copies)
Net result: -1,078 lines across the backend.
---
## Phase 3: Generation Service ✓
Extracted the three near-identical generation closures (`_run_generation`, `_run_retry`, `_run_regenerate`) and the background queue machinery from `main.py` into a new `services/` layer:
- `services/task_queue.py` — `create_background_task()`, `enqueue_generation()`, `init_queue()`, and the serial `_generation_worker`. Replaces the module-level globals and helpers that were in `main.py:63-92`.
- `services/generation.py` — single `run_generation()` function with a `mode` parameter (`"generate"`, `"retry"`, `"regenerate"`). Mode-specific persistence is handled by three small sync helpers (`_save_generate`, `_save_retry`, `_save_regenerate`). The shared pipeline (model loading, voice prompt creation, chunked inference, normalization, error handling, task manager lifecycle) is written once.
Route handlers in `main.py` are now thin: validate input, create/update DB row, resolve effects chain, then `enqueue_generation(run_generation(...))`.
Net result: ~240 lines of duplicated closure code replaced by a single 230-line service module + 50-line queue module.
---
## Phase 4: Route Extraction
Split `main.py` (72 routes) into domain-specific routers. After Phase 3, the route handlers should be thin — just validation, delegation, and response formatting.
### Target structure
```
backend/
app.py # FastAPI app creation, middleware, startup/shutdown
routes/
__init__.py
health.py # GET /, /health, /health/filesystem, /shutdown, /watchdog/disable (5 routes)
profiles.py # All /profiles/* routes (17 routes)
channels.py # All /channels/* routes (7 routes)
generations.py # /generate, /generate/stream, /generate/*/retry, regenerate, status (5 routes)
history.py # All /history/* routes (8 routes)
stories.py # All /stories/* routes (15 routes)
effects.py # All /effects/* routes + /generations/*/versions/* (11 routes)
audio.py # /audio/*, /samples/* (2 routes)
models.py # All /models/* routes (11 routes)
tasks.py # /tasks/*, /cache/* (3 routes)
cuda.py # /backend/cuda-* (4 routes)
services/
generation.py # TTS orchestration (from Phase 3)
model_status.py # HF cache inspection logic (currently inline at main.py:2251-2431)
```
`main.py` becomes a thin entry point that imports the app from `app.py` and runs uvicorn (preserving backward compat for `python -m backend.main`).
### Model status extraction
The `get_model_status` endpoint (`main.py:2251-2431`) is 180 lines of HuggingFace cache inspection that duplicates logic from `_is_model_cached` in the backends. Extract to `services/model_status.py` and reuse the shared `is_model_cached` from Phase 2 where possible.
---
## Phase 5: Database Cleanup
### Adopt Alembic
Replace the hand-rolled `_run_migrations()` (200 lines of manual ALTER TABLE + column existence checks) with Alembic.
**Why:**
- Current approach has no migration tracking — checks column existence on every startup
- Can't express complex migrations (data transforms, renames) safely
- No rollback path
- Already at 12 migration blocks and growing
**Migration steps:**
1. `pip install alembic` and add to `requirements.txt`
2. Run `alembic init alembic` to scaffold the config
3. Point `alembic/env.py` at the existing SQLAlchemy `Base.metadata` and engine
4. Create a baseline migration stamped as the current schema — this tells Alembic "the DB already has all this, don't recreate it":
```bash
alembic revision --autogenerate -m "baseline"
# Then stamp existing DBs so they skip the baseline:
alembic stamp head
```
5. Replace `_run_migrations()` in `init_db()` with `alembic.command.upgrade(config, "head")`
6. Move `_backfill_generation_versions` and `_seed_builtin_presets` into a post-migration hook or a dedicated seed step in `init_db()`
7. Delete the 200 lines of manual migration code
**Going forward**, new schema changes become:
```bash
# Auto-generate from model diff
alembic revision --autogenerate -m "add_whatever_column"
# Review the generated file, then it runs on next startup
```
**Target structure:**
```
backend/
alembic/
versions/
001_baseline.py
env.py
alembic.ini
database/
__init__.py # re-exports for backward compat
models.py # ORM model definitions (11 models, ~140 lines)
session.py # engine creation, init_db(), get_db()
seed.py # _backfill_generation_versions + _seed_builtin_presets
```
### Fix async-over-sync CRUD modules
`channels.py`, `history.py`, `stories.py`, `effects.py`, `versions.py`, `profiles.py` all declare `async def` but never `await`. They run synchronous SQLAlchemy queries directly, blocking the event loop. Two options:
- **Option A**: Drop `async` keyword, wrap calls in `asyncio.to_thread()` at the route layer
- **Option B**: Switch to async SQLAlchemy (`create_async_engine` + `AsyncSession`)
Option A is simpler and non-disruptive. Option B is cleaner long-term but touches every query.
---
## Phase 6: Polish
- Consolidate hardcoded constants (`24000` sample rate, `100MB`/`50MB` max file sizes, `HSA_OVERRIDE_GFX_VERSION`, CORS origins) into `config.py` or a `constants.py`
- Fix `hf_offline_patch.py` side-effect-on-import (runs patching twice — once on import, once explicitly in `mlx_backend.py`)
- Standardize error handling across routes (currently three different patterns)
- Rename `effects.py` (preset CRUD) to avoid confusion with `utils/effects.py` (DSP engine) — either rename to `effect_presets.py` or fold into routes
- Clean up test suite — the 4 manual integration scripts in `tests/` should either be converted to pytest or moved to a `scripts/` dir
---
## Phase 7: Style Guide & Tooling ✓
Added a Python style guide (`backend/STYLE_GUIDE.md`) and automated linting/formatting with ruff. Removed the redundant Makefile — the justfile is now the single task runner.
### Style guide
Codifies conventions for the refactor: Google-style docstrings, native 3.12 type syntax (`list[str]`, `X | None` — no `from __future__` or `typing.List`), `logging` module instead of `print()`, two-layer error handling (domain exceptions + route-layer HTTPException), import grouping (stdlib / third-party / local with isort enforcement), 120-char line length.
### Ruff config (`pyproject.toml`)
Added project-root `pyproject.toml` with ruff linter + formatter config. Rule sets: `F`, `E`, `W`, `I` (isort), `N` (naming), `UP` (pyupgrade to 3.12), `B` (bugbear), `SIM`, `RET`, `T20` (print detection), `PT` (pytest style), `RUF`. `T201` (print) is ignored during migration — remove once logging conversion is done.
Initial scan: 1,103 lint violations (879 auto-fixable), 38 files needing reformatting. Mostly whitespace (W293), type annotation modernization (UP045/UP006), and import sorting (I001). To be fixed file-by-file as files are touched, not in a big-bang pass.
### Justfile updates
- `just check` now runs both JS (Biome) and Python (ruff) checks
- Added `just check-python`, `just lint-python`, `just format-python`, `just fix-python`, `just test`
- `just setup-python` installs `ruff`, `pytest`, `pytest-asyncio` as dev tools
- Deleted `Makefile` and updated all references in `CHANGELOG.md`, `PATCH_NOTES.md`, `docs/plans/ADDING_TTS_ENGINES.md`
---
## Notes
- Each phase is independently shippable and testable
- Phase 1 is zero-risk deletion
- Phase 2 is self-contained within `backends/`
- Phase 3 sets up the extraction pattern needed for Phase 4
- Phase 4 is the largest change but should be mostly mechanical after Phase 3
- Phase 5 can run in parallel with Phase 4 since it touches different files
backend/STYLE_GUIDE.md
+15 -71
View File
@@ -275,7 +275,7 @@ async def create_profile(data: VoiceProfileCreate, db: Session = Depends(get_db)
### Rules for the refactor
1. **Don't declare `async def` unless the function awaits something.** The current CRUD modules break this -- they will be fixed per REFACTOR_PLAN Phase 5.
1. **Don't declare `async def` unless the function awaits something.** Several service modules still declare `async def` without awaiting -- these should be migrated to sync functions with `asyncio.to_thread()` at the route layer, or to real async SQLAlchemy.
2. **CPU-bound work** (audio processing, numpy operations) goes through `asyncio.to_thread()`:
```python
audio, sr = await asyncio.to_thread(load_audio, source_path)
@@ -367,68 +367,28 @@ Framework: **pytest** with `pytest-asyncio`.
---
## Project Layout (Post-Refactor Target)
From REFACTOR_PLAN.md Phase 4:
## Project Layout
```
backend/
app.py # FastAPI app, middleware, startup/shutdown
main.py # Entry point (imports app, runs uvicorn)
config.py # Data dirs, shared constants
errors.py # Custom exception classes
routes/
__init__.py
health.py
profiles.py
channels.py
generations.py
history.py
stories.py
effects.py
audio.py
models.py
tasks.py
cuda.py
services/
generation.py
task_queue.py
model_status.py
database/
__init__.py
models.py
session.py
seed.py
backends/
__init__.py
base.py
pytorch_backend.py
mlx_backend.py
luxtts_backend.py
chatterbox_backend.py
chatterbox_turbo_backend.py
utils/
audio.py
effects.py
progress.py
tasks.py
hf_progress.py
hf_offline_patch.py
cache.py
images.py
chunked_tts.py
tests/
conftest.py
test_cors.py
test_profiles.py
...
app.py # FastAPI app factory, CORS, lifecycle events
main.py # Entry point (imports app, runs uvicorn)
config.py # Data directory paths
models.py # Pydantic request/response schemas
server.py # Tauri sidecar launcher, parent-pid watchdog
routes/ # Thin HTTP handlers (validation, delegation, response formatting)
services/ # Business logic, CRUD, orchestration
backends/ # TTS/STT engine implementations
database/ # ORM models, session management, migrations, seeds
utils/ # Shared utilities (audio, effects, caching, progress)
tests/ # pytest suite
```
---
## Ruff Adoption
The `pyproject.toml` at the project root configures ruff for linting and formatting. Run:
`pyproject.toml` configures ruff for linting and formatting. Run:
```bash
# Lint (check)
@@ -441,20 +401,4 @@ ruff check backend/ --fix
ruff format backend/
```
During the refactor, introduce ruff fixes file-by-file as you touch them. Don't run `--fix` across the entire codebase in one shot -- that creates unreviewable diffs.
---
## Summary of Changes from Current State
| Area | Before | After |
|------|--------|-------|
| Line length | Uncontrolled (up to 160) | 120, enforced by ruff |
| Import order | Ad-hoc | isort-grouped, enforced |
| Type syntax | Mixed `List`/`list`, sporadic `__future__` | Native `list[]`, `X \| None`, no `__future__` |
| Logging | ~80% `print()` | `logging` module everywhere |
| Error handling | 3 inconsistent patterns | Domain exceptions + route-layer HTTPException |
| Async CRUD | Fake `async def` | Sync functions (Phase 5) or real async |
| Linting | None | Ruff with auto-fix |
| Formatting | None | Ruff format (Black-compatible) |
| Tests | Mix of pytest + manual scripts | pytest throughout, shared conftest |
Introduce ruff fixes file-by-file as you touch them. Don't run `--fix` across the entire codebase in one shot -- that creates unreviewable diffs.