mirror of
https://github.com/jamiepine/voicebox.git
synced 2026-09-19 14:50:38 -07:00
docs: audit mdx docs against multi-engine backend (#484)
* docs: audit mdx docs against multi-engine backend and refresh stale content Rewrote developer-facing docs that predated the TTSBackend Protocol / ModelConfig registry refactor (architecture, tts-generation, model-management, transcription). Updated user-facing docs to reflect all seven shipped engines (Qwen, Qwen CustomVoice, LuxTTS, Chatterbox, Chatterbox Turbo, TADA, Kokoro) instead of the outdated "5 engines" claim. Also fixes: - Stale app identifier (com.voicebox.app → sh.voicebox.app) - CUDA backend update flow (now two-archive split, not N-way chunks) - Whisper model list (removed tiny, added turbo) - Broken /development/ and /guides/ route links - Stale just commands and install steps (missing --no-deps chatterbox/tada) - Removed ASCII art diagrams from README and stories.mdx - History Generation schema sync with DB model Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]> * docs: add DeepWiki badge to README Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]> * docs: address PR review feedback - architecture.mdx: fix backends/ file list (remove nonexistent qwen_backend.py, rename tada_backend.py → hume_backend.py) - model-management.mdx: Kokoro language count 9 → 8 (matches ModelConfig) - model-management.mdx: ProgressManager path services/ → utils/ - tts-generation.mdx: ModelConfig example uses field(default_factory=...) — mutable default would raise at runtime - tts-generation.mdx: "1080p samples" → "on CUDA" (1080p is video, not audio) - PROJECT_STATUS.md: replace ASCII architecture diagram with prose (matches no-ASCII-art rule) Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]> * fix(app): guard against undefined engine in FloatingGenerateBox preset check form.getValues('engine') returns string | undefined; Set<string>.has() rejects undefined under strict mode. Added a truthy guard before the preset lookup. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]> --------- Co-authored-by: Claude Opus 4.7 (1M context) <[email protected]>
This commit is contained in:
Jamie Pine
co-authored by
Claude Opus 4.7
Claude Opus 4.7
parent
da6070155e
commit
ae91aa9a88
@@ -1,428 +0,0 @@
|
||||
# Voicebox API Refactor Plan
|
||||
|
||||
Date: 2026-03-19
|
||||
Status: Proposed
|
||||
Scope: Backend HTTP API structure, schemas, docs, and compatibility strategy
|
||||
|
||||
## Goals
|
||||
|
||||
- Make the API easier to understand and automate against.
|
||||
- Improve endpoint consistency without breaking the desktop app or existing local integrations.
|
||||
- Align generated docs and checked-in OpenAPI artifacts with the actual backend.
|
||||
- Separate app-facing resources from internal or operational actions.
|
||||
- Create a migration path toward a cleaner `v2` resource model while preserving `v1` routes during transition.
|
||||
|
||||
## Non-Goals
|
||||
|
||||
- Rewriting backend business logic or generation internals.
|
||||
- Introducing authentication for all deployment modes in the first pass.
|
||||
- Changing storage models or database schema unless required for API correctness.
|
||||
- Removing current routes immediately.
|
||||
|
||||
## Current Pain Points
|
||||
|
||||
- Mixed endpoint styles: resource-oriented (`/profiles`) and command-oriented (`/generate`, `/tasks/clear`) coexist.
|
||||
- Related generation resources are split across multiple namespaces: `/generate`, `/history`, `/audio`, `/effects`, and `/generations/.../versions`.
|
||||
- Response payloads vary widely: typed models, raw dicts with `message`, booleans, and `HTTPException(detail=...)` payloads.
|
||||
- Some async flows use exception-shaped `202` responses instead of first-class task contracts.
|
||||
- Checked-in OpenAPI output can drift from actual backend models.
|
||||
- Operational endpoints such as `/shutdown` are exposed in the same surface as user workflows.
|
||||
|
||||
## Guiding Principles
|
||||
|
||||
1. Prefer additive changes before destructive changes.
|
||||
2. Keep `v1` behavior working until the app and docs fully migrate.
|
||||
3. Add compatibility shims close to the routing layer, not deep in services.
|
||||
4. Treat OpenAPI as a release artifact that must be kept in sync.
|
||||
5. Standardize public contracts before renaming everything.
|
||||
|
||||
## Target API Shape
|
||||
|
||||
This is the intended end state, not the immediate first milestone.
|
||||
|
||||
### Core Resources
|
||||
|
||||
- `/profiles`
|
||||
- `/profiles/{profile_id}/samples`
|
||||
- `/profiles/{profile_id}/avatar`
|
||||
- `/profiles/{profile_id}/effects`
|
||||
- `/generations`
|
||||
- `/generations/{generation_id}`
|
||||
- `/generations/{generation_id}/status`
|
||||
- `/generations/{generation_id}/audio`
|
||||
- `/generations/{generation_id}/versions`
|
||||
- `/generations/{generation_id}/versions/{version_id}`
|
||||
- `/generations/{generation_id}/versions/{version_id}/audio`
|
||||
- `/stories`
|
||||
- `/stories/{story_id}/items`
|
||||
- `/effects/presets`
|
||||
- `/models`
|
||||
- `/models/{model_name}`
|
||||
- `/tasks`
|
||||
|
||||
### Operational or Internal Endpoints
|
||||
|
||||
Move under an explicit namespace and disable where appropriate:
|
||||
|
||||
- `/admin/shutdown`
|
||||
- `/admin/watchdog/disable`
|
||||
- `/admin/cache/clear`
|
||||
- `/admin/tasks/clear`
|
||||
|
||||
### Response Contract Direction
|
||||
|
||||
- Resource reads and writes return typed resource models.
|
||||
- Delete and action endpoints return small typed action result models.
|
||||
- Errors use a consistent structure.
|
||||
- Async actions return explicit task metadata instead of overloading `detail`.
|
||||
|
||||
## Migration Strategy Overview
|
||||
|
||||
The refactor is split into six phases. Phases 1-3 are the highest impact and safest to ship first.
|
||||
|
||||
| Phase | Focus | Est. Duration | Risk | Backward Compatibility |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| 1 | Documentation and contract correctness | 2-3 days | Low | Full |
|
||||
| 2 | Response and error consistency | 3-5 days | Low-Medium | Full |
|
||||
| 3 | Router structure and internal organization | 3-4 days | Low | Full |
|
||||
| 4 | Additive `v2` resource endpoints | 1-2 weeks | Medium | Full |
|
||||
| 5 | Client migration and deprecation rollout | 1 week | Medium | Full during rollout |
|
||||
| 6 | Cleanup and optional removals | 1-2 releases | Medium-High | Partial after notice |
|
||||
|
||||
## Phase 1: Fix Contract Drift First
|
||||
|
||||
Priority: Highest
|
||||
Outcome: The documented API matches the running backend.
|
||||
|
||||
### Problems Addressed
|
||||
|
||||
- `docs/openapi.json` can become stale.
|
||||
- Generated API reference pages may describe outdated request bodies.
|
||||
- App metadata still frames the backend too narrowly.
|
||||
|
||||
### Implementation Steps
|
||||
|
||||
1. Update FastAPI app metadata in `backend/app.py`.
|
||||
- Replace the old Qwen-specific description with a multi-engine Voicebox API description.
|
||||
- Add tags metadata for major domains if desired.
|
||||
2. Regenerate OpenAPI from the running app using the existing docs script flow.
|
||||
3. Compare `backend/models.py` to the checked-in schema.
|
||||
- Verify `GenerationRequest`, effects endpoints, stories endpoints, and model endpoints.
|
||||
4. Regenerate or refresh API reference pages under `docs/content/docs/api-reference/`.
|
||||
5. Add a CI check that fails if `docs/openapi.json` is out of date.
|
||||
6. Add a short maintainer note describing when schema regeneration is required.
|
||||
|
||||
### Backward Compatibility
|
||||
|
||||
- No route changes.
|
||||
- No payload changes.
|
||||
- Safe to release immediately.
|
||||
|
||||
### Success Criteria
|
||||
|
||||
- `docs/openapi.json` matches the live app.
|
||||
- Generated docs include all currently supported generate parameters.
|
||||
- No frontend code changes required.
|
||||
|
||||
## Phase 2: Standardize Responses and Errors
|
||||
|
||||
Priority: High
|
||||
Outcome: Clients can handle responses predictably.
|
||||
|
||||
### Problems Addressed
|
||||
|
||||
- Delete endpoints return ad hoc message dicts.
|
||||
- Toggle endpoints return special one-off payloads.
|
||||
- `202` async responses are encoded as `HTTPException(detail=...)` in some places.
|
||||
|
||||
### Implementation Steps
|
||||
|
||||
1. Add shared response models in `backend/models.py`.
|
||||
- `ActionResult`
|
||||
- `DeleteResult`
|
||||
- `ToggleFavoriteResponse`
|
||||
- `AcceptedTaskResponse`
|
||||
- `ApiError`
|
||||
2. Convert routes that currently return raw dicts to explicit `response_model`s.
|
||||
- `DELETE /profiles/{profile_id}`
|
||||
- `DELETE /history/{generation_id}`
|
||||
- `DELETE /stories/{story_id}`
|
||||
- `POST /tasks/clear`
|
||||
- `POST /cache/clear`
|
||||
- similar endpoints across routes
|
||||
3. Replace exception-shaped `202` responses in `transcription.py` with an explicit accepted response body.
|
||||
- Return `JSONResponse(status_code=202, content=...)` or typed FastAPI response model.
|
||||
4. Add a global exception handler for known API errors if helpful.
|
||||
- Normalize `ValueError` to `400` with a consistent error body.
|
||||
- Preserve FastAPI validation errors for now, or wrap them in a consistent top-level shape in a later pass.
|
||||
5. Document the stable error contract in the docs.
|
||||
|
||||
### Migration Strategy
|
||||
|
||||
- Keep field names inside successful payloads compatible where possible.
|
||||
- For existing dict responses, preserve the current keys while introducing typed models with the same shape.
|
||||
- For `202` flows, support both old and new client handling for one release if needed.
|
||||
|
||||
### Timeline Estimate
|
||||
|
||||
- 3-5 engineering days including tests and docs refresh.
|
||||
|
||||
### Success Criteria
|
||||
|
||||
- All mutation endpoints declare response models.
|
||||
- Clients can programmatically distinguish success, accepted, and error cases without special casing `detail` payloads.
|
||||
|
||||
## Phase 3: Normalize Router Structure Internally
|
||||
|
||||
Priority: High
|
||||
Outcome: The backend becomes easier to maintain before public path changes begin.
|
||||
|
||||
### Problems Addressed
|
||||
|
||||
- Route files hardcode full paths and are all mounted at root.
|
||||
- There is no consistent use of router prefixes or tags.
|
||||
- Route grouping in code does not cleanly express the public API shape.
|
||||
|
||||
### Implementation Steps
|
||||
|
||||
1. Add prefixes and tags to routers.
|
||||
- `profiles`: `prefix="/profiles"`
|
||||
- `generations`: `prefix="/generate"` for now or split additive aliases carefully
|
||||
- `history`: `prefix="/history"`
|
||||
- `effects`: `prefix="/effects"`
|
||||
- and so on
|
||||
2. Convert route declarations to relative paths within each router.
|
||||
3. Introduce a small route compatibility layer for routes that are likely to move later.
|
||||
- Example: helper functions that can be mounted under both old and new paths.
|
||||
4. Add explicit route tags so Swagger/OpenAPI groups are coherent.
|
||||
5. Document the intended public ownership of each namespace.
|
||||
|
||||
### Backward Compatibility
|
||||
|
||||
- No public path changes yet if existing paths are preserved through prefixes and aliases.
|
||||
- Mostly internal refactoring.
|
||||
|
||||
### Timeline Estimate
|
||||
|
||||
- 3-4 engineering days.
|
||||
|
||||
### Success Criteria
|
||||
|
||||
- All route modules use prefixes and tags.
|
||||
- Route registration in `backend/routes/__init__.py` becomes simpler.
|
||||
- OpenAPI groups read cleanly by domain.
|
||||
|
||||
## Phase 4: Introduce Additive `v2` Resource Endpoints
|
||||
|
||||
Priority: High
|
||||
Outcome: A cleaner API exists without breaking the current one.
|
||||
|
||||
### Problems Addressed
|
||||
|
||||
- Generation-related resources are fragmented.
|
||||
- Sample and audio endpoints are not consistently modeled as resources.
|
||||
- Command-style naming makes the API harder to reason about.
|
||||
|
||||
### New Endpoints to Add
|
||||
|
||||
These should be introduced alongside current endpoints, not as replacements.
|
||||
|
||||
- `POST /generations` -> alias for current `/generate`
|
||||
- `GET /generations` -> alias for current `/history`
|
||||
- `GET /generations/{id}` -> alias for current `/history/{id}`
|
||||
- `POST /generations/{id}/retry` -> alias for current `/generate/{id}/retry`
|
||||
- `POST /generations/{id}/regenerate` -> alias for current `/generate/{id}/regenerate`
|
||||
- `GET /generations/{id}/status` -> alias for current `/generate/{id}/status`
|
||||
- `POST /generations/stream` -> alias for current `/generate/stream`
|
||||
- `GET /generations/{id}/audio` -> alias for current `/audio/{generation_id}`
|
||||
- `GET /generations/{id}/export` -> alias for current `/history/{generation_id}/export`
|
||||
- `GET /generations/{id}/export-audio` -> alias for current `/history/{generation_id}/export-audio`
|
||||
- `GET /profiles/{profile_id}/samples/{sample_id}` or `GET /samples/{sample_id}` as a consciously chosen model
|
||||
- `PUT /profiles/{profile_id}/samples/{sample_id}` -> alias for current sample update route
|
||||
- `DELETE /profiles/{profile_id}/samples/{sample_id}` -> alias for current sample delete route
|
||||
|
||||
### Implementation Steps
|
||||
|
||||
1. Create new handler entry points that call the existing service functions.
|
||||
2. Keep old handlers in place, but mark them deprecated in OpenAPI.
|
||||
3. Add `summary` and `description` text clarifying preferred routes.
|
||||
4. Update frontend and docs examples to use new endpoints first.
|
||||
5. Add tests proving both old and new paths return equivalent responses.
|
||||
|
||||
### Migration Strategy
|
||||
|
||||
- Old paths remain functional for at least one stable release cycle.
|
||||
- New docs and client examples use `v2-style` resource routes immediately.
|
||||
- Include deprecation headers where feasible, for example:
|
||||
- `Deprecation: true`
|
||||
- `Sunset: <date>`
|
||||
- `Link: <new-doc-url>; rel="successor-version"`
|
||||
|
||||
### Timeline Estimate
|
||||
|
||||
- 1-2 weeks depending on test coverage and frontend updates.
|
||||
|
||||
### Success Criteria
|
||||
|
||||
- All major generation workflows are accessible through resource-oriented routes.
|
||||
- Old routes still work unchanged.
|
||||
|
||||
## Phase 5: Migrate First-Party Clients and Publish Deprecations
|
||||
|
||||
Priority: Medium
|
||||
Outcome: Voicebox itself stops depending on legacy paths.
|
||||
|
||||
### Problems Addressed
|
||||
|
||||
- The desktop app and docs may continue to reinforce old route shapes.
|
||||
- Third-party consumers need a visible migration path.
|
||||
|
||||
### Implementation Steps
|
||||
|
||||
1. Update `app/src/lib/api/client.ts` to use the new preferred endpoints.
|
||||
2. Regenerate or refresh any generated API clients.
|
||||
3. Update docs examples, tutorials, and code snippets to use preferred routes only.
|
||||
4. Add a changelog entry describing the migration path.
|
||||
5. Add runtime deprecation logging for legacy route usage in development mode.
|
||||
6. If feasible, expose a small `/health` or `/meta` field showing API version and deprecation window.
|
||||
|
||||
### Migration Strategy
|
||||
|
||||
- Keep old endpoints available but clearly documented as legacy.
|
||||
- Publish a mapping table from old route to new route.
|
||||
- Do not change request or response payloads during the same phase unless necessary.
|
||||
|
||||
### Timeline Estimate
|
||||
|
||||
- About 1 week including docs and app verification.
|
||||
|
||||
### Success Criteria
|
||||
|
||||
- First-party app no longer depends on legacy route names.
|
||||
- Docs do not advertise deprecated paths as the primary interface.
|
||||
|
||||
## Phase 6: Cleanup, Namespace Hardening, and Optional Breaking Changes
|
||||
|
||||
Priority: Medium
|
||||
Outcome: The API surface is cleaner and safer for remote or Docker use.
|
||||
|
||||
### Problems Addressed
|
||||
|
||||
- Internal/admin endpoints are mixed into the public API.
|
||||
- Legacy aliases increase maintenance cost forever if never retired.
|
||||
|
||||
### Implementation Steps
|
||||
|
||||
1. Move operational endpoints under `/admin` or `/internal`.
|
||||
- `/shutdown`
|
||||
- `/watchdog/disable`
|
||||
- `/tasks/clear`
|
||||
- `/cache/clear`
|
||||
2. Gate these endpoints behind configuration for non-local deployments.
|
||||
- Example: `VOICEBOX_ENABLE_ADMIN_API=true`
|
||||
3. Decide whether to remove or keep legacy aliases.
|
||||
- If removing, do so only after a published deprecation window.
|
||||
4. Remove deprecated docs pages and old examples.
|
||||
5. Tighten route-level tests to prevent accidental reintroduction of legacy patterns.
|
||||
|
||||
### Migration Strategy
|
||||
|
||||
- For desktop-only local use, aliases may remain indefinitely if removal cost outweighs benefit.
|
||||
- For published remote API guidance, hide admin endpoints from default docs even if they still exist.
|
||||
|
||||
### Timeline Estimate
|
||||
|
||||
- 1-2 releases after the additive migration is complete.
|
||||
|
||||
### Success Criteria
|
||||
|
||||
- Public docs expose a coherent resource API.
|
||||
- Operational endpoints are clearly separate or disabled in remote contexts.
|
||||
|
||||
## Cross-Cutting Work Items
|
||||
|
||||
These should happen throughout the migration, not only in a single phase.
|
||||
|
||||
### Testing
|
||||
|
||||
- Add route equivalence tests for old and new paths.
|
||||
- Add schema snapshot tests for OpenAPI generation.
|
||||
- Add response-shape tests for common mutations and async workflows.
|
||||
- Add contract tests for `202 Accepted` flows.
|
||||
|
||||
### Documentation
|
||||
|
||||
- Maintain an old-to-new endpoint mapping table.
|
||||
- Add per-endpoint examples for create profile, generate, apply effects, transcribe, and stories operations.
|
||||
- Explicitly document which endpoints are app-facing vs admin-facing.
|
||||
|
||||
### Observability
|
||||
|
||||
- Add warning logs when deprecated endpoints are used.
|
||||
- Track usage counts in development or optional telemetry-free local logs.
|
||||
|
||||
### Release Management
|
||||
|
||||
- Mention API changes in `CHANGELOG.md`.
|
||||
- Ensure docs and app updates ship in the same release as new preferred routes.
|
||||
|
||||
## Recommended Execution Order
|
||||
|
||||
If engineering time is limited, implement in this exact order:
|
||||
|
||||
1. Fix OpenAPI and docs drift.
|
||||
2. Standardize response models and accepted-task responses.
|
||||
3. Add router prefixes and tags internally.
|
||||
4. Add `/generations` aliases and sample path aliases.
|
||||
5. Migrate the first-party app to preferred routes.
|
||||
6. Deprecate or hide legacy/admin routes.
|
||||
|
||||
## Old-to-New Route Mapping
|
||||
|
||||
| Current Route | Preferred Route |
|
||||
| --- | --- |
|
||||
| `POST /generate` | `POST /generations` |
|
||||
| `POST /generate/stream` | `POST /generations/stream` |
|
||||
| `POST /generate/{id}/retry` | `POST /generations/{id}/retry` |
|
||||
| `POST /generate/{id}/regenerate` | `POST /generations/{id}/regenerate` |
|
||||
| `GET /generate/{id}/status` | `GET /generations/{id}/status` |
|
||||
| `GET /history` | `GET /generations` |
|
||||
| `GET /history/{id}` | `GET /generations/{id}` |
|
||||
| `GET /audio/{id}` | `GET /generations/{id}/audio` |
|
||||
| `GET /history/{id}/export` | `GET /generations/{id}/export` |
|
||||
| `GET /history/{id}/export-audio` | `GET /generations/{id}/export-audio` |
|
||||
| `PUT /profiles/samples/{sample_id}` | `PUT /profiles/{profile_id}/samples/{sample_id}` |
|
||||
| `DELETE /profiles/samples/{sample_id}` | `DELETE /profiles/{profile_id}/samples/{sample_id}` |
|
||||
| `POST /tasks/clear` | `POST /admin/tasks/clear` |
|
||||
| `POST /cache/clear` | `POST /admin/cache/clear` |
|
||||
| `POST /shutdown` | `POST /admin/shutdown` |
|
||||
| `POST /watchdog/disable` | `POST /admin/watchdog/disable` |
|
||||
|
||||
## Risks and Mitigations
|
||||
|
||||
### Risk: App regressions during endpoint migration
|
||||
|
||||
- Mitigation: Add new routes before changing client usage.
|
||||
- Mitigation: Keep payloads identical while paths change.
|
||||
|
||||
### Risk: Docs still drift after cleanup
|
||||
|
||||
- Mitigation: Add CI enforcement and a release checklist step.
|
||||
|
||||
### Risk: Third-party local scripts break on removal
|
||||
|
||||
- Mitigation: Prefer indefinite aliases for one-person local workflows unless maintenance becomes painful.
|
||||
|
||||
### Risk: Admin endpoints remain dangerous in remote mode
|
||||
|
||||
- Mitigation: Hide and gate them before promoting remote deployment more broadly.
|
||||
|
||||
## Definition of Done
|
||||
|
||||
The refactor can be considered complete when all of the following are true:
|
||||
|
||||
- OpenAPI, checked-in docs, and backend models match.
|
||||
- The preferred public API is resource-oriented and documented consistently.
|
||||
- The Voicebox app uses preferred routes exclusively.
|
||||
- Legacy routes are either deprecated with a timeline or intentionally retained as compatibility aliases.
|
||||
- Operational endpoints are clearly separated from the public app API.
|
||||
@@ -1,173 +0,0 @@
|
||||
# CUDA Libs as a Bolt-On Addon
|
||||
|
||||
## Problem
|
||||
|
||||
Every time we bump `__version__` (even for a UI tweak or bugfix), the exact-match version check in both `main.rs:222` and `cuda.py:237` invalidates the user's ~2.4GB CUDA binary, forcing a full redownload. The CUDA binary is the entire server rebuilt with NVIDIA libs included -- there's no separation between app logic and the CUDA runtime.
|
||||
|
||||
## Why This Is Hard With `--onefile`
|
||||
|
||||
The core tension is PyInstaller `--onefile` mode (`build_binary.py:39`). In onefile mode, everything -- Python code, all dependencies, torch, the NVIDIA `.dll`/`.so` files -- gets packed into a single self-extracting archive. There's no concept of "swap out one part." The binary IS the server.
|
||||
|
||||
## Options
|
||||
|
||||
### Option A: Switch to `--onedir` for the CUDA Build (Recommended)
|
||||
|
||||
Instead of `--onefile`, build the CUDA variant as a directory (a folder with the exe + all the shared libs alongside it). Then split the distribution into two archives:
|
||||
|
||||
1. **`voicebox-server-cuda` executable + non-NVIDIA deps** (~200-400MB) -- versioned with the app, redownloaded on every app update.
|
||||
2. **`cuda-libs-cu126.tar.gz`** (~2GB) -- the `nvidia.*` packages (cublas, cudnn, cuda_runtime, etc.), versioned independently (e.g., `cuda-libs-cu126-v1`). Only redownloaded when we bump the CUDA toolkit version or torch's CUDA dependency changes.
|
||||
|
||||
#### How it would work at runtime
|
||||
|
||||
- Tauri downloads the server binary archive and extracts it to `{data_dir}/backends/cuda/`
|
||||
- On first CUDA setup (or when cuda-libs version bumps), downloads and extracts the libs archive into the same directory
|
||||
- The CUDA server exe finds the `.dll`/`.so` files next to it (standard PyInstaller onedir behavior)
|
||||
- Version check becomes two checks: server version + cuda-libs version
|
||||
|
||||
#### Independent versioning
|
||||
|
||||
Add a `cuda-libs.json` manifest:
|
||||
|
||||
```json
|
||||
{"version": "cu126-v1", "torch_compat": ">=2.6.0,<2.8.0"}
|
||||
```
|
||||
|
||||
The server checks this on startup. The Tauri side checks it before launching. Only bump `cu126-v1` -> `cu126-v2` when we actually change the CUDA toolkit or torch major version.
|
||||
|
||||
#### Build pipeline changes
|
||||
|
||||
The CI `build-cuda-windows` job would build with `--onedir`, then separate the output into two archives. The CUDA libs archive could be built less frequently (only when torch/CUDA version changes) and stored as a pinned release asset.
|
||||
|
||||
#### Download experience
|
||||
|
||||
- First-time CUDA setup: ~2.4GB total (same as today)
|
||||
- Subsequent app updates: ~200-400MB for the server, CUDA libs stay cached
|
||||
- CUDA toolkit bump: ~2GB for just the libs
|
||||
|
||||
#### Pros
|
||||
|
||||
- PyInstaller `--onedir` natively produces this structure -- NVIDIA DLLs end up as discrete files in the output directory
|
||||
- The separation is natural: PyInstaller puts torch's NVIDIA deps in predictable paths (`nvidia/cublas/lib/`, etc.)
|
||||
- CUDA libs are highly stable -- only rebundle when changing CUDA toolkit version (e.g., cu126 -> cu128) or major torch version
|
||||
- Server updates become ~200-400MB instead of ~2.4GB
|
||||
- No library path hacking needed -- torch finds NVIDIA DLLs because they're in the same directory tree
|
||||
|
||||
#### Cons
|
||||
|
||||
- Onedir means a folder with hundreds of files instead of a single exe -- more complex to manage, extract, and clean up
|
||||
- Need to modify download/assembly logic in `cuda.py` to handle two separate archives
|
||||
- The Tauri side (`main.rs`) needs to point at an exe inside a directory rather than a standalone binary
|
||||
- Users who manually manage the file may find the folder structure confusing
|
||||
|
||||
#### TTS engine compatibility
|
||||
|
||||
No issues. The TTS engines are pure Python + torch. They don't care whether NVIDIA libs are inside the binary or sitting next to it -- torch's dynamic loader finds them either way.
|
||||
|
||||
---
|
||||
|
||||
### Option B: Keep `--onefile` but Externalize CUDA Libs via Library Path
|
||||
|
||||
Keep the server as a single `--onefile` binary (with NVIDIA packages excluded, same as the CPU build). Ship the CUDA libs as a separate download that gets extracted to `{data_dir}/backends/cuda-libs/`. Before launching, set the library search path to include that directory.
|
||||
|
||||
**Important caveat:** The CPU torch wheel (`whl/cpu`) doesn't have CUDA kernels compiled in -- it's a fundamentally different build. So the binary would need to be built with CUDA-compiled torch but with the NVIDIA runtime libraries excluded. The runtime libs (cublas, cudnn, etc.) would be provided externally.
|
||||
|
||||
#### How it would work
|
||||
|
||||
- Build ONE "CUDA-ready" server binary with CUDA-compiled torch but NVIDIA runtime packages excluded
|
||||
- Ship `cuda-libs-cu126-v1.tar.gz` separately (~2GB of `.dll`/`.so` files)
|
||||
- When launching, Tauri sets `PATH` (Windows) or `LD_LIBRARY_PATH` (Linux) to include the cuda-libs directory
|
||||
|
||||
#### Pros
|
||||
|
||||
- Single server binary for both CPU and CUDA users -- simplifies build pipeline enormously
|
||||
- True bolt-on CUDA libs with fully independent versioning
|
||||
- Server updates are always small (~150MB for the onefile binary)
|
||||
|
||||
#### Cons
|
||||
|
||||
- **Fragile on Windows.** PyInstaller `--onefile` extracts to a temp directory at runtime and the internal torch may not find externally-placed NVIDIA libs. DLL resolution on Windows is notoriously unreliable in this scenario.
|
||||
- `os.add_dll_directory()` only affects `LoadLibraryEx` with `LOAD_LIBRARY_SEARCH_USER_DIRS` flag -- not all DLL loads go through this path
|
||||
- PyInstaller's onefile bootloader may configure DLL search paths before Python code runs
|
||||
- Could work on Linux but is fragile on Windows
|
||||
|
||||
---
|
||||
|
||||
### Option C: Hybrid -- `--onefile` Server + Dynamic CUDA Lib Loading at Runtime
|
||||
|
||||
Build the server as `--onefile` with CUDA-compiled torch but with NVIDIA packages excluded. At startup, before torch initializes CUDA, explicitly load the NVIDIA shared libraries using `ctypes.CDLL` or `os.add_dll_directory()`.
|
||||
|
||||
In `server.py`, before any torch imports:
|
||||
|
||||
```python
|
||||
cuda_libs_dir = os.environ.get("VOICEBOX_CUDA_LIBS")
|
||||
if cuda_libs_dir and os.path.isdir(cuda_libs_dir):
|
||||
if sys.platform == "win32":
|
||||
os.add_dll_directory(cuda_libs_dir)
|
||||
os.environ["PATH"] = cuda_libs_dir + os.pathsep + os.environ.get("PATH", "")
|
||||
else:
|
||||
os.environ["LD_LIBRARY_PATH"] = cuda_libs_dir + ":" + os.environ.get("LD_LIBRARY_PATH", "")
|
||||
```
|
||||
|
||||
#### Pros
|
||||
|
||||
- Single server binary, true bolt-on CUDA libs
|
||||
- Clean separation of concerns
|
||||
- Independent versioning
|
||||
|
||||
#### Cons
|
||||
|
||||
- Needs careful testing with each torch version -- CUDA initialization happens deep in C++ extension layer
|
||||
- On Windows, `os.add_dll_directory()` may not cover all DLL load paths
|
||||
- PyInstaller's onefile bootloader may have already configured DLL search paths before Python code runs
|
||||
- Most complex to get right and maintain
|
||||
|
||||
## Recommendation
|
||||
|
||||
**Option A (`--onedir` with split archives)** is the most reliable path:
|
||||
|
||||
1. **It actually works.** `--onedir` puts all files on disk as regular files. Torch finds NVIDIA DLLs because they're in the same directory tree, exactly as they would be in a normal pip install.
|
||||
2. **Natural separation.** PyInstaller's `--onedir` output already separates the NVIDIA `.dll`/`.so` files into `nvidia/` subdirectories. We can split the output directory into "core" and "nvidia-libs" archives after building.
|
||||
3. **Independent versioning is straightforward.** A `cuda-libs.json` manifest controls when redownloads are needed.
|
||||
4. **Build pipeline simplification.** Build CUDA libs archive less frequently, store as a pinned release asset.
|
||||
|
||||
The main cost is managing a directory instead of a single file, but we already have sophisticated download/assembly infrastructure in `cuda.py` with manifests and split parts. Extending that to handle two archives is incremental work.
|
||||
|
||||
## Tauri Compatibility (Validated)
|
||||
|
||||
Tauri handles PyInstaller `--onedir` with no issues. The key insight is that we're **not** using a static sidecar for CUDA -- we're downloading and extracting at runtime (the existing `cuda.py` + `main.rs` flow). For runtime-launched processes, Tauri's `tauri::shell::Command` supports arbitrary directories natively.
|
||||
|
||||
### The critical change in `main.rs`
|
||||
|
||||
The only Tauri-side change needed is adding `.current_dir()` when spawning the CUDA backend:
|
||||
|
||||
```rust
|
||||
let cuda_dir = data_dir.join("backends/cuda");
|
||||
let exe_path = cuda_dir.join("voicebox-server-cuda.exe");
|
||||
|
||||
let mut cmd = app.shell().command(exe_path.to_str().unwrap());
|
||||
cmd = cmd.current_dir(&cuda_dir); // PyInstaller finds all DLLs relative to exe
|
||||
cmd = cmd.args(["--data-dir", &data_dir_str, "--port", &port_str, "--parent-pid", &parent_pid_str]);
|
||||
```
|
||||
|
||||
`.current_dir()` tells the PyInstaller bootloader that everything (DLLs, `nvidia/cublas/lib/`, `_internal/`, torch extensions, etc.) lives relative to the exe. Torch finds the NVIDIA libs exactly as it does in a normal `pip install` or dev environment -- no `LD_LIBRARY_PATH` hacks, no `os.add_dll_directory` gymnastics.
|
||||
|
||||
### Community evidence
|
||||
|
||||
- Multiple Tauri users run this exact pattern: Nuitka folders (exe + pythonXX.dll + supporting files), multi-file .NET apps, and PyInstaller onedir backends (GitHub issues #5719, discussion #5206).
|
||||
- The shell plugin explicitly supports `cwd` in both Rust and JS APIs.
|
||||
- No reports of torch/CUDA-specific breakage -- the onedir layout is identical to what PyInstaller produces in normal usage.
|
||||
|
||||
### Known gotcha: process termination on Windows
|
||||
|
||||
PyInstaller onedir creates a parent bootloader + child Python process on Windows. `child.kill()` only hits the outer process in some cases (Tauri issue #11686). Mitigation: keep a reference to the parent PID or use `taskkill /F /T` for clean shutdown. This is not a blocker -- our existing `--parent-pid` watchdog mechanism in `server.py` already handles orphan cleanup.
|
||||
|
||||
## Next Steps
|
||||
|
||||
1. Prototype: Build the current CUDA binary with `--onedir` and verify torch CUDA works from the output directory
|
||||
2. Measure the size split: how much is NVIDIA libs vs everything else
|
||||
3. Design the two-archive download flow and dual version checking
|
||||
4. Update `cuda.py` for dual-archive extraction (server core + cuda-libs)
|
||||
5. Update `main.rs`: change launch path to `backends/cuda/` dir + add `.current_dir()`
|
||||
6. Add `ensure_cuda_structure()` helper in Rust to verify exe + nvidia/ subdirs exist before spawning
|
||||
7. Update CI pipeline: `build-cuda-windows` produces two archives instead of split parts
|
||||
8. ~~Update `split_binary.py` or replace with archive-based distribution~~ Done: replaced with `package_cuda.py`
|
||||
Reference in New Issue
Block a user