# CUDA Backend Swap — Implementation Summary

> Status: **Complete** | Branch: `feat/cuda-backend-swap` | Created: 2026-03-12

## What This Is

A standalone feature that lets users download a CUDA-enabled backend binary (~2.4 GB) and swap it in via a backend-only restart. The frontend stays running, all UI state is preserved. This solves the #1 user pain point: 19 open issues about "GPU not detected" caused by GitHub's 2 GB release asset limit preventing CUDA binaries from shipping in official releases.

## How It Works

```
User clicks "Download CUDA Backend" in Settings
  → Backend fetches manifest from GitHub Releases
  → Downloads split parts (<2 GB each), concatenates them
  → SHA-256 integrity check on reassembled binary
  → Binary placed in {app_data_dir}/backends/voicebox-server-cuda
  → User clicks "Switch to CUDA Backend"
  → Tauri kills CPU process, launches CUDA binary, frontend reconnects
  → On all future app launches, CUDA binary is auto-detected and used
```

The CUDA binary is functionally identical to the CPU binary — same FastAPI app, same endpoints, same code. The only difference is PyTorch compiled with CUDA 12.1 and bundled CUDA runtime libraries.

## Architecture Decisions

**Backend-only restart, not full app restart.** The Tauri shell kills the current `voicebox-server` process, waits 1 second for port release, and spawns the new binary. The React frontend stays running. Health polling detects the new backend within seconds.

**No provider/subprocess architecture.** This is explicitly not the PR #33 approach (10K+ lines, 136 files, 22 bugs). One process at a time. The CUDA binary replaces the CPU binary — it doesn't run alongside it.

**Data directory, not app bundle.** The CUDA binary lives in `{app_data_dir}/backends/`, which persists across app updates and avoids code-signing issues. The bundled CPU binary in the app bundle is untouched.

**Version mismatch protection.** On startup, Rust runs `voicebox-server-cuda --version` and compares to the app version from `tauri.conf.json`. If they don't match (e.g., after an app update), it falls back to the bundled CPU binary silently.

**GitHub Releases distribution.** The CUDA binary is split into <2 GB chunks (GitHub's asset limit) via `scripts/split_binary.py`. The app downloads a manifest, fetches each part, concatenates them, and runs a SHA-256 integrity check to verify reassembly. No external hosting needed.

## Files Changed

### New Files

| File | Lines | Purpose |
|------|-------|---------|
| `backend/cuda_download.py` | ~190 | Download split parts from GitHub Releases, reassemble, verify integrity |
| `scripts/split_binary.py` | ~80 | Split large binary into <2 GB chunks with SHA-256 manifest |
| `.github/workflows/build-cuda.yml` | ~70 | CI workflow: build CUDA binary, split, upload to GitHub Releases |
| `app/src/components/ServerSettings/GpuAcceleration.tsx` | 371 | GPU Acceleration UI card (status, download, restart, delete) |
| `docs/plans/CUDA_BACKEND_SWAP.md` | 581 | Original implementation plan (5 phases with code sketches) |
| `docs/plans/CUDA_BACKEND_SWAP_FINAL.md` | this file | Final implementation summary |
| `docs/plans/PROJECT_STATUS.md` | 462 | Full project triage (all PRs, issues, architecture) |
| `docs/plans/PR33_CUDA_PROVIDER_REVIEW.md` | ~350 | Detailed code review of PR #33 (22 bugs documented) |

### Modified Files

| File | What Changed |
|------|-------------|
| `backend/build_binary.py` | Added `--cuda` flag, parameterized output binary name |
| `backend/server.py` | Added `--version` flag, auto-detect backend variant from binary name (`VOICEBOX_BACKEND_VARIANT` env var) |
| `backend/main.py` | 4 new endpoints (`/backend/cuda-status`, `/backend/download-cuda`, `/backend/cuda`, `/backend/cuda-progress`), health endpoint returns `backend_variant` |
| `backend/models.py` | `HealthResponse` model: added `backend_variant` field |
| `backend/requirements.txt` | Added `httpx>=0.27.0` for async HTTP downloads |
| `tauri/src-tauri/src/main.rs` | `restart_server` command (stop → wait → start), `start_server` checks for CUDA binary in data dir and launches via `shell().command()`, version mismatch check |
| `app/src/platform/types.ts` | `PlatformLifecycle.restartServer()` added |
| `tauri/src/platform/lifecycle.ts` | `restartServer()` implementation via `invoke('restart_server')` |
| `web/src/platform/lifecycle.ts` | `restartServer()` noop for web platform |
| `app/src/lib/api/types.ts` | `CudaStatus`, `CudaDownloadProgress` interfaces; `HealthResponse` updated with `gpu_type`, `backend_type`, `backend_variant` |
| `app/src/lib/api/client.ts` | `getCudaStatus()`, `downloadCudaBackend()`, `deleteCudaBackend()` methods |
| `app/src/components/ServerTab/ServerTab.tsx` | Wired in `<GpuAcceleration />` component (Tauri-only) |

## Backend API Endpoints

| Method | Path | Purpose |
|--------|------|---------|
| `GET` | `/backend/cuda-status` | Returns `{ available, active, binary_path, downloading, download_progress }` |
| `POST` | `/backend/download-cuda` | Starts background download; returns immediately. Track via SSE. |
| `DELETE` | `/backend/cuda` | Deletes CUDA binary (blocked if CUDA is currently active) |
| `GET` | `/backend/cuda-progress` | SSE stream of download progress (reuses existing `ProgressManager`) |

The existing `GET /health` endpoint now returns two new fields:
- `backend_type`: `"pytorch"` or `"mlx"` (existing detection)
- `backend_variant`: `"cpu"` or `"cuda"` (set from `VOICEBOX_BACKEND_VARIANT` env var)

## Frontend UI States

The `GpuAcceleration` card in Server Settings handles these states:

1. **Native GPU detected** (MPS, MLX, XPU, DirectML) — Shows info message, no download needed
2. **No CUDA binary** — Download button with size estimate, description of requirements
3. **Downloading** — SSE-driven progress bar with bytes/total and percentage
4. **Downloaded, not active** — "Switch to CUDA Backend" button + "Remove" option
5. **CUDA active** — Shows CUDA badge, "Switch to CPU Backend" button
6. **Restarting** — Spinner with phase text, 1s health polling as safety net
7. **Error** — Red error message with details

### Key UX detail: switching to CPU

Since `start_server` always prefers the CUDA binary if it exists on disk, "Switch to CPU" must delete the CUDA binary first, then restart. The user can re-download later. This avoids a persistent configuration mechanism (no new state to manage, no new config file, no DB column).

## Rust: Server Lifecycle

```
start_server
  ├── Check for CUDA binary at {data_dir}/backends/voicebox-server-cuda
  ├── If found: run --version, compare to app version
  │   ├── Match: launch via shell().command() with --data-dir, --port
  │   └── Mismatch: log warning, fall through to CPU
  └── Else: launch bundled sidecar via shell().sidecar()

restart_server
  ├── stop_server (kill process tree)
  ├── wait 1 second for port release
  └── start_server (auto-detects CUDA)
```

## What This Doesn't Cover

- **AMD GPU / ROCm / DirectML binary** — Same pattern, different PyTorch build. Future PR.
- **Linux CUDA** — Same approach, just another CI matrix entry. Can ship same release.
- **Multi-model support** — LuxTTS, Chatterbox, etc. are a separate architectural concern (in-process model registry). Independent of binary variant.
- **Download resume** — If download is interrupted, it restarts from scratch. Acceptable for v1.
- **Remote server CUDA** — Users running voicebox-server on a remote machine manage their own binaries. This feature is for the desktop app.

## Testing Checklist

- [ ] Build CUDA binary locally with `python backend/build_binary.py --cuda`
- [ ] `voicebox-server-cuda --version` prints correct version
- [ ] Place CUDA binary in `{data_dir}/backends/`, launch app → auto-detects and uses it
- [ ] Version mismatch: rename binary to have wrong version → falls back to CPU
- [ ] Frontend: GpuAcceleration card shows correct state for CPU, CUDA available, CUDA active
- [ ] Download flow: POST triggers download, SSE progress works, completion updates status
- [ ] Switch to CUDA: restart works, health endpoint shows `backend_variant: "cuda"`
- [ ] Switch to CPU: deletes binary, restarts, health shows `backend_variant: "cpu"`
- [ ] Delete CUDA while active: returns 409 error
- [ ] Split binary script: `python scripts/split_binary.py` creates manifest + parts + sha256
- [ ] Native GPU (macOS MPS): shows info message, no download section