Compare commits

..
Author SHA1 Message Date
Jamie Pine 3bdc18f278 Remove orphaned react-qr-code entries from lockfile
bun.lock was out of date with package.json (react-qr-code was removed
without reinstalling), failing the frozen-lockfile install in CI.
2026-07-05 03:17:33 -07:00
Jamie Pine 0b8fd31c89 Address review feedback on cloud login
- time out status polling after 2 min so an abandoned browser flow
  doesn't leave the button stuck on "Waiting for browser…"
- handle non-JSON / non-object payloads from the exchange and account
  endpoints instead of 500ing after the state is consumed
- make singleton row creation race-safe (IntegrityError -> re-query)
- clear device_name on disconnect along with the rest of the metadata
- serve the dashboard URL from /cloud/status so the Manage link follows
  VOICEBOX_CLOUD_URL instead of hardcoding production
- keep a "Disconnecting…" label on the disconnect button while pending
2026-07-04 21:07:59 -07:00
James Pine 376afad852 Add "Log in with browser" cloud device login
Connects the desktop app to Voicebox Cloud without the user ever handling an
API key. One button in Settings → General opens the system browser to
voicebox.sh, the user authorizes while signed in, and the credential lands
back in the app automatically.

Backend (FastAPI):
- /cloud/login/start opens the browser to the cloud authorize page with a
  state we mint; the existing loopback server catches the redirect at
  /cloud/callback and exchanges the one-time code (server-to-server, over TLS)
  for a voicebox_ API key, verifies it against the API, and stores it.
- /cloud/status and /cloud/disconnect back the settings UI.
- state round-trip guards against login-CSRF; the key never crosses a browser
  URL and is never exposed to the frontend (status returns a prefix only).
- CloudSettings singleton row; config gains VOICEBOX_CLOUD_URL /
  VOICEBOX_CLOUD_API_URL (default the prod hosts, overridable for dev).

Frontend (React):
- CloudSection in Settings → General: "Log in with browser", polls status,
  shows the connected device + a dashboard link. API keys are the advanced
  path only, surfaced in the web dashboard.

The key is stored in the local app DB for now; OS keychain is a marked
follow-up.
2026-06-28 16:51:18 -07:00
218 changed files with 3826 additions and 9991 deletions
-4
View File
@@ -37,7 +37,3 @@ replace = "version": "{new_version}"
[bumpversion:file:backend/__init__.py]
search = __version__ = "{current_version}"
replace = __version__ = "{new_version}"
[bumpversion:file:backend/pyproject.toml]
search = version = "{current_version}"
replace = version = "{new_version}"
-2
View File
@@ -1,2 +0,0 @@
package.json text eol=lf
scripts/*.sh text eol=lf
-66
View File
@@ -6,10 +6,6 @@ on:
branches:
- main
concurrency:
group: ci-${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
frontend-quality:
runs-on: ubuntu-latest
@@ -23,70 +19,8 @@ jobs:
- name: Install dependencies
run: bun install --frozen-lockfile
- name: Lint (biome)
run: bun run lint
- name: Typecheck app + web
run: bun run typecheck
- name: Unit tests
run: bun run test
- name: Build web smoke test
run: bun run build:web
backend-quality:
# macOS arm64 matches the primary user platform and lets the MLX-path
# tests run instead of being skipped.
runs-on: macos-14
steps:
- uses: actions/checkout@v4
- name: Setup Python
uses: actions/setup-python@v5
with:
python-version: "3.12"
- name: Cache pip downloads
uses: actions/cache@v4
with:
path: ~/Library/Caches/pip
key: pip-${{ runner.os }}-${{ hashFiles('backend/requirements*.txt', 'justfile') }}
restore-keys: pip-${{ runner.os }}-
- name: Install just
run: brew install just
- name: Install backend dependencies
run: just setup-python
- name: Lint (ruff)
run: venv/bin/ruff check .
working-directory: backend
- name: Run tests
run: venv/bin/python -m pytest tests -q
working-directory: backend
rust-quality:
runs-on: macos-14
steps:
- uses: actions/checkout@v4
- name: Rust cache
uses: Swatinem/rust-cache@v2
with:
workspaces: tauri/src-tauri
- name: Stub sidecar binaries
# tauri-build validates externalBin paths; real sidecars are only
# produced by the release pipeline.
run: |
mkdir -p tauri/src-tauri/binaries tauri/dist
touch tauri/src-tauri/binaries/voicebox-server-aarch64-apple-darwin
touch tauri/src-tauri/binaries/voicebox-mcp-aarch64-apple-darwin
- name: Cargo check
run: cargo check --manifest-path tauri/src-tauri/Cargo.toml
-61
View File
@@ -340,64 +340,3 @@ jobs:
name: voicebox-server-cuda-windows
path: backend/dist/voicebox-server-cuda/
retention-days: 7
build-rocm-windows:
runs-on: windows-latest
permissions:
contents: write
steps:
- uses: actions/checkout@v4
- name: Setup Python
uses: actions/setup-python@v5
with:
# ROCm wheels are cp312-cp312-specific — build_binary.py --rocm enforces this.
python-version: "3.12"
cache: "pip"
- name: Install Python dependencies
run: |
python -m pip install --upgrade pip
pip install pyinstaller
pip install -r backend/requirements.txt
pip install --no-deps chatterbox-tts
pip install --no-deps hume-tada
- name: Build ROCm server binary (onedir)
shell: bash
working-directory: backend
# build_binary.py --rocm pulls the official AMD Radeon torch + rocm_sdk
# wheels (rocm-rel-7.2.1) itself when ROCm torch is not already present,
# then restores the dev torch afterwards.
run: python build_binary.py --rocm
- name: Package into server core + ROCm libs archives
shell: bash
run: |
python scripts/package_rocm.py \
backend/dist/voicebox-server-rocm/ \
--output release-assets/ \
--rocm-libs-version rocm7.2-v1 \
--torch-compat ">=2.9.0,<2.10.0"
- name: Upload archives to GitHub Release
if: startsWith(github.ref, 'refs/tags/')
uses: softprops/action-gh-release@v2
with:
files: |
release-assets/voicebox-server-rocm.tar.gz
release-assets/voicebox-server-rocm.tar.gz.sha256
release-assets/rocm-libs-rocm7.2-v1.tar.gz
release-assets/rocm-libs-rocm7.2-v1.tar.gz.sha256
release-assets/rocm-libs.json
draft: true
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: Upload onedir as workflow artifact
uses: actions/upload-artifact@v4
with:
name: voicebox-server-rocm-windows
path: backend/dist/voicebox-server-rocm/
retention-days: 7
-12
View File
@@ -37,7 +37,6 @@ Thumbs.db
# Data (user-generated)
data/
!data/.gitkeep
output/
# Logs
*.log
@@ -69,14 +68,3 @@ temp/
backend/tests/results/
backend/tests/fixtures/reference_voice.wav
backend/tests/fixtures/reference_voice.txt
# Local AI assistant workspace settings
.claude/settings.local.json
# Local worktrees / agent workspaces
.worktrees/
.hermes/
# Local MLX experiments
mlx-test/
-11
View File
@@ -5,17 +5,6 @@
# Changelog
## [Unreleased]
### Linux
- **ROCm setup works on Linux AMD systems.** Docker ROCm builds now keep PyTorch
on the ROCm wheel index during dependency installation, so later installs do
not replace it with CUDA wheels. The ROCm compose overlay no longer assumes
Ubuntu render/video group IDs; the container joins the groups that own the GPU
device nodes at startup. Native Linux setup now picks ROCm wheels for AMD GPUs
and CUDA wheels for NVIDIA GPUs before installing backend dependencies.
## [0.5.0] - 2026-04-22
**The Capture release.** Voicebox stops being just a voice-cloning studio and becomes a full AI voice studio. Hold a key anywhere on your machine, speak, release — the transcript lands in the focused text field. Flip the primitive around and any MCP-aware agent — Claude Code, Cursor, Spacebot — speaks back through an on-screen pill in one of your cloned voices. A local LLM sits between the two, so transcripts come out clean and voice profiles can carry a personality that reshapes what the agent says before it gets spoken.
+26 -24
View File
@@ -18,9 +18,9 @@ Thank you for your interest in contributing to Voicebox! This document provides
curl -fsSL https://bun.sh/install | bash
```
- **[Python 3.12+](https://python.org)** - For backend development
- **[Python 3.11+](https://python.org)** - For backend development
```bash
python --version # Should be 3.12 or higher
python --version # Should be 3.11 or higher
```
- **[Rust](https://rustup.rs)** - For Tauri desktop app (installed automatically by Tauri CLI)
@@ -91,7 +91,7 @@ On Windows, to build with CUDA support for local testing:
just build-local # Build CPU + CUDA server binaries + Tauri installer
```
This builds the CPU sidecar (bundled with the app), the CUDA binary (placed in `%APPDATA%/sh.voicebox.app/backends/` for runtime GPU switching), and the installable Tauri app.
This builds the CPU sidecar (bundled with the app), the CUDA binary (placed in `%APPDATA%/com.voicebox.app/backends/` for runtime GPU switching), and the installable Tauri app.
Creates platform-specific installers (`.dmg`, `.msi`, `.AppImage`) in `tauri/src-tauri/target/release/bundle/`.
@@ -115,6 +115,14 @@ just build-server
This makes PyInstaller use your local qwen-tts version instead of the pip-installed package.
### Generate OpenAPI Client
After starting the backend server:
```bash
./scripts/generate-api.sh
```
This downloads the OpenAPI schema and generates the TypeScript client in `app/src/lib/api/`
### Convert Assets to Web Formats
To optimize images and videos for the web, run:
@@ -204,7 +212,7 @@ export const ProfileCard = (props) => { ... }
- Follow PEP 8 style guide
- Use type hints
- Use async/await for I/O operations
- Format and lint with ruff (configured in `backend/pyproject.toml`)
- Format with Black (if configured)
```python
# Good
@@ -234,12 +242,9 @@ voicebox/
│ ├── lib/ # Utilities and API client
│ └── hooks/ # React hooks
├── backend/ # Python FastAPI server
│ ├── main.py # Entry point (FastAPI app assembled in app.py)
│ ├── routes/ # API routers, one per domain
│ ├── services/ # Business logic (generation, transcription, profiles, ...)
│ ├── backends/ # TTS engine implementations
│ ├── database/ # SQLAlchemy models, sessions, migrations
│ └── tests/ # pytest suite
│ ├── main.py # API routes
│ ├── tts.py # Voice synthesis
│ └── ...
├── tauri/ # Desktop app wrapper
│ └── src-tauri/ # Rust backend
└── scripts/ # Build scripts
@@ -284,26 +289,23 @@ voicebox/
When adding new API endpoints:
1. **Add the route to the relevant router in `backend/routes/`** (new routers get registered in `backend/routes/__init__.py`)
1. **Add route in `backend/main.py`**
2. **Create Pydantic models in `backend/models.py`**
3. **Implement business logic in appropriate module**
4. **Update OpenAPI schema** (automatic with FastAPI)
5. **Update the TypeScript client** — add matching types to `app/src/lib/api/types.ts` and a method to `app/src/lib/api/client.ts`
5. **Regenerate TypeScript client:**
```bash
bun run generate:api
```
6. **Update `backend/README.md`** with endpoint documentation
## Testing
Backend tests live in `backend/tests/` and run with pytest:
Currently, testing is primarily manual. When adding tests:
```bash
cd backend
venv/bin/python -m pytest tests
```
CI runs the backend test suite on every PR, along with frontend lint and typecheck (`bun run lint`, `bun run typecheck`) and a `cargo check` of the Tauri app (see `.github/workflows/ci.yml`). Add backend tests alongside your changes where it makes sense.
- **Frontend**: Vitest for React component tests (coverage is still sparse — contributions welcome)
- **E2E**: Playwright for end-to-end tests (future)
- **Backend**: Use pytest for Python tests
- **Frontend**: Use Vitest for React component tests
- **E2E**: Use Playwright for end-to-end tests (future)
## Pull Request Process
@@ -361,7 +363,7 @@ See [docs/content/docs/overview/troubleshooting.mdx](docs/content/docs/overview/
**Quick fixes:**
- **Backend won't start:** Check Python version (3.12+), ensure venv is activated, install dependencies
- **Backend won't start:** Check Python version (3.11+), ensure venv is activated, install dependencies
- **Tauri build fails:** Ensure Rust is installed, clean build with `cd tauri/src-tauri && cargo clean`
- **OpenAPI client generation fails:** Ensure backend is running, check `curl http://localhost:17493/openapi.json`
@@ -377,7 +379,7 @@ See [docs/content/docs/overview/troubleshooting.mdx](docs/content/docs/overview/
- [README.md](README.md) - Project overview
- [backend/README.md](backend/README.md) - API documentation
- [docs/PROJECT_STATUS.md](docs/PROJECT_STATUS.md) - Living engineering roadmap: architecture, shipped vs in-flight work, prioritized open issues, candidate TTS engines under evaluation, architectural bottlenecks. Keep this updated when you ship significant features, close or backlog a model integration, or identify new bottlenecks.
- [docs/content/docs/developer/autoupdater.mdx](docs/content/docs/developer/autoupdater.mdx) - Auto-updater setup (published at [voicebox.sh docs](https://voicebox.sh/docs/developer/autoupdater))
- [docs/AUTOUPDATER_QUICKSTART.md](docs/AUTOUPDATER_QUICKSTART.md) - Auto-updater setup
- [SECURITY.md](SECURITY.md) - Security policy
- [CHANGELOG.md](CHANGELOG.md) - Version history
+10 -40
View File
@@ -1,15 +1,8 @@
# ============================================================
# Voicebox — Local TTS Server with Web UI
# Voicebox — Local TTS Server with Web UI (CPU)
# 3-stage build: Frontend → Python deps → Runtime
#
# Build variants:
# CPU (default): docker compose up --build
# ROCm (AMD GPU): docker compose -f docker-compose.yml -f docker-compose.rocm.yml up --build
# ============================================================
# Top-level ARG so it is visible to all stages.
ARG PYTORCH_VARIANT=cpu
# === Stage 1: Build frontend ===
FROM oven/bun:1 AS frontend
@@ -20,11 +13,8 @@ COPY package.json bun.lock CHANGELOG.md ./
COPY app/ ./app/
COPY web/ ./web/
# Normalize line endings first (a Windows CRLF checkout would otherwise
# defeat the `-z 's/,\n ]/…/'` match below, since it's LF-anchored), then
# strip workspaces not needed for web build, and fix trailing comma
RUN sed -i 's/\r$//' package.json && \
sed -i '/"tauri"/d; /"landing"/d' package.json && \
# Strip workspaces not needed for web build, and fix trailing comma
RUN sed -i '/"tauri"/d; /"landing"/d' package.json && \
sed -i -z 's/,\n ]/\n ]/' package.json
RUN bun install --no-save
# Build frontend (skip tsc — upstream has pre-existing type errors)
@@ -34,9 +24,6 @@ RUN cd web && bunx --bun vite build
# === Stage 2: Build Python dependencies ===
FROM python:3.11-slim AS backend-builder
# Re-declare ARG inside the stage (Docker scoping requirement).
ARG PYTORCH_VARIANT=cpu
WORKDIR /build
RUN apt-get update && apt-get install -y --no-install-recommends \
@@ -47,40 +34,26 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
RUN pip install --no-cache-dir --upgrade pip
COPY backend/requirements.txt .
# ROCm wheel index. Default 6.3 (RDNA1/2/3); set ROCM_VERSION=7.2 for RDNA4.
ARG ROCM_VERSION=6.3
# For ROCm, make the PyTorch ROCm index primary so every install below resolves
# torch to ROCm wheels instead of the default CUDA build.
RUN if [ "$PYTORCH_VARIANT" = "rocm" ]; then \
pip install --no-cache-dir --prefix=/install \
--index-url "https://download.pytorch.org/whl/rocm${ROCM_VERSION}" \
torch torchaudio && \
printf '[global]\nindex-url = https://download.pytorch.org/whl/rocm%s\nextra-index-url = https://pypi.org/simple\n' "$ROCM_VERSION" > /etc/pip.conf; \
fi
RUN pip install --no-cache-dir --prefix=/install -r requirements.txt
RUN pip install --no-cache-dir --prefix=/install --no-deps chatterbox-tts
RUN pip install --no-cache-dir --prefix=/install --no-deps hume-tada
RUN pip install --no-cache-dir --prefix=/install \
git+https://github.com/QwenLM/Qwen3-TTS.git@022e286b98fbec7e1e916cb940cdf532cd9f488e
git+https://github.com/QwenLM/Qwen3-TTS.git
# === Stage 3: Runtime ===
FROM python:3.11-slim
# Create non-root user; the entrypoint joins GPU device groups at runtime.
# Create non-root user for security
RUN groupadd -r voicebox && \
useradd -r -g voicebox -m -s /bin/bash voicebox
WORKDIR /app
# Install only runtime system dependencies (gosu drops root in the entrypoint)
# Install only runtime system dependencies
RUN apt-get update && apt-get install -y --no-install-recommends \
ffmpeg \
curl \
gosu \
&& rm -rf /var/lib/apt/lists/*
# Copy installed Python packages from builder stage
@@ -96,6 +69,9 @@ COPY --from=frontend --chown=voicebox:voicebox /build/web/dist /app/frontend/
RUN mkdir -p /app/data/generations /app/data/profiles /app/data/cache \
&& chown -R voicebox:voicebox /app/data
# Switch to non-root user
USER voicebox
# Expose the API port
EXPOSE 17493
@@ -103,11 +79,5 @@ EXPOSE 17493
HEALTHCHECK --interval=30s --timeout=10s --retries=3 --start-period=60s \
CMD curl -f http://localhost:17493/health || exit 1
# Entrypoint joins GPU groups then drops to the voicebox user.
# Normalize CRLF (a Windows checkout otherwise leaves the shebang as
# `#!/bin/sh\r`, which Linux can't resolve — reported as a misleading
# "no such file or directory" even though the file exists).
COPY --chmod=755 scripts/rocm-entrypoint.sh /usr/local/bin/entrypoint.sh
RUN sed -i 's/\r$//' /usr/local/bin/entrypoint.sh
ENTRYPOINT ["/usr/local/bin/entrypoint.sh"]
# Start the FastAPI server
CMD ["uvicorn", "backend.main:app", "--host", "0.0.0.0", "--port", "17493"]
+4 -4
View File
@@ -6,8 +6,8 @@ We release patches for security vulnerabilities. Which versions are eligible for
| Version | Supported |
| ------- | ------------------ |
| 0.5.x | :white_check_mark: |
| < 0.5 | :x: |
| 0.3.x | :white_check_mark: |
| < 0.3 | :x: |
## Reporting a Vulnerability
@@ -39,7 +39,7 @@ We will:
### For Developers
- **Dependencies** - Keep all dependencies up to date
- **CI checks** - Every PR must pass typecheck, lint, backend tests, and `cargo check` before merging
- **Code review** - All PRs require review before merging
- **Secrets** - Never commit API keys or signing keys
- **Signing** - All releases are cryptographically signed
@@ -82,7 +82,7 @@ Timeline may vary based on severity and complexity.
## Security Updates
Security updates will be:
- Released as patch versions (e.g., 0.5.1)
- Released as patch versions (e.g., 0.3.2)
- Documented in CHANGELOG.md
- Announced via GitHub releases
- Automatically delivered via auto-updater
+1 -1
View File
@@ -6,7 +6,7 @@
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>voicebox</title>
<script>
(() => {
(function () {
try {
var theme = 'system';
var raw = localStorage.getItem('voicebox-ui');
-2
View File
@@ -7,7 +7,6 @@
"dev": "vite",
"build": "vite build",
"typecheck": "tsc -p tsconfig.json --noEmit",
"test": "bun test",
"preview": "vite preview",
"lint": "biome lint src",
"lint:fix": "biome lint --write src",
@@ -61,7 +60,6 @@
},
"devDependencies": {
"@tailwindcss/vite": "^4.1.18",
"@types/bun": "^1.3.4",
"@types/react": "^18.3.0",
"@types/react-dom": "^18.3.0",
"@vitejs/plugin-react": "^4.3.0",
@@ -35,17 +35,19 @@ export function DictateWindow() {
};
}, []);
// Mirrored from the main window: true only when dictation is armed and the
// user opted into keeping the microphone ready.
const [micWarm, setMicWarm] = useState(false);
// Snapshot of the focused UI element at chord-start, shipped over from
// Rust on the ``dictate:start`` payload. Held in a ref so it survives
// the 1–2 s transcribe + refine window — the paste only fires once the
// final text comes back.
const focusRef = useRef<FocusSnapshot | null>(null);
const session = useCaptureRecordingSession({
keepMicWarm: micWarm,
onFinalText: async (text, _capture, allowAutoPaste, context) => {
// Focus is the snapshot taken at chord-start and threaded through as this
// take's context, so it survives the 1–2 s transcribe + refine window and
// overlapping dictations can't paste into each other's target.
const focus = context as FocusSnapshot | null;
onFinalText: async (text, _capture, allowAutoPaste) => {
const focus = focusRef.current;
// Consume-once: a second chord before this fires would overwrite
// focusRef, but nulling it here guards against the late-arriving
// refine-result firing a paste after the user has moved on.
focusRef.current = null;
if (!allowAutoPaste) return;
if (!focus || !text.trim()) return;
try {
@@ -70,41 +72,23 @@ export function DictateWindow() {
sessionRef.current = session;
useEffect(() => {
let disposed = false;
const unlistens: UnlistenFn[] = [];
const registrations = [
const unlistens: Promise<UnlistenFn>[] = [];
unlistens.push(
listen<{ focus: FocusSnapshot | null }>('dictate:start', (event) => {
sessionRef.current.startRecording(event.payload?.focus ?? null);
focusRef.current = event.payload?.focus ?? null;
sessionRef.current.startRecording();
}),
);
unlistens.push(
listen('dictate:stop', () => {
// Forward stops that arrive while getUserMedia is still resolving.
sessionRef.current.stopRecording();
if (sessionRef.current.isRecording) sessionRef.current.stopRecording();
}),
listen<boolean>('dictate:warm', (event) => {
setMicWarm(Boolean(event.payload));
}),
];
Promise.all(registrations)
.then((registered) => {
if (disposed) {
for (const unlisten of registered) unlisten();
return;
}
unlistens.push(...registered);
emit('dictate:warm-request').catch(() => {});
})
.catch((err) => console.warn('[dictate] event listener registration failed:', err));
);
return () => {
disposed = true;
for (const unlisten of unlistens) unlisten();
for (const p of unlistens) p.then((fn) => fn()).catch(() => {});
};
}, []);
useEffect(() => {
if (micWarm) void session.prewarm();
else session.releaseWarm();
}, [micWarm, session.prewarm, session.releaseWarm]);
// --- Agent-speak cycle ---------------------------------------------------
const [speaking, setSpeaking] = useState<{
@@ -139,7 +139,7 @@ export function EngineModelSelector({ form, compact, selectedProfile }: EngineMo
<SelectValue />
</SelectTrigger>
</FormControl>
<SelectContent side={compact ? 'top' : undefined}>
<SelectContent>
{availableOptions.map((opt) => (
<SelectItem key={opt.value} value={opt.value} className={itemClass}>
{opt.label}
@@ -555,7 +555,7 @@ export function FloatingGenerateBox({
<SelectTrigger className="h-8 text-xs bg-card border-border rounded-full hover:bg-background/50 transition-all w-full">
<SelectValue placeholder={t('generation.voiceSelector.placeholder')} />
</SelectTrigger>
<SelectContent side="top">
<SelectContent>
{profiles?.map((profile) => (
<SelectItem key={profile.id} value={profile.id} className="text-xs">
{profile.name}
@@ -582,7 +582,7 @@ export function FloatingGenerateBox({
<SelectValue />
</SelectTrigger>
</FormControl>
<SelectContent side="top">
<SelectContent>
{engineLangs.map((lang) => (
<SelectItem key={lang.value} value={lang.value} className="text-xs">
{lang.label}
@@ -610,7 +610,7 @@ export function FloatingGenerateBox({
<SelectTrigger className="h-8 text-xs bg-card border-border rounded-full hover:bg-background/50 transition-all">
<SelectValue placeholder={t('generation.effects.none')} />
</SelectTrigger>
<SelectContent side="top">
<SelectContent>
<SelectItem value="none" className="text-xs">
{t('generation.effects.none')}
</SelectItem>
@@ -5,7 +5,7 @@ import { Button } from '@/components/ui/button';
import { Card, CardContent, CardHeader, CardTitle } from '@/components/ui/card';
import { Progress } from '@/components/ui/progress';
import { apiClient } from '@/lib/api/client';
import type { CudaDownloadProgress, RocmDownloadProgress } from '@/lib/api/types';
import type { CudaDownloadProgress } from '@/lib/api/types';
import { useServerHealth } from '@/lib/hooks/useServer';
import { usePlatform } from '@/platform/PlatformContext';
import { useServerStore } from '@/stores/serverStore';
@@ -21,9 +21,6 @@ export function GpuAcceleration() {
const [restartPhase, setRestartPhase] = useState<RestartPhase>('idle');
const [error, setError] = useState<string | null>(null);
const [downloadProgress, setDownloadProgress] = useState<CudaDownloadProgress | null>(null);
const [rocmDownloadProgress, setRocmDownloadProgress] = useState<RocmDownloadProgress | null>(
null,
);
const healthPollRef = useRef<ReturnType<typeof setInterval> | null>(null);
// Query CUDA backend status
@@ -39,26 +36,10 @@ export function GpuAcceleration() {
enabled: !!health, // Only fetch when backend is reachable
});
// Query ROCm backend status
const {
data: rocmStatus,
isLoading: _rocmStatusLoading,
refetch: refetchRocmStatus,
} = useQuery({
queryKey: ['rocm-status', serverUrl],
queryFn: () => apiClient.getRocmStatus(),
refetchInterval: (query) => (query.state.status === 'pending' ? false : 10000),
retry: 1,
enabled: !!health, // Only fetch when backend is reachable
});
// Derived state
const isCurrentlyCuda = health?.backend_variant === 'cuda';
const isCurrentlyRocm = health?.backend_variant === 'rocm';
const cudaAvailable = cudaStatus?.available ?? false;
const cudaDownloading = cudaStatus?.downloading ?? false;
const rocmAvailable = rocmStatus?.available ?? false;
const rocmDownloading = rocmStatus?.downloading ?? false;
// Clean up health poll on unmount
useEffect(() => {
@@ -70,7 +51,7 @@ export function GpuAcceleration() {
};
}, []);
// SSE progress tracking during CUDA download
// SSE progress tracking during download
useEffect(() => {
if (!cudaDownloading || !serverUrl) {
return;
@@ -107,43 +88,6 @@ export function GpuAcceleration() {
};
}, [cudaDownloading, serverUrl, refetchCudaStatus]);
// SSE progress tracking during ROCm download
useEffect(() => {
if (!rocmDownloading || !serverUrl) {
return;
}
const eventSource = new EventSource(`${serverUrl}/backend/rocm-progress`);
eventSource.onmessage = (event) => {
try {
const data = JSON.parse(event.data) as RocmDownloadProgress;
setRocmDownloadProgress(data);
if (data.status === 'complete') {
eventSource.close();
setRocmDownloadProgress(null);
refetchRocmStatus();
} else if (data.status === 'error') {
eventSource.close();
setError(data.error || 'Download failed');
setRocmDownloadProgress(null);
refetchRocmStatus();
}
} catch (e) {
console.error('Error parsing ROCm progress event:', e);
}
};
eventSource.onerror = () => {
eventSource.close();
};
return () => {
eventSource.close();
};
}, [rocmDownloading, serverUrl, refetchRocmStatus]);
// Start aggressive health polling during restart
const startHealthPolling = useCallback(() => {
if (healthPollRef.current) return;
@@ -169,7 +113,7 @@ export function GpuAcceleration() {
}, 1000);
}, [queryClient]);
const handleDownloadCuda = async () => {
const handleDownload = async () => {
setError(null);
try {
await apiClient.downloadCudaBackend();
@@ -184,21 +128,6 @@ export function GpuAcceleration() {
}
};
const handleDownloadRocm = async () => {
setError(null);
try {
await apiClient.downloadRocmBackend();
refetchRocmStatus();
} catch (e: unknown) {
const msg = e instanceof Error ? e.message : 'Failed to start download';
if (msg.includes('already downloaded')) {
refetchRocmStatus();
} else {
setError(msg);
}
}
};
const handleRestart = async () => {
setError(null);
setRestartPhase('stopping');
@@ -225,17 +154,18 @@ export function GpuAcceleration() {
}
};
const handleSwitchToCpuFromCuda = async () => {
const handleSwitchToCpu = async () => {
// To switch to CPU: delete the CUDA binary, then restart.
// start_server always prefers CUDA if present, so we must remove it first.
setError(null);
setRestartPhase('stopping');
try {
// Tell Rust launcher to skip GPU binary detection on next start.
// We cannot delete an active .exe on Windows, so we override instead.
await platform.lifecycle.setBackendOverride('cpu');
await apiClient.deleteCudaBackend();
setRestartPhase('waiting');
startHealthPolling();
await platform.lifecycle.restartServer();
// Invoke resolved — server is likely ready
if (healthPollRef.current) {
clearInterval(healthPollRef.current);
healthPollRef.current = null;
@@ -254,36 +184,7 @@ export function GpuAcceleration() {
}
};
const handleSwitchToCpuFromRocm = async () => {
setError(null);
setRestartPhase('stopping');
try {
// Tell Rust launcher to skip GPU binary detection on next start.
// We cannot delete an active .exe on Windows, so we override instead.
await platform.lifecycle.setBackendOverride('cpu');
setRestartPhase('waiting');
startHealthPolling();
await platform.lifecycle.restartServer();
if (healthPollRef.current) {
clearInterval(healthPollRef.current);
healthPollRef.current = null;
}
setRestartPhase('ready');
queryClient.invalidateQueries();
setTimeout(() => setRestartPhase('idle'), 2000);
} catch (e: unknown) {
setRestartPhase('idle');
if (healthPollRef.current) {
clearInterval(healthPollRef.current);
healthPollRef.current = null;
}
setError(e instanceof Error ? e.message : 'Failed to switch to CPU');
refetchRocmStatus();
}
};
const handleDeleteCuda = async () => {
const handleDelete = async () => {
setError(null);
try {
await apiClient.deleteCudaBackend();
@@ -293,16 +194,6 @@ export function GpuAcceleration() {
}
};
const handleDeleteRocm = async () => {
setError(null);
try {
await apiClient.deleteRocmBackend();
refetchRocmStatus();
} catch (e: unknown) {
setError(e instanceof Error ? e.message : 'Failed to delete ROCm backend');
}
};
const formatBytes = (bytes: number): string => {
if (bytes === 0) return '0 B';
const k = 1024;
@@ -314,7 +205,7 @@ export function GpuAcceleration() {
// Don't render until health data is available
if (!health) return null;
// If the system already has native GPU (MPS, ROCm active, etc.), only show info - no download needed
// If the system already has native GPU (MPS, etc.), only show info - no CUDA needed
const hasNativeGpu =
health.gpu_available &&
!isCurrentlyCuda &&
@@ -350,6 +241,8 @@ export function GpuAcceleration() {
)}
</div>
{/* Native GPU detected - no CUDA download needed */}
{/* Currently running CUDA - show switch back to CPU */}
{isCurrentlyCuda && platform.metadata.isTauri && (
<>
@@ -368,12 +261,7 @@ export function GpuAcceleration() {
Running with CUDA GPU acceleration. Switch back to CPU if needed (you can
re-download later).
</p>
<Button
onClick={handleSwitchToCpuFromCuda}
variant="outline"
className="w-full"
size="sm"
>
<Button onClick={handleSwitchToCpu} variant="outline" className="w-full" size="sm">
<RotateCw className="h-4 w-4 mr-2" />
Switch to CPU Backend
</Button>
@@ -388,207 +276,39 @@ export function GpuAcceleration() {
</>
)}
{/* Currently running ROCm - show switch back to CPU */}
{isCurrentlyRocm && platform.metadata.isTauri && (
{/* CUDA download/manage section - show when no native GPU and not currently running CUDA */}
{!hasNativeGpu && !isCurrentlyCuda && (
<>
{restartPhase !== 'idle' ? (
<div className="flex items-center gap-2 p-3 rounded-lg bg-primary/5 border">
<Loader2 className="h-4 w-4 animate-spin" />
<span className="text-sm">
{restartPhase === 'stopping' && 'Stopping server...'}
{restartPhase === 'waiting' && 'Restarting server...'}
{restartPhase === 'ready' && 'Server restarted successfully!'}
</span>
</div>
) : (
<div className="space-y-3">
<p className="text-sm text-muted-foreground">
Running with ROCm GPU acceleration for AMD. Switch back to CPU if needed (you can
re-download later).
</p>
<Button
onClick={handleSwitchToCpuFromRocm}
variant="outline"
className="w-full"
size="sm"
>
<RotateCw className="h-4 w-4 mr-2" />
Switch to CPU Backend
</Button>
</div>
)}
{error && (
<div className="flex items-center gap-2 text-sm text-destructive">
<AlertCircle className="h-4 w-4 shrink-0" />
<span>{error}</span>
</div>
)}
</>
)}
{/* Backend download/manage sections - show when no native GPU and not currently running GPU */}
{!hasNativeGpu && !isCurrentlyCuda && !isCurrentlyRocm && (
<>
{/* CUDA Section */}
<div className="space-y-4">
<div className="text-sm font-medium">NVIDIA (CUDA)</div>
{/* CUDA Download progress */}
{cudaDownloading && downloadProgress && (
<div className="space-y-2">
<div className="flex items-center justify-between text-sm">
<div className="flex items-center gap-2">
<Loader2 className="h-4 w-4 animate-spin" />
<span>
{downloadProgress.filename ||
(cudaAvailable
? 'Updating CUDA backend...'
: 'Downloading CUDA backend...')}
</span>
</div>
{downloadProgress.total > 0 && (
<span className="text-muted-foreground">
{downloadProgress.progress.toFixed(1)}%
</span>
)}
{/* Download progress (manual download or auto-update) */}
{cudaDownloading && downloadProgress && (
<div className="space-y-2">
<div className="flex items-center justify-between text-sm">
<div className="flex items-center gap-2">
<Loader2 className="h-4 w-4 animate-spin" />
<span>
{downloadProgress.filename ||
(cudaAvailable
? 'Updating CUDA backend...'
: 'Downloading CUDA backend...')}
</span>
</div>
{downloadProgress.total > 0 && (
<>
<Progress value={downloadProgress.progress} className="h-2" />
<div className="text-xs text-muted-foreground">
{formatBytes(downloadProgress.current)} /{' '}
{formatBytes(downloadProgress.total)}
</div>
</>
<span className="text-muted-foreground">
{downloadProgress.progress.toFixed(1)}%
</span>
)}
</div>
)}
{/* CUDA Actions */}
{restartPhase === 'idle' && !cudaDownloading && (
<div className="space-y-2">
{!cudaAvailable && (
<div className="space-y-3">
<p className="text-sm text-muted-foreground">
Download the CUDA backend (~2.4 GB) for NVIDIA GPU acceleration. Requires an
NVIDIA GPU with CUDA support.
</p>
<Button onClick={handleDownloadCuda} className="w-full" size="sm">
<Download className="h-4 w-4 mr-2" />
Download CUDA Backend
</Button>
{downloadProgress.total > 0 && (
<>
<Progress value={downloadProgress.progress} className="h-2" />
<div className="text-xs text-muted-foreground">
{formatBytes(downloadProgress.current)} /{' '}
{formatBytes(downloadProgress.total)}
</div>
)}
{cudaAvailable && platform.metadata.isTauri && (
<div className="space-y-3">
<p className="text-sm text-muted-foreground">
CUDA backend is downloaded and ready. Restart the server to enable GPU
acceleration.
</p>
<Button onClick={handleRestart} className="w-full" size="sm">
<RotateCw className="h-4 w-4 mr-2" />
Switch to CUDA Backend
</Button>
</div>
)}
{cudaAvailable && (
<Button
onClick={handleDeleteCuda}
variant="ghost"
className="w-full text-muted-foreground hover:text-destructive"
size="sm"
>
<Trash2 className="h-4 w-4 mr-2" />
Remove CUDA Backend
</Button>
)}
</div>
)}
</div>
{/* Divider */}
<div className="border-t" />
{/* ROCm Section */}
<div className="space-y-4">
<div className="text-sm font-medium">AMD (ROCm)</div>
{/* ROCm Download progress */}
{rocmDownloading && rocmDownloadProgress && (
<div className="space-y-2">
<div className="flex items-center justify-between text-sm">
<div className="flex items-center gap-2">
<Loader2 className="h-4 w-4 animate-spin" />
<span>
{rocmDownloadProgress.filename ||
(rocmAvailable
? 'Updating ROCm backend...'
: 'Downloading ROCm backend...')}
</span>
</div>
{rocmDownloadProgress.total > 0 && (
<span className="text-muted-foreground">
{rocmDownloadProgress.progress.toFixed(1)}%
</span>
)}
</div>
{rocmDownloadProgress.total > 0 && (
<>
<Progress value={rocmDownloadProgress.progress} className="h-2" />
<div className="text-xs text-muted-foreground">
{formatBytes(rocmDownloadProgress.current)} /{' '}
{formatBytes(rocmDownloadProgress.total)}
</div>
</>
)}
</div>
)}
{/* ROCm Actions */}
{restartPhase === 'idle' && !rocmDownloading && (
<div className="space-y-2">
{!rocmAvailable && (
<div className="space-y-3">
<p className="text-sm text-muted-foreground">
Download the ROCm backend (~2-3 GB) for AMD GPU acceleration. Requires an
AMD Radeon GPU with ROCm support.
</p>
<Button onClick={handleDownloadRocm} className="w-full" size="sm">
<Download className="h-4 w-4 mr-2" />
Download AMD ROCm Backend
</Button>
</div>
)}
{rocmAvailable && platform.metadata.isTauri && (
<div className="space-y-3">
<p className="text-sm text-muted-foreground">
ROCm backend is downloaded and ready. Restart the server to enable AMD GPU
acceleration.
</p>
<Button onClick={handleRestart} className="w-full" size="sm">
<RotateCw className="h-4 w-4 mr-2" />
Switch to ROCm Backend
</Button>
</div>
)}
{rocmAvailable && (
<Button
onClick={handleDeleteRocm}
variant="ghost"
className="w-full text-muted-foreground hover:text-destructive"
size="sm"
>
<Trash2 className="h-4 w-4 mr-2" />
Remove ROCm Backend
</Button>
)}
</div>
)}
</div>
</>
)}
</div>
)}
{/* Restart in progress */}
{restartPhase !== 'idle' && (
@@ -609,6 +329,52 @@ export function GpuAcceleration() {
<span>{error}</span>
</div>
)}
{/* Actions */}
{restartPhase === 'idle' && !cudaDownloading && (
<div className="space-y-2">
{/* Not downloaded yet - show download button */}
{!cudaAvailable && (
<div className="space-y-3">
<p className="text-sm text-muted-foreground">
Download the CUDA backend (~2.4 GB) for NVIDIA GPU acceleration. Requires an
NVIDIA GPU with CUDA support.
</p>
<Button onClick={handleDownload} className="w-full" size="sm">
<Download className="h-4 w-4 mr-2" />
Download CUDA Backend
</Button>
</div>
)}
{/* Downloaded but not active - show switch button */}
{cudaAvailable && platform.metadata.isTauri && (
<div className="space-y-3">
<p className="text-sm text-muted-foreground">
CUDA backend is downloaded and ready. Restart the server to enable GPU
acceleration.
</p>
<Button onClick={handleRestart} className="w-full" size="sm">
<RotateCw className="h-4 w-4 mr-2" />
Switch to CUDA Backend
</Button>
</div>
)}
{/* Delete option when downloaded (and not active) */}
{cudaAvailable && (
<Button
onClick={handleDelete}
variant="ghost"
className="w-full text-muted-foreground "
size="sm"
>
<Trash2 className="h-4 w-4 mr-2" />
Remove CUDA Backend
</Button>
)}
</div>
)}
</>
)}
</CardContent>
@@ -138,7 +138,6 @@ export function CapturesPage() {
const allowAutoPaste = settings?.allow_auto_paste ?? true;
const defaultVoiceId = settings?.default_playback_voice_id ?? null;
const hotkeyEnabled = settings?.hotkey_enabled ?? false;
const keepMicWarm = settings?.keep_mic_warm ?? false;
const pushToTalkKeys = settings?.chord_push_to_talk_keys ?? defaultChordKeys('push');
const toggleToTalkKeys = settings?.chord_toggle_to_talk_keys ?? defaultChordKeys('toggle');
@@ -222,22 +221,6 @@ export function CapturesPage() {
<InputMonitoringNotice enabled={hotkeyEnabled} />
</div>
<SettingRow
title={t('settings.captures.dictation.keepMicWarm.title')}
description={t('settings.captures.dictation.keepMicWarm.description')}
htmlFor="keepMicWarm"
action={
<Toggle
id="keepMicWarm"
checked={keepMicWarm}
disabled={!hotkeyEnabled}
onCheckedChange={(v) => {
update({ keep_mic_warm: v });
}}
/>
}
/>
<SettingRow
title={t('settings.captures.dictation.pushToTalk.title')}
description={t('settings.captures.dictation.pushToTalk.description')}
+99 -316
View File
@@ -5,7 +5,7 @@ import { useTranslation } from 'react-i18next';
import { Button } from '@/components/ui/button';
import { Progress } from '@/components/ui/progress';
import { apiClient } from '@/lib/api/client';
import type { CudaDownloadProgress, RocmDownloadProgress, HealthResponse } from '@/lib/api/types';
import type { CudaDownloadProgress, HealthResponse } from '@/lib/api/types';
import { useServerHealth } from '@/lib/hooks/useServer';
import { usePlatform } from '@/platform/PlatformContext';
import { useServerStore } from '@/stores/serverStore';
@@ -50,10 +50,7 @@ function GpuInfoCard({ health }: { health: HealthResponse }) {
: null;
const gpuBackend = hasGpu ? health.gpu_type!.replace(/\s*\(.+\)$/, '') : null;
const isApple = gpuBackend === 'MPS' || gpuBackend === 'Metal';
const showBackendVariant =
health.backend_variant &&
health.backend_variant !== 'cpu' &&
health.backend_variant.toLowerCase() !== gpuBackend?.toLowerCase();
const showBackendVariant = health.backend_variant && health.backend_variant !== 'cpu';
return (
<div className="rounded-lg border border-border/60 p-4">
@@ -118,14 +115,10 @@ export function GpuPage() {
const [restartPhase, setRestartPhase] = useState<RestartPhase>('idle');
const [error, setError] = useState<string | null>(null);
const [cudaStreaming, setCudaStreaming] = useState(false);
const [rocmStreaming, setRocmStreaming] = useState(false);
const [downloadProgress, setDownloadProgress] = useState<CudaDownloadProgress | null>(null);
const [rocmDownloadProgress, setRocmDownloadProgress] = useState<RocmDownloadProgress | null>(
null,
);
const healthPollRef = useRef<ReturnType<typeof setInterval> | null>(null);
// Hold the latest `t` in a ref so the CUDA progress SSE effect below doesn't
// tear down and reconnect the EventSource every time the language changes.
const tRef = useRef(t);
useEffect(() => {
tRef.current = t;
@@ -143,27 +136,9 @@ export function GpuPage() {
enabled: !!health,
});
const {
data: rocmStatus,
isLoading: _rocmStatusLoading,
refetch: refetchRocmStatus,
} = useQuery({
queryKey: ['rocm-status', serverUrl],
queryFn: () => apiClient.getRocmStatus(),
refetchInterval: (query) => (query.state.status === 'pending' ? false : 10000),
retry: 1,
enabled: !!health,
});
const isCurrentlyCuda = health?.backend_variant === 'cuda';
const isCurrentlyRocm = health?.backend_variant === 'rocm';
const cudaAvailable = cudaStatus?.available ?? false;
const cudaDownloading = cudaStatus?.downloading ?? false;
const rocmAvailable = rocmStatus?.available ?? false;
const rocmDownloading = rocmStatus?.downloading ?? false;
// The ROCm backend only applies to AMD GPUs on Windows. Show the section when
// the backend detects applicable hardware, or it is already downloaded/active.
const supportsRocm = (health?.supports_rocm ?? false) || rocmAvailable || isCurrentlyRocm;
useEffect(() => {
return () => {
@@ -175,7 +150,7 @@ export function GpuPage() {
}, []);
useEffect(() => {
if ((!cudaDownloading && !cudaStreaming) || !serverUrl) return;
if (!cudaDownloading || !serverUrl) return;
const eventSource = new EventSource(`${serverUrl}/backend/cuda-progress`);
@@ -187,13 +162,11 @@ export function GpuPage() {
if (data.status === 'complete') {
eventSource.close();
setDownloadProgress(null);
setCudaStreaming(false);
refetchCudaStatus();
} else if (data.status === 'error') {
eventSource.close();
setError(data.error || tRef.current('settings.gpu.errors.downloadFailed'));
setDownloadProgress(null);
setCudaStreaming(false);
refetchCudaStatus();
}
} catch (e) {
@@ -203,50 +176,12 @@ export function GpuPage() {
eventSource.onerror = () => {
eventSource.close();
setCudaStreaming(false);
};
return () => {
eventSource.close();
};
}, [cudaDownloading, cudaStreaming, serverUrl, refetchCudaStatus]);
useEffect(() => {
if ((!rocmDownloading && !rocmStreaming) || !serverUrl) return;
const eventSource = new EventSource(`${serverUrl}/backend/rocm-progress`);
eventSource.onmessage = (event) => {
try {
const data = JSON.parse(event.data) as RocmDownloadProgress;
setRocmDownloadProgress(data);
if (data.status === 'complete') {
eventSource.close();
setRocmDownloadProgress(null);
setRocmStreaming(false);
refetchRocmStatus();
} else if (data.status === 'error') {
eventSource.close();
setError(data.error || tRef.current('settings.gpu.errors.downloadFailed'));
setRocmDownloadProgress(null);
setRocmStreaming(false);
refetchRocmStatus();
}
} catch (e) {
console.error('Error parsing ROCm progress event:', e);
}
};
eventSource.onerror = () => {
eventSource.close();
setRocmStreaming(false);
};
return () => {
eventSource.close();
};
}, [rocmDownloading, rocmStreaming, serverUrl, refetchRocmStatus]);
}, [cudaDownloading, serverUrl, refetchCudaStatus]);
const clearHealthPolling = useCallback(() => {
if (healthPollRef.current) {
@@ -289,11 +224,10 @@ export function GpuPage() {
[platform, startHealthPolling, clearHealthPolling],
);
const handleDownloadCuda = async () => {
const handleDownload = async () => {
setError(null);
try {
await apiClient.downloadCudaBackend();
setCudaStreaming(true);
refetchCudaStatus();
} catch (e: unknown) {
const msg = e instanceof Error ? e.message : t('settings.gpu.errors.downloadStart');
@@ -305,64 +239,28 @@ export function GpuPage() {
}
};
const handleDownloadRocm = async () => {
const handleRestart = async () => {
setError(null);
try {
await apiClient.downloadRocmBackend();
setRocmStreaming(true);
refetchRocmStatus();
await restartServerWithPolling(t('settings.gpu.errors.restartFailed'));
} catch (e: unknown) {
const msg = e instanceof Error ? e.message : t('settings.gpu.errors.downloadStart');
if (msg.includes('already downloaded')) {
refetchRocmStatus();
} else {
setError(msg);
}
setError(e instanceof Error ? e.message : t('settings.gpu.errors.restartFailed'));
}
};
const handleSwitchToCpu = async () => {
setError(null);
setRestartPhase('stopping');
try {
await platform.lifecycle.setBackendOverride('cpu');
await apiClient.deleteCudaBackend();
await restartServerWithPolling(t('settings.gpu.errors.switchCpu'));
} catch (e: unknown) {
setRestartPhase('idle');
setError(e instanceof Error ? e.message : t('settings.gpu.errors.switchCpu'));
refetchCudaStatus();
refetchRocmStatus();
}
};
const handleSwitchToCuda = async () => {
setError(null);
setRestartPhase('stopping');
try {
await platform.lifecycle.setBackendOverride('cuda');
await restartServerWithPolling(t('settings.gpu.errors.restartFailed'));
} catch (e: unknown) {
setRestartPhase('idle');
setError(e instanceof Error ? e.message : t('settings.gpu.errors.restartFailed'));
refetchCudaStatus();
}
};
const handleSwitchToRocm = async () => {
setError(null);
setRestartPhase('stopping');
try {
await platform.lifecycle.setBackendOverride('rocm');
await restartServerWithPolling(t('settings.gpu.errors.restartFailed'));
} catch (e: unknown) {
setRestartPhase('idle');
setError(e instanceof Error ? e.message : t('settings.gpu.errors.restartFailed'));
refetchRocmStatus();
}
};
const handleDeleteCuda = async () => {
const handleDelete = async () => {
setError(null);
try {
await apiClient.deleteCudaBackend();
@@ -372,16 +270,6 @@ export function GpuPage() {
}
};
const handleDeleteRocm = async () => {
setError(null);
try {
await apiClient.deleteRocmBackend();
refetchRocmStatus();
} catch (e: unknown) {
setError(e instanceof Error ? e.message : t('settings.gpu.errors.deleteRocm'));
}
};
const formatBytes = (bytes: number): string => {
if (bytes === 0) return '0 B';
const k = 1024;
@@ -395,7 +283,6 @@ export function GpuPage() {
const hasNativeGpu =
health.gpu_available &&
!isCurrentlyCuda &&
!isCurrentlyRocm &&
health.gpu_type &&
!health.gpu_type.includes('CUDA');
@@ -403,188 +290,33 @@ export function GpuPage() {
<div className="space-y-8 max-w-2xl">
<GpuInfoCard health={health} />
{!hasNativeGpu && !isCurrentlyCuda && !isCurrentlyRocm && (
<>
<SettingSection
title={t('settings.gpu.cuda.title')}
description={t('settings.gpu.cuda.description')}
>
{cudaDownloading && downloadProgress && (
<SettingRow title={t('settings.gpu.cuda.downloading')}>
<div className="space-y-1.5">
<Progress value={downloadProgress.progress} className="h-2" />
<div className="flex items-center justify-between text-xs text-muted-foreground">
<span>
{downloadProgress.filename ||
(cudaAvailable
? t('settings.gpu.cuda.updating')
: t('settings.gpu.cuda.downloadingShort'))}
</span>
<span>
{downloadProgress.total > 0
? `${formatBytes(downloadProgress.current)} / ${formatBytes(downloadProgress.total)}`
: `${downloadProgress.progress.toFixed(1)}%`}
</span>
</div>
</div>
</SettingRow>
)}
{restartPhase !== 'idle' && (
<SettingRow
title={
restartPhase === 'ready'
? t('settings.gpu.restart.ready')
: restartPhase === 'waiting'
? t('settings.gpu.restart.waiting')
: t('settings.gpu.restart.stopping')
}
action={<Loader2 className="h-4 w-4 animate-spin text-muted-foreground" />}
/>
)}
{error && (
<SettingRow title={t('common.error')}>
<div className="flex items-center gap-2 text-sm text-destructive">
<AlertCircle className="h-4 w-4 shrink-0" />
<span>{error}</span>
</div>
</SettingRow>
)}
{restartPhase === 'idle' && !cudaDownloading && (
<>
{!cudaAvailable && !isCurrentlyCuda && (
<SettingRow
title={t('settings.gpu.download.title')}
description={t('settings.gpu.download.description')}
action={
<Button onClick={handleDownloadCuda} size="sm">
<Download className="h-3.5 w-3.5 mr-1.5" />
{t('settings.gpu.download.button')}
</Button>
}
/>
)}
{cudaAvailable && !isCurrentlyCuda && platform.metadata.isTauri && (
<SettingRow
title={t('settings.gpu.switchToCuda.title')}
description={t('settings.gpu.switchToCuda.description')}
action={
<Button onClick={handleSwitchToCuda} size="sm">
<RotateCw className="h-3.5 w-3.5 mr-1.5" />
{t('settings.gpu.switchToCuda.button')}
</Button>
}
/>
)}
{cudaAvailable && !isCurrentlyCuda && (
<SettingRow
title={t('settings.gpu.remove.title')}
description={t('settings.gpu.remove.description')}
action={
<Button
onClick={handleDeleteCuda}
variant="ghost"
size="sm"
className="text-muted-foreground hover:text-destructive"
>
<Trash2 className="h-3.5 w-3.5 mr-1.5" />
{t('settings.gpu.remove.button')}
</Button>
}
/>
)}
</>
)}
</SettingSection>
{supportsRocm && (
<SettingSection
title={t('settings.gpu.rocm.title')}
description={t('settings.gpu.rocm.description')}
>
{rocmDownloading && rocmDownloadProgress && (
<SettingRow title={t('settings.gpu.rocm.downloading')}>
<div className="space-y-1.5">
<Progress value={rocmDownloadProgress.progress} className="h-2" />
<div className="flex items-center justify-between text-xs text-muted-foreground">
<span>
{rocmDownloadProgress.filename ||
(rocmAvailable
? t('settings.gpu.rocm.updating')
: t('settings.gpu.rocm.downloadingShort'))}
</span>
<span>
{rocmDownloadProgress.total > 0
? `${formatBytes(rocmDownloadProgress.current)} / ${formatBytes(rocmDownloadProgress.total)}`
: `${rocmDownloadProgress.progress.toFixed(1)}%`}
</span>
</div>
</div>
</SettingRow>
)}
{restartPhase === 'idle' && !rocmDownloading && (
<>
{!rocmAvailable && !isCurrentlyRocm && (
<SettingRow
title={t('settings.gpu.downloadRocm.title')}
description={t('settings.gpu.downloadRocm.description')}
action={
<Button onClick={handleDownloadRocm} size="sm">
<Download className="h-3.5 w-3.5 mr-1.5" />
{t('settings.gpu.downloadRocm.button')}
</Button>
}
/>
)}
{rocmAvailable && !isCurrentlyRocm && platform.metadata.isTauri && (
<SettingRow
title={t('settings.gpu.switchToRocm.title')}
description={t('settings.gpu.switchToRocm.description')}
action={
<Button onClick={handleSwitchToRocm} size="sm">
<RotateCw className="h-3.5 w-3.5 mr-1.5" />
{t('settings.gpu.switchToRocm.button')}
</Button>
}
/>
)}
{rocmAvailable && !isCurrentlyRocm && (
<SettingRow
title={t('settings.gpu.removeRocm.title')}
description={t('settings.gpu.removeRocm.description')}
action={
<Button
onClick={handleDeleteRocm}
variant="ghost"
size="sm"
className="text-muted-foreground hover:text-destructive"
>
<Trash2 className="h-3.5 w-3.5 mr-1.5" />
{t('settings.gpu.removeRocm.button')}
</Button>
}
/>
)}
</>
)}
</SettingSection>
)}
</>
)}
{(isCurrentlyCuda || isCurrentlyRocm) && platform.metadata.isTauri && (
{!hasNativeGpu && !isCurrentlyCuda && (
<SettingSection
title={isCurrentlyCuda ? t('settings.gpu.cuda.activeTitle') : t('settings.gpu.rocm.activeTitle')}
description={t('settings.gpu.activeBackend.description')}
title={t('settings.gpu.cuda.title')}
description={t('settings.gpu.cuda.description')}
>
{restartPhase !== 'idle' ? (
{cudaDownloading && downloadProgress && (
<SettingRow title={t('settings.gpu.cuda.downloading')}>
<div className="space-y-1.5">
<Progress value={downloadProgress.progress} className="h-2" />
<div className="flex items-center justify-between text-xs text-muted-foreground">
<span>
{downloadProgress.filename ||
(cudaAvailable
? t('settings.gpu.cuda.updating')
: t('settings.gpu.cuda.downloadingShort'))}
</span>
<span>
{downloadProgress.total > 0
? `${formatBytes(downloadProgress.current)} / ${formatBytes(downloadProgress.total)}`
: `${downloadProgress.progress.toFixed(1)}%`}
</span>
</div>
</div>
</SettingRow>
)}
{restartPhase !== 'idle' && (
<SettingRow
title={
restartPhase === 'ready'
@@ -595,18 +327,8 @@ export function GpuPage() {
}
action={<Loader2 className="h-4 w-4 animate-spin text-muted-foreground" />}
/>
) : (
<SettingRow
title={t('settings.gpu.switchToCpu.title')}
description={t('settings.gpu.switchToCpu.description')}
action={
<Button onClick={handleSwitchToCpu} variant="outline" size="sm">
<RotateCw className="h-3.5 w-3.5 mr-1.5" />
{t('settings.gpu.switchToCpu.button')}
</Button>
}
/>
)}
{error && (
<SettingRow title={t('common.error')}>
<div className="flex items-center gap-2 text-sm text-destructive">
@@ -615,6 +337,67 @@ export function GpuPage() {
</div>
</SettingRow>
)}
{restartPhase === 'idle' && !cudaDownloading && (
<>
{!cudaAvailable && !isCurrentlyCuda && (
<SettingRow
title={t('settings.gpu.download.title')}
description={t('settings.gpu.download.description')}
action={
<Button onClick={handleDownload} size="sm">
<Download className="h-3.5 w-3.5 mr-1.5" />
{t('settings.gpu.download.button')}
</Button>
}
/>
)}
{cudaAvailable && !isCurrentlyCuda && platform.metadata.isTauri && (
<SettingRow
title={t('settings.gpu.switchToCuda.title')}
description={t('settings.gpu.switchToCuda.description')}
action={
<Button onClick={handleRestart} size="sm">
<RotateCw className="h-3.5 w-3.5 mr-1.5" />
{t('settings.gpu.switchToCuda.button')}
</Button>
}
/>
)}
{isCurrentlyCuda && platform.metadata.isTauri && (
<SettingRow
title={t('settings.gpu.switchToCpu.title')}
description={t('settings.gpu.switchToCpu.description')}
action={
<Button onClick={handleSwitchToCpu} variant="outline" size="sm">
<RotateCw className="h-3.5 w-3.5 mr-1.5" />
{t('settings.gpu.switchToCpu.button')}
</Button>
}
/>
)}
{cudaAvailable && !isCurrentlyCuda && (
<SettingRow
title={t('settings.gpu.remove.title')}
description={t('settings.gpu.remove.description')}
action={
<Button
onClick={handleDelete}
variant="ghost"
size="sm"
className="text-muted-foreground "
>
<Trash2 className="h-3.5 w-3.5 mr-1.5" />
{t('settings.gpu.remove.button')}
</Button>
}
/>
)}
</>
)}
</SettingSection>
)}
@@ -38,7 +38,6 @@ import {
useUpdateStoryItemVolume,
} from '@/lib/hooks/useStories';
import { cn } from '@/lib/utils/cn';
import { computeTrimValues } from '@/lib/utils/trim';
import { useGenerationStore } from '@/stores/generationStore';
import { useStoryStore } from '@/stores/storyStore';
@@ -601,17 +600,41 @@ export function StoryTrackEditor({ storyId, items }: StoryTrackEditorProps) {
const deltaMs = pixelsToMs(deltaX); // Signed delta in milliseconds
const { item, initialTrimStart, initialTrimEnd } = trimStartItemRef.current;
const newTrimValues = computeTrimValues(
trimSide,
deltaMs,
initialTrimStart,
initialTrimEnd,
item.duration * 1000,
);
if (!newTrimValues) return;
const originalDurationMs = item.duration * 1000;
let newTrimStart = initialTrimStart;
let newTrimEnd = initialTrimEnd;
if (trimSide === 'start') {
// Moving right increases trim_start (trims more from start)
// Moving left decreases trim_start (restores from start)
newTrimStart = Math.round(
Math.max(
0,
Math.min(initialTrimStart + deltaMs, originalDurationMs - initialTrimEnd - 100),
),
);
} else {
// Moving right decreases trim_end (restores from end)
// Moving left increases trim_end (trims more from end)
newTrimEnd = Math.round(
Math.max(
0,
Math.min(initialTrimEnd - deltaMs, originalDurationMs - initialTrimStart - 100),
),
);
}
// Validate that we don't exceed duration
if (newTrimStart + newTrimEnd >= originalDurationMs - 100) {
return; // Don't allow trimming to less than 100ms
}
// Update temporary trim values for visual feedback
setTempTrimValues(newTrimValues);
setTempTrimValues({
trim_start_ms: newTrimStart,
trim_end_ms: newTrimEnd,
});
},
[trimmingItem, trimSide, trimStartX, pixelsToMs],
);
-6
View File
@@ -3,18 +3,14 @@ import LanguageDetector from 'i18next-browser-languagedetector';
import { initReactI18next } from 'react-i18next';
import en from './locales/en/translation.json';
import ja from './locales/ja/translation.json';
import ptBR from './locales/pt-BR/translation.json';
import zhCN from './locales/zh-CN/translation.json';
import zhTW from './locales/zh-TW/translation.json';
import fr from './locales/fr/translation.json';
export const SUPPORTED_LANGUAGES = [
{ code: 'en', label: 'English' },
{ code: 'pt-BR', label: 'Português (Brasil)' },
{ code: 'ja', label: '日本語' },
{ code: 'zh-CN', label: '简体中文' },
{ code: 'zh-TW', label: '繁體中文' },
{ code: 'fr', label: 'Français' },
] as const;
export type LanguageCode = (typeof SUPPORTED_LANGUAGES)[number]['code'];
@@ -25,11 +21,9 @@ i18n
.init({
resources: {
en: { translation: en },
'pt-BR': { translation: ptBR },
ja: { translation: ja },
'zh-CN': { translation: zhCN },
'zh-TW': { translation: zhTW },
fr: { translation: fr },
},
fallbackLng: 'en',
supportedLngs: SUPPORTED_LANGUAGES.map((l) => l.code),
+7 -43
View File
@@ -760,13 +760,8 @@
}
},
"general": {
"docs": {
"title": "Read the Docs"
},
"discord": {
"title": "Join the Discord",
"subtitle": "Get help & share voices"
},
"docs": { "title": "Read the Docs" },
"discord": { "title": "Join the Discord", "subtitle": "Get help & share voices" },
"serverUrl": {
"title": "Server URL",
"description": "The address of your voicebox backend server.",
@@ -887,10 +882,6 @@
"title": "Global shortcut",
"description": "Hold the shortcut to record from anywhere on your machine. Release to transcribe."
},
"keepMicWarm": {
"title": "Keep microphone ready",
"description": "Hold the microphone open while dictation is enabled so the first words are never clipped. The macOS microphone indicator stays lit while it's on."
},
"pushToTalk": {
"title": "Push-to-talk shortcut",
"description": "Hold these keys anywhere on your system to record. Release to stop and transcribe.",
@@ -1100,15 +1091,11 @@
"active": "Active",
"cuda": {
"title": "CUDA Backend",
"activeTitle": "CUDA Backend Active",
"description": "NVIDIA GPU acceleration via a downloadable CUDA backend.",
"downloading": "Downloading CUDA backend…",
"downloadingShort": "Downloading…",
"updating": "Updating…"
},
"activeBackend": {
"description": "GPU acceleration is currently enabled."
},
"restart": {
"ready": "Server restarted successfully",
"waiting": "Restarting server…",
@@ -1126,9 +1113,10 @@
},
"switchToCpu": {
"title": "Switch to CPU backend",
"description": "Disable GPU acceleration. You can re-download the GPU backend later.",
"description": "Disable GPU acceleration. You can re-download CUDA later.",
"button": "Switch"
}, "remove": {
},
"remove": {
"title": "Remove CUDA backend",
"description": "Delete the downloaded CUDA binary to free disk space.",
"button": "Remove"
@@ -1138,33 +1126,9 @@
"downloadStart": "Failed to start download",
"restartFailed": "Restart failed",
"switchCpu": "Failed to switch to CPU",
"deleteCuda": "Failed to delete CUDA backend",
"deleteRocm": "Failed to delete ROCm backend"
"deleteCuda": "Failed to delete CUDA backend"
},
"footer": "Voicebox automatically detects and uses the best available GPU on your system. On Apple Silicon Macs, the MLX backend runs natively on the Neural Engine and GPU via Metal Performance Shaders (MPS), with no additional setup required. On Windows, you can download optional CUDA (NVIDIA) or ROCm (AMD) backends for hardware-accelerated inference. Intel XPU and DirectML are also supported where available through PyTorch. When no GPU is detected, Voicebox falls back to CPU — all engines still work, just slower.",
"rocm": {
"title": "AMD ROCm Backend",
"activeTitle": "ROCm Backend Active",
"description": "AMD GPU acceleration via a downloadable ROCm backend.",
"downloading": "Downloading ROCm backend…",
"downloadingShort": "Downloading…",
"updating": "Updating…"
},
"downloadRocm": {
"title": "Download AMD ROCm backend",
"description": "~2-3 GB download. Requires an AMD Radeon GPU with ROCm support.",
"button": "Download"
},
"switchToRocm": {
"title": "Switch to ROCm backend",
"description": "ROCm backend is downloaded and ready. Restart to enable.",
"button": "Restart"
},
"removeRocm": {
"title": "Remove ROCm backend",
"description": "Delete the downloaded ROCm binary to free disk space.",
"button": "Remove"
}
"footer": "Voicebox automatically detects and uses the best available GPU on your system. On Apple Silicon Macs, the MLX backend runs natively on the Neural Engine and GPU via Metal Performance Shaders (MPS), with no additional setup required. On Windows and Linux with NVIDIA GPUs, you can download an optional CUDA backend for hardware-accelerated inference. AMD ROCm, Intel XPU, and DirectML are also supported where available through PyTorch. When no GPU is detected, Voicebox falls back to CPU — all engines still work, just slower."
},
"logs": {
"title": "Server Logs",
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
+15 -19
View File
@@ -1,5 +1,4 @@
import type { LanguageCode } from '@/lib/constants/languages';
import { formatErrorDetail } from '@/lib/api/errors';
import { useServerStore } from '@/stores/serverStore';
import type {
ActiveTasksResponse,
@@ -21,7 +20,6 @@ import type {
PresetVoice,
PersonalityTextResponse,
ProfileSampleResponse,
RocmStatus,
StoryCreate,
StoryDetailResponse,
StoryItemBatchUpdate,
@@ -56,6 +54,21 @@ import type {
CloudStatus,
} from './types';
function formatErrorDetail(detail: unknown, fallback: string): string {
if (typeof detail === 'string') return detail;
if (Array.isArray(detail)) {
return detail
.map((e: Record<string, unknown>) => e.msg || e.message || JSON.stringify(e))
.join('; ');
}
if (detail && typeof detail === 'object') {
const obj = detail as Record<string, unknown>;
if (typeof obj.message === 'string') return obj.message;
return JSON.stringify(detail);
}
return fallback;
}
class ApiClient {
private getBaseUrl(): string {
const serverUrl = useServerStore.getState().serverUrl;
@@ -682,23 +695,6 @@ class ApiClient {
});
}
// ROCm Backend Management
async getRocmStatus(): Promise<RocmStatus> {
return this.request<RocmStatus>('/backend/rocm-status');
}
async downloadRocmBackend(): Promise<{ message: string; progress_key: string }> {
return this.request<{ message: string; progress_key: string }>('/backend/download-rocm', {
method: 'POST',
});
}
async deleteRocmBackend(): Promise<{ message: string }> {
return this.request<{ message: string }>('/backend/rocm', {
method: 'DELETE',
});
}
// Stories
async listStories(): Promise<StoryResponse[]> {
return this.request<StoryResponse[]>('/stories');
+25
View File
@@ -0,0 +1,25 @@
/* generated using openapi-typescript-codegen -- do not edit */
/* istanbul ignore file */
/* tslint:disable */
/* eslint-disable */
import type { ApiRequestOptions } from './ApiRequestOptions';
import type { ApiResult } from './ApiResult';
export class ApiError extends Error {
public readonly url: string;
public readonly status: number;
public readonly statusText: string;
public readonly body: any;
public readonly request: ApiRequestOptions;
constructor(request: ApiRequestOptions, response: ApiResult, message: string) {
super(message);
this.name = 'ApiError';
this.url = response.url;
this.status = response.status;
this.statusText = response.statusText;
this.body = response.body;
this.request = request;
}
}
+17
View File
@@ -0,0 +1,17 @@
/* generated using openapi-typescript-codegen -- do not edit */
/* istanbul ignore file */
/* tslint:disable */
/* eslint-disable */
export type ApiRequestOptions = {
readonly method: 'GET' | 'PUT' | 'POST' | 'DELETE' | 'OPTIONS' | 'HEAD' | 'PATCH';
readonly url: string;
readonly path?: Record<string, any>;
readonly cookies?: Record<string, any>;
readonly headers?: Record<string, any>;
readonly query?: Record<string, any>;
readonly formData?: Record<string, any>;
readonly body?: any;
readonly mediaType?: string;
readonly responseHeader?: string;
readonly errors?: Record<number, string>;
};
+11
View File
@@ -0,0 +1,11 @@
/* generated using openapi-typescript-codegen -- do not edit */
/* istanbul ignore file */
/* tslint:disable */
/* eslint-disable */
export type ApiResult = {
readonly url: string;
readonly ok: boolean;
readonly status: number;
readonly statusText: string;
readonly body: any;
};
+130
View File
@@ -0,0 +1,130 @@
/* generated using openapi-typescript-codegen -- do not edit */
/* istanbul ignore file */
/* tslint:disable */
/* eslint-disable */
export class CancelError extends Error {
constructor(message: string) {
super(message);
this.name = 'CancelError';
}
public get isCancelled(): boolean {
return true;
}
}
export interface OnCancel {
readonly isResolved: boolean;
readonly isRejected: boolean;
readonly isCancelled: boolean;
(cancelHandler: () => void): void;
}
export class CancelablePromise<T> implements Promise<T> {
#isResolved: boolean;
#isRejected: boolean;
#isCancelled: boolean;
readonly #cancelHandlers: (() => void)[];
readonly #promise: Promise<T>;
#resolve?: (value: T | PromiseLike<T>) => void;
#reject?: (reason?: any) => void;
constructor(
executor: (
resolve: (value: T | PromiseLike<T>) => void,
reject: (reason?: any) => void,
onCancel: OnCancel,
) => void,
) {
this.#isResolved = false;
this.#isRejected = false;
this.#isCancelled = false;
this.#cancelHandlers = [];
this.#promise = new Promise<T>((resolve, reject) => {
this.#resolve = resolve;
this.#reject = reject;
const onResolve = (value: T | PromiseLike<T>): void => {
if (this.#isResolved || this.#isRejected || this.#isCancelled) {
return;
}
this.#isResolved = true;
if (this.#resolve) this.#resolve(value);
};
const onReject = (reason?: any): void => {
if (this.#isResolved || this.#isRejected || this.#isCancelled) {
return;
}
this.#isRejected = true;
if (this.#reject) this.#reject(reason);
};
const onCancel = (cancelHandler: () => void): void => {
if (this.#isResolved || this.#isRejected || this.#isCancelled) {
return;
}
this.#cancelHandlers.push(cancelHandler);
};
Object.defineProperty(onCancel, 'isResolved', {
get: (): boolean => this.#isResolved,
});
Object.defineProperty(onCancel, 'isRejected', {
get: (): boolean => this.#isRejected,
});
Object.defineProperty(onCancel, 'isCancelled', {
get: (): boolean => this.#isCancelled,
});
return executor(onResolve, onReject, onCancel as OnCancel);
});
}
get [Symbol.toStringTag]() {
return 'Cancellable Promise';
}
public then<TResult1 = T, TResult2 = never>(
onFulfilled?: ((value: T) => TResult1 | PromiseLike<TResult1>) | null,
onRejected?: ((reason: any) => TResult2 | PromiseLike<TResult2>) | null,
): Promise<TResult1 | TResult2> {
return this.#promise.then(onFulfilled, onRejected);
}
public catch<TResult = never>(
onRejected?: ((reason: any) => TResult | PromiseLike<TResult>) | null,
): Promise<T | TResult> {
return this.#promise.catch(onRejected);
}
public finally(onFinally?: (() => void) | null): Promise<T> {
return this.#promise.finally(onFinally);
}
public cancel(): void {
if (this.#isResolved || this.#isRejected || this.#isCancelled) {
return;
}
this.#isCancelled = true;
if (this.#cancelHandlers.length) {
try {
for (const cancelHandler of this.#cancelHandlers) {
cancelHandler();
}
} catch (error) {
console.warn('Cancellation threw an error', error);
return;
}
}
this.#cancelHandlers.length = 0;
if (this.#reject) this.#reject(new CancelError('Request aborted'));
}
public get isCancelled(): boolean {
return this.#isCancelled;
}
}
+32
View File
@@ -0,0 +1,32 @@
/* generated using openapi-typescript-codegen -- do not edit */
/* istanbul ignore file */
/* tslint:disable */
/* eslint-disable */
import type { ApiRequestOptions } from './ApiRequestOptions';
type Resolver<T> = (options: ApiRequestOptions) => Promise<T>;
type Headers = Record<string, string>;
export type OpenAPIConfig = {
BASE: string;
VERSION: string;
WITH_CREDENTIALS: boolean;
CREDENTIALS: 'include' | 'omit' | 'same-origin';
TOKEN?: string | Resolver<string> | undefined;
USERNAME?: string | Resolver<string> | undefined;
PASSWORD?: string | Resolver<string> | undefined;
HEADERS?: Headers | Resolver<Headers> | undefined;
ENCODE_PATH?: ((path: string) => string) | undefined;
};
export const OpenAPI: OpenAPIConfig = {
BASE: '',
VERSION: '0.1.0',
WITH_CREDENTIALS: false,
CREDENTIALS: 'include',
TOKEN: undefined,
USERNAME: undefined,
PASSWORD: undefined,
HEADERS: undefined,
ENCODE_PATH: undefined,
};
+341
View File
@@ -0,0 +1,341 @@
/* generated using openapi-typescript-codegen -- do not edit */
/* istanbul ignore file */
/* tslint:disable */
/* eslint-disable */
import { ApiError } from './ApiError';
import type { ApiRequestOptions } from './ApiRequestOptions';
import type { ApiResult } from './ApiResult';
import { CancelablePromise } from './CancelablePromise';
import type { OnCancel } from './CancelablePromise';
import type { OpenAPIConfig } from './OpenAPI';
export const isDefined = <T>(
value: T | null | undefined,
): value is Exclude<T, null | undefined> => {
return value !== undefined && value !== null;
};
export const isString = (value: any): value is string => {
return typeof value === 'string';
};
export const isStringWithValue = (value: any): value is string => {
return isString(value) && value !== '';
};
export const isBlob = (value: any): value is Blob => {
return (
typeof value === 'object' &&
typeof value.type === 'string' &&
typeof value.stream === 'function' &&
typeof value.arrayBuffer === 'function' &&
typeof value.constructor === 'function' &&
typeof value.constructor.name === 'string' &&
/^(Blob|File)$/.test(value.constructor.name) &&
/^(Blob|File)$/.test(value[Symbol.toStringTag])
);
};
export const isFormData = (value: any): value is FormData => {
return value instanceof FormData;
};
export const base64 = (str: string): string => {
try {
return btoa(str);
} catch (err) {
// @ts-ignore
return Buffer.from(str).toString('base64');
}
};
export const getQueryString = (params: Record<string, any>): string => {
const qs: string[] = [];
const append = (key: string, value: any) => {
qs.push(`${encodeURIComponent(key)}=${encodeURIComponent(String(value))}`);
};
const process = (key: string, value: any) => {
if (isDefined(value)) {
if (Array.isArray(value)) {
value.forEach((v) => {
process(key, v);
});
} else if (typeof value === 'object') {
Object.entries(value).forEach(([k, v]) => {
process(`${key}[${k}]`, v);
});
} else {
append(key, value);
}
}
};
Object.entries(params).forEach(([key, value]) => {
process(key, value);
});
if (qs.length > 0) {
return `?${qs.join('&')}`;
}
return '';
};
const getUrl = (config: OpenAPIConfig, options: ApiRequestOptions): string => {
const encoder = config.ENCODE_PATH || encodeURI;
const path = options.url
.replace('{api-version}', config.VERSION)
.replace(/{(.*?)}/g, (substring: string, group: string) => {
if (options.path?.hasOwnProperty(group)) {
return encoder(String(options.path[group]));
}
return substring;
});
const url = `${config.BASE}${path}`;
if (options.query) {
return `${url}${getQueryString(options.query)}`;
}
return url;
};
export const getFormData = (options: ApiRequestOptions): FormData | undefined => {
if (options.formData) {
const formData = new FormData();
const process = (key: string, value: any) => {
if (isString(value) || isBlob(value)) {
formData.append(key, value);
} else {
formData.append(key, JSON.stringify(value));
}
};
Object.entries(options.formData)
.filter(([_, value]) => isDefined(value))
.forEach(([key, value]) => {
if (Array.isArray(value)) {
value.forEach((v) => process(key, v));
} else {
process(key, value);
}
});
return formData;
}
return undefined;
};
type Resolver<T> = (options: ApiRequestOptions) => Promise<T>;
export const resolve = async <T>(
options: ApiRequestOptions,
resolver?: T | Resolver<T>,
): Promise<T | undefined> => {
if (typeof resolver === 'function') {
return (resolver as Resolver<T>)(options);
}
return resolver;
};
export const getHeaders = async (
config: OpenAPIConfig,
options: ApiRequestOptions,
): Promise<Headers> => {
const [token, username, password, additionalHeaders] = await Promise.all([
resolve(options, config.TOKEN),
resolve(options, config.USERNAME),
resolve(options, config.PASSWORD),
resolve(options, config.HEADERS),
]);
const headers = Object.entries({
Accept: 'application/json',
...additionalHeaders,
...options.headers,
})
.filter(([_, value]) => isDefined(value))
.reduce(
(headers, [key, value]) => ({
...headers,
[key]: String(value),
}),
{} as Record<string, string>,
);
if (isStringWithValue(token)) {
headers['Authorization'] = `Bearer ${token}`;
}
if (isStringWithValue(username) && isStringWithValue(password)) {
const credentials = base64(`${username}:${password}`);
headers['Authorization'] = `Basic ${credentials}`;
}
if (options.body !== undefined) {
if (options.mediaType) {
headers['Content-Type'] = options.mediaType;
} else if (isBlob(options.body)) {
headers['Content-Type'] = options.body.type || 'application/octet-stream';
} else if (isString(options.body)) {
headers['Content-Type'] = 'text/plain';
} else if (!isFormData(options.body)) {
headers['Content-Type'] = 'application/json';
}
}
return new Headers(headers);
};
export const getRequestBody = (options: ApiRequestOptions): any => {
if (options.body !== undefined) {
if (options.mediaType?.includes('/json')) {
return JSON.stringify(options.body);
} else if (isString(options.body) || isBlob(options.body) || isFormData(options.body)) {
return options.body;
} else {
return JSON.stringify(options.body);
}
}
return undefined;
};
export const sendRequest = async (
config: OpenAPIConfig,
options: ApiRequestOptions,
url: string,
body: any,
formData: FormData | undefined,
headers: Headers,
onCancel: OnCancel,
): Promise<Response> => {
const controller = new AbortController();
const request: RequestInit = {
headers,
body: body ?? formData,
method: options.method,
signal: controller.signal,
};
if (config.WITH_CREDENTIALS) {
request.credentials = config.CREDENTIALS;
}
onCancel(() => controller.abort());
return await fetch(url, request);
};
export const getResponseHeader = (
response: Response,
responseHeader?: string,
): string | undefined => {
if (responseHeader) {
const content = response.headers.get(responseHeader);
if (isString(content)) {
return content;
}
}
return undefined;
};
export const getResponseBody = async (response: Response): Promise<any> => {
if (response.status !== 204) {
try {
const contentType = response.headers.get('Content-Type');
if (contentType) {
const jsonTypes = ['application/json', 'application/problem+json'];
const isJSON = jsonTypes.some((type) => contentType.toLowerCase().startsWith(type));
if (isJSON) {
return await response.json();
} else {
return await response.text();
}
}
} catch (error) {
console.error(error);
}
}
return undefined;
};
export const catchErrorCodes = (options: ApiRequestOptions, result: ApiResult): void => {
const errors: Record<number, string> = {
400: 'Bad Request',
401: 'Unauthorized',
403: 'Forbidden',
404: 'Not Found',
500: 'Internal Server Error',
502: 'Bad Gateway',
503: 'Service Unavailable',
...options.errors,
};
const error = errors[result.status];
if (error) {
throw new ApiError(options, result, error);
}
if (!result.ok) {
const errorStatus = result.status ?? 'unknown';
const errorStatusText = result.statusText ?? 'unknown';
const errorBody = (() => {
try {
return JSON.stringify(result.body, null, 2);
} catch (e) {
return undefined;
}
})();
throw new ApiError(
options,
result,
`Generic Error: status: ${errorStatus}; status text: ${errorStatusText}; body: ${errorBody}`,
);
}
};
/**
* Request method
* @param config The OpenAPI configuration object
* @param options The request options from the service
* @returns CancelablePromise<T>
* @throws ApiError
*/
export const request = <T>(
config: OpenAPIConfig,
options: ApiRequestOptions,
): CancelablePromise<T> => {
return new CancelablePromise(async (resolve, reject, onCancel) => {
try {
const url = getUrl(config, options);
const formData = getFormData(options);
const body = getRequestBody(options);
const headers = await getHeaders(config, options);
if (!onCancel.isCancelled) {
const response = await sendRequest(config, options, url, body, formData, headers, onCancel);
const responseBody = await getResponseBody(response);
const responseHeader = getResponseHeader(response, options.responseHeader);
const result: ApiResult = {
url,
ok: response.ok,
status: response.status,
statusText: response.statusText,
body: responseHeader ?? responseBody,
};
catchErrorCodes(options, result);
resolve(result.body);
}
} catch (error) {
reject(error);
}
});
};
-58
View File
@@ -1,58 +0,0 @@
import { describe, expect, test } from 'bun:test';
import { formatErrorDetail } from './errors';
const FALLBACK = 'HTTP error! status: 500';
describe('formatErrorDetail', () => {
test('returns string details as-is', () => {
expect(formatErrorDetail('Profile not found', FALLBACK)).toBe('Profile not found');
});
test('returns empty string details as-is (not the fallback)', () => {
expect(formatErrorDetail('', FALLBACK)).toBe('');
});
test('joins FastAPI validation error arrays on msg', () => {
const detail = [
{ loc: ['body', 'text'], msg: 'field required', type: 'value_error.missing' },
{ loc: ['body', 'seed'], msg: 'value is not a valid integer', type: 'type_error.integer' },
];
expect(formatErrorDetail(detail, FALLBACK)).toBe(
'field required; value is not a valid integer',
);
});
test('falls back to message key within array entries', () => {
expect(formatErrorDetail([{ message: 'boom' }], FALLBACK)).toBe('boom');
});
test('stringifies array entries with neither msg nor message', () => {
expect(formatErrorDetail([{ code: 42 }], FALLBACK)).toBe('{"code":42}');
});
test('returns empty string for an empty array', () => {
expect(formatErrorDetail([], FALLBACK)).toBe('');
});
test('uses message property of object details', () => {
expect(formatErrorDetail({ message: 'engine offline' }, FALLBACK)).toBe('engine offline');
});
test('stringifies objects without a string message', () => {
expect(formatErrorDetail({ message: 42, hint: 'x' }, FALLBACK)).toBe(
'{"message":42,"hint":"x"}',
);
expect(formatErrorDetail({ error: 'nested' }, FALLBACK)).toBe('{"error":"nested"}');
});
test('falls back for null, undefined, and primitives', () => {
expect(formatErrorDetail(null, FALLBACK)).toBe(FALLBACK);
expect(formatErrorDetail(undefined, FALLBACK)).toBe(FALLBACK);
expect(formatErrorDetail(404, FALLBACK)).toBe(FALLBACK);
expect(formatErrorDetail(true, FALLBACK)).toBe(FALLBACK);
});
test('preserves unicode in messages', () => {
expect(formatErrorDetail('模型未加载 🎙️', FALLBACK)).toBe('模型未加载 🎙️');
});
});
-21
View File
@@ -1,21 +0,0 @@
/**
* Normalizes a FastAPI error `detail` payload into a human-readable message.
*
* FastAPI returns `detail` as a plain string for HTTPException, an array of
* validation error objects for 422 responses, or an arbitrary object for
* custom handlers. Anything unrecognized falls back to the provided default.
*/
export function formatErrorDetail(detail: unknown, fallback: string): string {
if (typeof detail === 'string') return detail;
if (Array.isArray(detail)) {
return detail
.map((e: Record<string, unknown>) => e.msg || e.message || JSON.stringify(e))
.join('; ');
}
if (detail && typeof detail === 'object') {
const obj = detail as Record<string, unknown>;
if (typeof obj.message === 'string') return obj.message;
return JSON.stringify(detail);
}
return fallback;
}
+44
View File
@@ -0,0 +1,44 @@
/* generated using openapi-typescript-codegen -- do not edit */
/* istanbul ignore file */
/* tslint:disable */
/* eslint-disable */
export { ApiError } from './core/ApiError';
export { CancelablePromise, CancelError } from './core/CancelablePromise';
export { OpenAPI } from './core/OpenAPI';
export type { OpenAPIConfig } from './core/OpenAPI';
export type { Body_add_profile_sample_profiles__profile_id__samples_post } from './models/Body_add_profile_sample_profiles__profile_id__samples_post';
export type { Body_transcribe_audio_transcribe_post } from './models/Body_transcribe_audio_transcribe_post';
export type { GenerationRequest } from './models/GenerationRequest';
export type { GenerationResponse } from './models/GenerationResponse';
export type { HealthResponse } from './models/HealthResponse';
export type { HistoryListResponse } from './models/HistoryListResponse';
export type { HistoryResponse } from './models/HistoryResponse';
export type { HTTPValidationError } from './models/HTTPValidationError';
export type { ModelDownloadRequest } from './models/ModelDownloadRequest';
export type { ModelStatus } from './models/ModelStatus';
export type { ModelStatusListResponse } from './models/ModelStatusListResponse';
export type { ProfileSampleResponse } from './models/ProfileSampleResponse';
export type { TranscriptionResponse } from './models/TranscriptionResponse';
export type { ValidationError } from './models/ValidationError';
export type { VoiceProfileCreate } from './models/VoiceProfileCreate';
export type { VoiceProfileResponse } from './models/VoiceProfileResponse';
export { $Body_add_profile_sample_profiles__profile_id__samples_post } from './schemas/$Body_add_profile_sample_profiles__profile_id__samples_post';
export { $Body_transcribe_audio_transcribe_post } from './schemas/$Body_transcribe_audio_transcribe_post';
export { $GenerationRequest } from './schemas/$GenerationRequest';
export { $GenerationResponse } from './schemas/$GenerationResponse';
export { $HealthResponse } from './schemas/$HealthResponse';
export { $HistoryListResponse } from './schemas/$HistoryListResponse';
export { $HistoryResponse } from './schemas/$HistoryResponse';
export { $HTTPValidationError } from './schemas/$HTTPValidationError';
export { $ModelDownloadRequest } from './schemas/$ModelDownloadRequest';
export { $ModelStatus } from './schemas/$ModelStatus';
export { $ModelStatusListResponse } from './schemas/$ModelStatusListResponse';
export { $ProfileSampleResponse } from './schemas/$ProfileSampleResponse';
export { $TranscriptionResponse } from './schemas/$TranscriptionResponse';
export { $ValidationError } from './schemas/$ValidationError';
export { $VoiceProfileCreate } from './schemas/$VoiceProfileCreate';
export { $VoiceProfileResponse } from './schemas/$VoiceProfileResponse';
export { DefaultService } from './services/DefaultService';
@@ -0,0 +1,8 @@
/* generated using openapi-typescript-codegen -- do not edit */
/* istanbul ignore file */
/* tslint:disable */
/* eslint-disable */
export type Body_add_profile_sample_profiles__profile_id__samples_post = {
file: Blob;
reference_text: string;
};
@@ -0,0 +1,8 @@
/* generated using openapi-typescript-codegen -- do not edit */
/* istanbul ignore file */
/* tslint:disable */
/* eslint-disable */
export type Body_transcribe_audio_transcribe_post = {
file: Blob;
language?: string | null;
};
@@ -0,0 +1,15 @@
/* generated using openapi-typescript-codegen -- do not edit */
/* istanbul ignore file */
/* tslint:disable */
/* eslint-disable */
/**
* Request model for voice generation.
*/
export type GenerationRequest = {
profile_id: string;
text: string;
language?: string;
seed?: number | null;
model_size?: string | null;
instruct?: string | null;
};
@@ -0,0 +1,18 @@
/* generated using openapi-typescript-codegen -- do not edit */
/* istanbul ignore file */
/* tslint:disable */
/* eslint-disable */
/**
* Response model for voice generation.
*/
export type GenerationResponse = {
id: string;
profile_id: string;
text: string;
language: string;
audio_path: string;
duration: number;
seed: number | null;
instruct: string | null;
created_at: string;
};
@@ -0,0 +1,8 @@
/* generated using openapi-typescript-codegen -- do not edit */
/* istanbul ignore file */
/* tslint:disable */
/* eslint-disable */
import type { ValidationError } from './ValidationError';
export type HTTPValidationError = {
detail?: Array<ValidationError>;
};
+15
View File
@@ -0,0 +1,15 @@
/* generated using openapi-typescript-codegen -- do not edit */
/* istanbul ignore file */
/* tslint:disable */
/* eslint-disable */
/**
* Response model for health check.
*/
export type HealthResponse = {
status: string;
model_loaded: boolean;
model_downloaded?: boolean | null;
model_size?: string | null;
gpu_available: boolean;
vram_used_mb?: number | null;
};
@@ -0,0 +1,12 @@
/* generated using openapi-typescript-codegen -- do not edit */
/* istanbul ignore file */
/* tslint:disable */
/* eslint-disable */
import type { HistoryResponse } from './HistoryResponse';
/**
* Response model for history list.
*/
export type HistoryListResponse = {
items: Array<HistoryResponse>;
total: number;
};
+19
View File
@@ -0,0 +1,19 @@
/* generated using openapi-typescript-codegen -- do not edit */
/* istanbul ignore file */
/* tslint:disable */
/* eslint-disable */
/**
* Response model for history entry (includes profile name).
*/
export type HistoryResponse = {
id: string;
profile_id: string;
profile_name: string;
text: string;
language: string;
audio_path: string;
duration: number;
seed: number | null;
instruct: string | null;
created_at: string;
};
@@ -0,0 +1,10 @@
/* generated using openapi-typescript-codegen -- do not edit */
/* istanbul ignore file */
/* tslint:disable */
/* eslint-disable */
/**
* Request model for triggering model download.
*/
export type ModelDownloadRequest = {
model_name: string;
};
+15
View File
@@ -0,0 +1,15 @@
/* generated using openapi-typescript-codegen -- do not edit */
/* istanbul ignore file */
/* tslint:disable */
/* eslint-disable */
/**
* Response model for model status.
*/
export type ModelStatus = {
model_name: string;
display_name: string;
downloaded: boolean;
downloading?: boolean; // True if download is in progress
size_mb?: number | null;
loaded?: boolean;
};
@@ -0,0 +1,11 @@
/* generated using openapi-typescript-codegen -- do not edit */
/* istanbul ignore file */
/* tslint:disable */
/* eslint-disable */
import type { ModelStatus } from './ModelStatus';
/**
* Response model for model status list.
*/
export type ModelStatusListResponse = {
models: Array<ModelStatus>;
};
@@ -0,0 +1,13 @@
/* generated using openapi-typescript-codegen -- do not edit */
/* istanbul ignore file */
/* tslint:disable */
/* eslint-disable */
/**
* Response model for profile sample.
*/
export type ProfileSampleResponse = {
id: string;
profile_id: string;
audio_path: string;
reference_text: string;
};
@@ -0,0 +1,11 @@
/* generated using openapi-typescript-codegen -- do not edit */
/* istanbul ignore file */
/* tslint:disable */
/* eslint-disable */
/**
* Response model for transcription.
*/
export type TranscriptionResponse = {
text: string;
duration: number;
};
@@ -0,0 +1,9 @@
/* generated using openapi-typescript-codegen -- do not edit */
/* istanbul ignore file */
/* tslint:disable */
/* eslint-disable */
export type ValidationError = {
loc: Array<string | number>;
msg: string;
type: string;
};
@@ -0,0 +1,12 @@
/* generated using openapi-typescript-codegen -- do not edit */
/* istanbul ignore file */
/* tslint:disable */
/* eslint-disable */
/**
* Request model for creating a voice profile.
*/
export type VoiceProfileCreate = {
name: string;
description?: string | null;
language?: string;
};
@@ -0,0 +1,15 @@
/* generated using openapi-typescript-codegen -- do not edit */
/* istanbul ignore file */
/* tslint:disable */
/* eslint-disable */
/**
* Response model for voice profile.
*/
export type VoiceProfileResponse = {
id: string;
name: string;
description: string | null;
language: string;
created_at: string;
updated_at: string;
};
@@ -0,0 +1,17 @@
/* generated using openapi-typescript-codegen -- do not edit */
/* istanbul ignore file */
/* tslint:disable */
/* eslint-disable */
export const $Body_add_profile_sample_profiles__profile_id__samples_post = {
properties: {
file: {
type: 'binary',
isRequired: true,
format: 'binary',
},
reference_text: {
type: 'string',
isRequired: true,
},
},
} as const;
@@ -0,0 +1,24 @@
/* generated using openapi-typescript-codegen -- do not edit */
/* istanbul ignore file */
/* tslint:disable */
/* eslint-disable */
export const $Body_transcribe_audio_transcribe_post = {
properties: {
file: {
type: 'binary',
isRequired: true,
format: 'binary',
},
language: {
type: 'any-of',
contains: [
{
type: 'string',
},
{
type: 'null',
},
],
},
},
} as const;
@@ -0,0 +1,46 @@
/* generated using openapi-typescript-codegen -- do not edit */
/* istanbul ignore file */
/* tslint:disable */
/* eslint-disable */
export const $GenerationRequest = {
description: `Request model for voice generation.`,
properties: {
profile_id: {
type: 'string',
isRequired: true,
},
text: {
type: 'string',
isRequired: true,
maxLength: 5000,
minLength: 1,
},
language: {
type: 'string',
pattern: '^(en|zh)$',
},
seed: {
type: 'any-of',
contains: [
{
type: 'number',
},
{
type: 'null',
},
],
},
model_size: {
type: 'any-of',
contains: [
{
type: 'string',
pattern: '^(1\\.7B|0\\.6B)$',
},
{
type: 'null',
},
],
},
},
} as const;
@@ -0,0 +1,50 @@
/* generated using openapi-typescript-codegen -- do not edit */
/* istanbul ignore file */
/* tslint:disable */
/* eslint-disable */
export const $GenerationResponse = {
description: `Response model for voice generation.`,
properties: {
id: {
type: 'string',
isRequired: true,
},
profile_id: {
type: 'string',
isRequired: true,
},
text: {
type: 'string',
isRequired: true,
},
language: {
type: 'string',
isRequired: true,
},
audio_path: {
type: 'string',
isRequired: true,
},
duration: {
type: 'number',
isRequired: true,
},
seed: {
type: 'any-of',
contains: [
{
type: 'number',
},
{
type: 'null',
},
],
isRequired: true,
},
created_at: {
type: 'string',
isRequired: true,
format: 'date-time',
},
},
} as const;
@@ -0,0 +1,14 @@
/* generated using openapi-typescript-codegen -- do not edit */
/* istanbul ignore file */
/* tslint:disable */
/* eslint-disable */
export const $HTTPValidationError = {
properties: {
detail: {
type: 'array',
contains: {
type: 'ValidationError',
},
},
},
} as const;
@@ -0,0 +1,54 @@
/* generated using openapi-typescript-codegen -- do not edit */
/* istanbul ignore file */
/* tslint:disable */
/* eslint-disable */
export const $HealthResponse = {
description: `Response model for health check.`,
properties: {
status: {
type: 'string',
isRequired: true,
},
model_loaded: {
type: 'boolean',
isRequired: true,
},
model_downloaded: {
type: 'any-of',
contains: [
{
type: 'boolean',
},
{
type: 'null',
},
],
},
model_size: {
type: 'any-of',
contains: [
{
type: 'string',
},
{
type: 'null',
},
],
},
gpu_available: {
type: 'boolean',
isRequired: true,
},
vram_used_mb: {
type: 'any-of',
contains: [
{
type: 'number',
},
{
type: 'null',
},
],
},
},
} as const;
@@ -0,0 +1,20 @@
/* generated using openapi-typescript-codegen -- do not edit */
/* istanbul ignore file */
/* tslint:disable */
/* eslint-disable */
export const $HistoryListResponse = {
description: `Response model for history list.`,
properties: {
items: {
type: 'array',
contains: {
type: 'HistoryResponse',
},
isRequired: true,
},
total: {
type: 'number',
isRequired: true,
},
},
} as const;
@@ -0,0 +1,54 @@
/* generated using openapi-typescript-codegen -- do not edit */
/* istanbul ignore file */
/* tslint:disable */
/* eslint-disable */
export const $HistoryResponse = {
description: `Response model for history entry (includes profile name).`,
properties: {
id: {
type: 'string',
isRequired: true,
},
profile_id: {
type: 'string',
isRequired: true,
},
profile_name: {
type: 'string',
isRequired: true,
},
text: {
type: 'string',
isRequired: true,
},
language: {
type: 'string',
isRequired: true,
},
audio_path: {
type: 'string',
isRequired: true,
},
duration: {
type: 'number',
isRequired: true,
},
seed: {
type: 'any-of',
contains: [
{
type: 'number',
},
{
type: 'null',
},
],
isRequired: true,
},
created_at: {
type: 'string',
isRequired: true,
format: 'date-time',
},
},
} as const;
@@ -0,0 +1,13 @@
/* generated using openapi-typescript-codegen -- do not edit */
/* istanbul ignore file */
/* tslint:disable */
/* eslint-disable */
export const $ModelDownloadRequest = {
description: `Request model for triggering model download.`,
properties: {
model_name: {
type: 'string',
isRequired: true,
},
},
} as const;
+35
View File
@@ -0,0 +1,35 @@
/* generated using openapi-typescript-codegen -- do not edit */
/* istanbul ignore file */
/* tslint:disable */
/* eslint-disable */
export const $ModelStatus = {
description: `Response model for model status.`,
properties: {
model_name: {
type: 'string',
isRequired: true,
},
display_name: {
type: 'string',
isRequired: true,
},
downloaded: {
type: 'boolean',
isRequired: true,
},
size_mb: {
type: 'any-of',
contains: [
{
type: 'number',
},
{
type: 'null',
},
],
},
loaded: {
type: 'boolean',
},
},
} as const;
@@ -0,0 +1,16 @@
/* generated using openapi-typescript-codegen -- do not edit */
/* istanbul ignore file */
/* tslint:disable */
/* eslint-disable */
export const $ModelStatusListResponse = {
description: `Response model for model status list.`,
properties: {
models: {
type: 'array',
contains: {
type: 'ModelStatus',
},
isRequired: true,
},
},
} as const;
@@ -0,0 +1,25 @@
/* generated using openapi-typescript-codegen -- do not edit */
/* istanbul ignore file */
/* tslint:disable */
/* eslint-disable */
export const $ProfileSampleResponse = {
description: `Response model for profile sample.`,
properties: {
id: {
type: 'string',
isRequired: true,
},
profile_id: {
type: 'string',
isRequired: true,
},
audio_path: {
type: 'string',
isRequired: true,
},
reference_text: {
type: 'string',
isRequired: true,
},
},
} as const;
@@ -0,0 +1,17 @@
/* generated using openapi-typescript-codegen -- do not edit */
/* istanbul ignore file */
/* tslint:disable */
/* eslint-disable */
export const $TranscriptionResponse = {
description: `Response model for transcription.`,
properties: {
text: {
type: 'string',
isRequired: true,
},
duration: {
type: 'number',
isRequired: true,
},
},
} as const;
@@ -0,0 +1,31 @@
/* generated using openapi-typescript-codegen -- do not edit */
/* istanbul ignore file */
/* tslint:disable */
/* eslint-disable */
export const $ValidationError = {
properties: {
loc: {
type: 'array',
contains: {
type: 'any-of',
contains: [
{
type: 'string',
},
{
type: 'number',
},
],
},
isRequired: true,
},
msg: {
type: 'string',
isRequired: true,
},
type: {
type: 'string',
isRequired: true,
},
},
} as const;
@@ -0,0 +1,31 @@
/* generated using openapi-typescript-codegen -- do not edit */
/* istanbul ignore file */
/* tslint:disable */
/* eslint-disable */
export const $VoiceProfileCreate = {
description: `Request model for creating a voice profile.`,
properties: {
name: {
type: 'string',
isRequired: true,
maxLength: 100,
minLength: 1,
},
description: {
type: 'any-of',
contains: [
{
type: 'string',
maxLength: 500,
},
{
type: 'null',
},
],
},
language: {
type: 'string',
pattern: '^(en|zh)$',
},
},
} as const;
@@ -0,0 +1,43 @@
/* generated using openapi-typescript-codegen -- do not edit */
/* istanbul ignore file */
/* tslint:disable */
/* eslint-disable */
export const $VoiceProfileResponse = {
description: `Response model for voice profile.`,
properties: {
id: {
type: 'string',
isRequired: true,
},
name: {
type: 'string',
isRequired: true,
},
description: {
type: 'any-of',
contains: [
{
type: 'string',
},
{
type: 'null',
},
],
isRequired: true,
},
language: {
type: 'string',
isRequired: true,
},
created_at: {
type: 'string',
isRequired: true,
format: 'date-time',
},
updated_at: {
type: 'string',
isRequired: true,
format: 'date-time',
},
},
} as const;
+459
View File
@@ -0,0 +1,459 @@
/* generated using openapi-typescript-codegen -- do not edit */
/* istanbul ignore file */
/* tslint:disable */
/* eslint-disable */
import type { Body_add_profile_sample_profiles__profile_id__samples_post } from '../models/Body_add_profile_sample_profiles__profile_id__samples_post';
import type { Body_transcribe_audio_transcribe_post } from '../models/Body_transcribe_audio_transcribe_post';
import type { GenerationRequest } from '../models/GenerationRequest';
import type { GenerationResponse } from '../models/GenerationResponse';
import type { HealthResponse } from '../models/HealthResponse';
import type { HistoryListResponse } from '../models/HistoryListResponse';
import type { HistoryResponse } from '../models/HistoryResponse';
import type { ModelDownloadRequest } from '../models/ModelDownloadRequest';
import type { ModelStatusListResponse } from '../models/ModelStatusListResponse';
import type { ProfileSampleResponse } from '../models/ProfileSampleResponse';
import type { TranscriptionResponse } from '../models/TranscriptionResponse';
import type { VoiceProfileCreate } from '../models/VoiceProfileCreate';
import type { VoiceProfileResponse } from '../models/VoiceProfileResponse';
import type { CancelablePromise } from '../core/CancelablePromise';
import { OpenAPI } from '../core/OpenAPI';
import { request as __request } from '../core/request';
export class DefaultService {
/**
* Root
* Root endpoint.
* @returns any Successful Response
* @throws ApiError
*/
public static rootGet(): CancelablePromise<any> {
return __request(OpenAPI, {
method: 'GET',
url: '/',
});
}
/**
* Health
* Health check endpoint.
* @returns HealthResponse Successful Response
* @throws ApiError
*/
public static healthHealthGet(): CancelablePromise<HealthResponse> {
return __request(OpenAPI, {
method: 'GET',
url: '/health',
});
}
/**
* List Profiles
* List all voice profiles.
* @returns VoiceProfileResponse Successful Response
* @throws ApiError
*/
public static listProfilesProfilesGet(): CancelablePromise<Array<VoiceProfileResponse>> {
return __request(OpenAPI, {
method: 'GET',
url: '/profiles',
});
}
/**
* Create Profile
* Create a new voice profile.
* @returns VoiceProfileResponse Successful Response
* @throws ApiError
*/
public static createProfileProfilesPost({
requestBody,
}: {
requestBody: VoiceProfileCreate;
}): CancelablePromise<VoiceProfileResponse> {
return __request(OpenAPI, {
method: 'POST',
url: '/profiles',
body: requestBody,
mediaType: 'application/json',
errors: {
422: `Validation Error`,
},
});
}
/**
* Get Profile
* Get a voice profile by ID.
* @returns VoiceProfileResponse Successful Response
* @throws ApiError
*/
public static getProfileProfilesProfileIdGet({
profileId,
}: {
profileId: string;
}): CancelablePromise<VoiceProfileResponse> {
return __request(OpenAPI, {
method: 'GET',
url: '/profiles/{profile_id}',
path: {
profile_id: profileId,
},
errors: {
422: `Validation Error`,
},
});
}
/**
* Update Profile
* Update a voice profile.
* @returns VoiceProfileResponse Successful Response
* @throws ApiError
*/
public static updateProfileProfilesProfileIdPut({
profileId,
requestBody,
}: {
profileId: string;
requestBody: VoiceProfileCreate;
}): CancelablePromise<VoiceProfileResponse> {
return __request(OpenAPI, {
method: 'PUT',
url: '/profiles/{profile_id}',
path: {
profile_id: profileId,
},
body: requestBody,
mediaType: 'application/json',
errors: {
422: `Validation Error`,
},
});
}
/**
* Delete Profile
* Delete a voice profile.
* @returns any Successful Response
* @throws ApiError
*/
public static deleteProfileProfilesProfileIdDelete({
profileId,
}: {
profileId: string;
}): CancelablePromise<any> {
return __request(OpenAPI, {
method: 'DELETE',
url: '/profiles/{profile_id}',
path: {
profile_id: profileId,
},
errors: {
422: `Validation Error`,
},
});
}
/**
* Add Profile Sample
* Add a sample to a voice profile.
* @returns ProfileSampleResponse Successful Response
* @throws ApiError
*/
public static addProfileSampleProfilesProfileIdSamplesPost({
profileId,
formData,
}: {
profileId: string;
formData: Body_add_profile_sample_profiles__profile_id__samples_post;
}): CancelablePromise<ProfileSampleResponse> {
return __request(OpenAPI, {
method: 'POST',
url: '/profiles/{profile_id}/samples',
path: {
profile_id: profileId,
},
formData: formData,
mediaType: 'multipart/form-data',
errors: {
422: `Validation Error`,
},
});
}
/**
* Get Profile Samples
* Get all samples for a profile.
* @returns ProfileSampleResponse Successful Response
* @throws ApiError
*/
public static getProfileSamplesProfilesProfileIdSamplesGet({
profileId,
}: {
profileId: string;
}): CancelablePromise<Array<ProfileSampleResponse>> {
return __request(OpenAPI, {
method: 'GET',
url: '/profiles/{profile_id}/samples',
path: {
profile_id: profileId,
},
errors: {
422: `Validation Error`,
},
});
}
/**
* Delete Profile Sample
* Delete a profile sample.
* @returns any Successful Response
* @throws ApiError
*/
public static deleteProfileSampleProfilesSamplesSampleIdDelete({
sampleId,
}: {
sampleId: string;
}): CancelablePromise<any> {
return __request(OpenAPI, {
method: 'DELETE',
url: '/profiles/samples/{sample_id}',
path: {
sample_id: sampleId,
},
errors: {
422: `Validation Error`,
},
});
}
/**
* Generate Speech
* Generate speech from text using a voice profile.
* @returns GenerationResponse Successful Response
* @throws ApiError
*/
public static generateSpeechGeneratePost({
requestBody,
}: {
requestBody: GenerationRequest;
}): CancelablePromise<GenerationResponse> {
return __request(OpenAPI, {
method: 'POST',
url: '/generate',
body: requestBody,
mediaType: 'application/json',
errors: {
422: `Validation Error`,
},
});
}
/**
* List History
* List generation history with optional filters.
* @returns HistoryListResponse Successful Response
* @throws ApiError
*/
public static listHistoryHistoryGet({
profileId,
search,
limit = 50,
offset,
}: {
profileId?: string | null;
search?: string | null;
limit?: number;
offset?: number;
}): CancelablePromise<HistoryListResponse> {
return __request(OpenAPI, {
method: 'GET',
url: '/history',
query: {
profile_id: profileId,
search: search,
limit: limit,
offset: offset,
},
errors: {
422: `Validation Error`,
},
});
}
/**
* Get Generation
* Get a generation by ID.
* @returns HistoryResponse Successful Response
* @throws ApiError
*/
public static getGenerationHistoryGenerationIdGet({
generationId,
}: {
generationId: string;
}): CancelablePromise<HistoryResponse> {
return __request(OpenAPI, {
method: 'GET',
url: '/history/{generation_id}',
path: {
generation_id: generationId,
},
errors: {
422: `Validation Error`,
},
});
}
/**
* Delete Generation
* Delete a generation.
* @returns any Successful Response
* @throws ApiError
*/
public static deleteGenerationHistoryGenerationIdDelete({
generationId,
}: {
generationId: string;
}): CancelablePromise<any> {
return __request(OpenAPI, {
method: 'DELETE',
url: '/history/{generation_id}',
path: {
generation_id: generationId,
},
errors: {
422: `Validation Error`,
},
});
}
/**
* Get Stats
* Get generation statistics.
* @returns any Successful Response
* @throws ApiError
*/
public static getStatsHistoryStatsGet(): CancelablePromise<any> {
return __request(OpenAPI, {
method: 'GET',
url: '/history/stats',
});
}
/**
* Transcribe Audio
* Transcribe audio file to text.
* @returns TranscriptionResponse Successful Response
* @throws ApiError
*/
public static transcribeAudioTranscribePost({
formData,
}: {
formData: Body_transcribe_audio_transcribe_post;
}): CancelablePromise<TranscriptionResponse> {
return __request(OpenAPI, {
method: 'POST',
url: '/transcribe',
formData: formData,
mediaType: 'multipart/form-data',
errors: {
422: `Validation Error`,
},
});
}
/**
* Get Audio
* Serve generated audio file.
* @returns any Successful Response
* @throws ApiError
*/
public static getAudioAudioGenerationIdGet({
generationId,
}: {
generationId: string;
}): CancelablePromise<any> {
return __request(OpenAPI, {
method: 'GET',
url: '/audio/{generation_id}',
path: {
generation_id: generationId,
},
errors: {
422: `Validation Error`,
},
});
}
/**
* Load Model
* Manually load TTS model.
* @returns any Successful Response
* @throws ApiError
*/
public static loadModelModelsLoadPost({
modelSize = '1.7B',
}: {
modelSize?: string;
}): CancelablePromise<any> {
return __request(OpenAPI, {
method: 'POST',
url: '/models/load',
query: {
model_size: modelSize,
},
errors: {
422: `Validation Error`,
},
});
}
/**
* Unload Model
* Unload TTS model to free memory.
* @returns any Successful Response
* @throws ApiError
*/
public static unloadModelModelsUnloadPost(): CancelablePromise<any> {
return __request(OpenAPI, {
method: 'POST',
url: '/models/unload',
});
}
/**
* Get Model Progress
* Get model download progress via Server-Sent Events.
* @returns any Successful Response
* @throws ApiError
*/
public static getModelProgressModelsProgressModelNameGet({
modelName,
}: {
modelName: string;
}): CancelablePromise<any> {
return __request(OpenAPI, {
method: 'GET',
url: '/models/progress/{model_name}',
path: {
model_name: modelName,
},
errors: {
422: `Validation Error`,
},
});
}
/**
* Get Model Status
* Get status of all available models.
* @returns ModelStatusListResponse Successful Response
* @throws ApiError
*/
public static getModelStatusModelsStatusGet(): CancelablePromise<ModelStatusListResponse> {
return __request(OpenAPI, {
method: 'GET',
url: '/models/status',
});
}
/**
* Trigger Model Download
* Trigger download of a specific model.
* @returns any Successful Response
* @throws ApiError
*/
public static triggerModelDownloadModelsDownloadPost({
requestBody,
}: {
requestBody: ModelDownloadRequest;
}): CancelablePromise<any> {
return __request(OpenAPI, {
method: 'POST',
url: '/models/download',
body: requestBody,
mediaType: 'application/json',
errors: {
422: `Validation Error`,
},
});
}
}
+1 -26
View File
@@ -213,10 +213,6 @@ export interface CaptureSettings {
/** Whether the global keyboard hotkey is armed. Off by default — turning
* this on triggers the macOS Input Monitoring TCC prompt. */
hotkey_enabled: boolean;
/** Hold the mic open while dictation is enabled so push-to-talk doesn't clip
* the first words. Off by default — when on, the OS mic indicator stays lit
* the whole time dictation is enabled. */
keep_mic_warm: boolean;
/** keytap key names. Defaults are platform-specific right-hand modifiers. */
chord_push_to_talk_keys: string[];
/** keytap key names. Toggle adds Space to the platform-specific PTT chord. */
@@ -273,8 +269,7 @@ export interface HealthResponse {
gpu_type?: string;
vram_used_mb?: number;
backend_type?: string;
backend_variant?: string; // "cpu", "cuda", or "rocm"
supports_rocm?: boolean; // AMD GPU on Windows — the ROCm backend is applicable
backend_variant?: string; // "cpu" or "cuda"
}
export interface CudaDownloadProgress {
@@ -296,26 +291,6 @@ export interface CudaStatus {
download_progress?: CudaDownloadProgress;
}
export interface RocmDownloadProgress {
model_name: string;
current: number;
total: number;
progress: number;
filename?: string;
status: 'downloading' | 'extracting' | 'complete' | 'error';
timestamp: string;
error?: string;
}
export interface RocmStatus {
available: boolean; // ROCm binary exists on disk
active: boolean; // Currently running the ROCm binary
binary_path?: string;
rocm_libs_version?: string;
downloading: boolean; // Download in progress
download_progress?: RocmDownloadProgress;
}
export interface ModelProgress {
model_name: string;
current: number;
-70
View File
@@ -1,70 +0,0 @@
import { describe, expect, test } from 'bun:test';
import {
ALL_LANGUAGES,
ENGINE_LANGUAGES,
LANGUAGE_CODES,
LANGUAGE_OPTIONS,
getLanguageOptionsForEngine,
} from './languages';
describe('ENGINE_LANGUAGES', () => {
test('every engine maps only to codes defined in ALL_LANGUAGES', () => {
for (const [engine, codes] of Object.entries(ENGINE_LANGUAGES)) {
for (const code of codes) {
expect(ALL_LANGUAGES[code], `${engine} references unknown code "${code}"`).toBeDefined();
}
}
});
test('no engine lists a language twice', () => {
for (const [engine, codes] of Object.entries(ENGINE_LANGUAGES)) {
expect(new Set(codes).size, `${engine} has duplicate codes`).toBe(codes.length);
}
});
test('every engine supports at least English', () => {
for (const codes of Object.values(ENGINE_LANGUAGES)) {
expect(codes).toContain('en');
}
});
test('English-only engines list exactly one language', () => {
expect(ENGINE_LANGUAGES.luxtts).toEqual(['en']);
expect(ENGINE_LANGUAGES.chatterbox_turbo).toEqual(['en']);
});
test('qwen and qwen_custom_voice support the same languages', () => {
expect(ENGINE_LANGUAGES.qwen_custom_voice).toEqual(ENGINE_LANGUAGES.qwen);
});
});
describe('getLanguageOptionsForEngine', () => {
test('builds value/label pairs from ALL_LANGUAGES', () => {
expect(getLanguageOptionsForEngine('luxtts')).toEqual([{ value: 'en', label: 'English' }]);
});
test('preserves the engine declaration order', () => {
const values = getLanguageOptionsForEngine('qwen').map((o) => o.value);
expect(values).toEqual([...ENGINE_LANGUAGES.qwen]);
});
test('falls back to qwen languages for unknown engines', () => {
expect(getLanguageOptionsForEngine('does-not-exist')).toEqual(
getLanguageOptionsForEngine('qwen'),
);
});
});
describe('language option exports', () => {
test('LANGUAGE_CODES covers every ALL_LANGUAGES key exactly once', () => {
const codes: string[] = [...LANGUAGE_CODES].sort();
expect(codes).toEqual(Object.keys(ALL_LANGUAGES).sort());
expect(new Set(LANGUAGE_CODES).size).toBe(LANGUAGE_CODES.length);
});
test('LANGUAGE_OPTIONS labels match ALL_LANGUAGES', () => {
for (const option of LANGUAGE_OPTIONS) {
expect(option.label).toBe(ALL_LANGUAGES[option.value]);
}
});
});
+139 -371
View File
@@ -4,45 +4,12 @@ import { convertToWav } from '@/lib/utils/audio';
interface UseAudioRecordingOptions {
maxDurationSeconds?: number;
// ``context`` is whatever was handed to ``startRecording`` for this take,
// threaded back untouched so callers can correlate the result with the
// recording it came from (the dictate window pairs it with the focus
// snapshot captured at chord-start).
onRecordingComplete?: (blob: Blob, duration?: number, context?: unknown) => void;
/**
* Keep the microphone ``MediaStream`` open between recordings instead of
* tearing it down on every stop. This is what removes the "first words get
* clipped" problem on push-to-talk dictation: ``getUserMedia`` on macOS can
* take several hundred ms — up to a second cold — to hand back a stream, and
* ``MediaRecorder`` only starts capturing *after* it resolves, so everything
* spoken in that window is lost. With a warm stream already open, the next
* ``startRecording`` skips ``getUserMedia`` entirely.
*
* Off by default: the voice-clone sample recorders release the device
* immediately, and the dictation session only opts in when the user enables
* the "keep microphone ready" setting. While on, the warm stream stays open —
* and the OS mic-in-use indicator stays lit — until it's explicitly released
* (dictation disabled or the setting turned off), so the trade-off is visible
* and user-controlled rather than a background mic that's always warm.
*/
keepWarm?: boolean;
onRecordingComplete?: (blob: Blob, duration?: number) => void;
}
// Audio constraints for capture. Kept identical to the previous inline value so
// this change is purely about *when* the stream is opened, not *how*.
const AUDIO_CONSTRAINTS: MediaTrackConstraints = {
echoCancellation: true,
noiseSuppression: true,
autoGainControl: true,
};
const streamHasLiveAudio = (stream: MediaStream | null): stream is MediaStream =>
!!stream && stream.getAudioTracks().some((t) => t.readyState === 'live');
export function useAudioRecording({
maxDurationSeconds,
onRecordingComplete,
keepWarm = false,
}: UseAudioRecordingOptions = {}) {
const platform = usePlatform();
const [isRecording, setIsRecording] = useState(false);
@@ -50,392 +17,195 @@ export function useAudioRecording({
const [error, setError] = useState<string | null>(null);
const mediaRecorderRef = useRef<MediaRecorder | null>(null);
const chunksRef = useRef<Blob[]>([]);
// The stream currently backing the MediaRecorder. When ``keepWarm`` is set
// this is the same object as ``warmStreamRef`` and is *not* torn down on
// stop; otherwise it's stopped as soon as the recording completes.
const streamRef = useRef<MediaStream | null>(null);
// Persistent pre-opened stream reused across recordings when ``keepWarm``.
const warmStreamRef = useRef<MediaStream | null>(null);
const timerRef = useRef<number | null>(null);
const startTimeRef = useRef<number | null>(null);
const cancelledRef = useRef<boolean>(false);
// Mirror of ``isRecording`` for reads inside callbacks that would otherwise
// close over a stale render.
const isRecordingRef = useRef(false);
// A ``getUserMedia`` call in flight, shared so concurrent acquirers (prewarm
// plus an immediate chord) coalesce onto one stream instead of each opening —
// and orphaning — their own.
const acquiringRef = useRef<Promise<MediaStream> | null>(null);
// True from ``startRecording`` entry until the recorder is actually running
// (or has failed), so a stop that arrives mid-acquisition can be deferred.
const startingRef = useRef(false);
// True from MediaRecorder.stop() until onstop has snapshotted the take's
// shared refs. React state and MediaRecorder.state both flip before onstop,
// so without this gate a rapid next chord can clear chunks/duration/cancel
// state out from under the recorder that is still finalising.
const finishingRef = useRef(false);
const pendingStopRef = useRef(false);
// Bumped per recording so a stale recorder's ``onstop`` can tell it's no
// longer the active one before it touches the shared stream refs.
const recordingCounterRef = useRef(0);
// Bumped whenever the warm stream is released/aborted so a ``getUserMedia``
// still in flight can tell its result is stale and stop it instead of
// adopting a live mic after disable/unmount.
const acquireGenRef = useRef(0);
// Set when a release is requested mid-recording; the onstop path performs the
// deferred release once capture finishes rather than yanking the device now.
const releaseAfterStopRef = useRef(false);
// Keeps the ref in lockstep with the state so the synchronous stop path reads
// a fresh value without waiting for a rerender.
const setRecording = useCallback((next: boolean) => {
isRecordingRef.current = next;
setIsRecording(next);
}, []);
const startRecording = useCallback(async () => {
try {
setError(null);
chunksRef.current = [];
cancelledRef.current = false;
setDuration(0);
const releaseWarmStream = useCallback(() => {
// Invalidate any getUserMedia still in flight so its stream is stopped on
// resolve rather than adopted as the warm stream.
acquireGenRef.current += 1;
// Don't tear the device out from under an active/starting recording — the
// warm stream is the one backing it; defer to the onstop path instead.
if (isRecordingRef.current || startingRef.current) {
releaseAfterStopRef.current = true;
return;
}
warmStreamRef.current?.getTracks().forEach((track) => {
track.stop();
});
warmStreamRef.current = null;
}, []);
// Assert that getUserMedia is reachable, mirroring the previous inline guard
// (Tauri webviews occasionally expose ``navigator.mediaDevices`` a beat late).
const assertMediaDevices = useCallback(async () => {
if (typeof navigator === 'undefined') {
throw new Error('Navigator API is not available. This might be a Tauri configuration issue.');
}
if (!navigator.mediaDevices || !navigator.mediaDevices.getUserMedia) {
await new Promise((resolve) => setTimeout(resolve, 100));
if (!navigator.mediaDevices || !navigator.mediaDevices.getUserMedia) {
throw new Error(
platform.metadata.isTauri
? 'Microphone access is not available. Please ensure:\n1. The app has microphone permissions in System Settings (macOS: System Settings > Privacy & Security > Microphone)\n2. You restart the app after granting permissions\n3. You are using Tauri v2 with a webview that supports getUserMedia'
: 'Microphone access is not available. Please ensure you are using a secure context (HTTPS or localhost) and that your browser has microphone permissions enabled.',
);
// Check if getUserMedia is available
// In Tauri, navigator.mediaDevices might not be available immediately
if (typeof navigator === 'undefined') {
const errorMsg =
'Navigator API is not available. This might be a Tauri configuration issue.';
setError(errorMsg);
throw new Error(errorMsg);
}
}
}, [platform.metadata.isTauri]);
// Return a live capture stream, reusing the warm one when available so the
// hot path (chord-down → record) never waits on getUserMedia.
const acquireStream = useCallback(async (): Promise<MediaStream> => {
// Captured separately so it stays typed as the full stream after the live
// check narrows ``warmStreamRef.current`` itself.
const existing = warmStreamRef.current;
if (streamHasLiveAudio(warmStreamRef.current)) {
return warmStreamRef.current;
}
// Coalesce concurrent acquirers onto one getUserMedia call so prewarm and
// an immediate chord can't open two streams.
if (acquiringRef.current) return acquiringRef.current;
// A dead warm stream (device unplugged / tracks ended) — drop it and reopen.
if (existing) {
existing.getTracks().forEach((track) => {
track.stop();
});
warmStreamRef.current = null;
}
const gen = acquireGenRef.current;
const acquisition = (async () => {
await assertMediaDevices();
if (!navigator.mediaDevices || !navigator.mediaDevices.getUserMedia) {
// Try waiting a bit for Tauri webview to initialize
await new Promise((resolve) => setTimeout(resolve, 100));
if (!navigator.mediaDevices || !navigator.mediaDevices.getUserMedia) {
console.error('MediaDevices check:', {
hasNavigator: typeof navigator !== 'undefined',
hasMediaDevices: !!navigator?.mediaDevices,
hasGetUserMedia: !!navigator?.mediaDevices?.getUserMedia,
isTauri: platform.metadata.isTauri,
});
const errorMsg = platform.metadata.isTauri
? 'Microphone access is not available. Please ensure:\n1. The app has microphone permissions in System Settings (macOS: System Settings > Privacy & Security > Microphone)\n2. You restart the app after granting permissions\n3. You are using Tauri v2 with a webview that supports getUserMedia'
: 'Microphone access is not available. Please ensure you are using a secure context (HTTPS or localhost) and that your browser has microphone permissions enabled.';
setError(errorMsg);
throw new Error(errorMsg);
}
}
// Request microphone access
const stream = await navigator.mediaDevices.getUserMedia({
audio: AUDIO_CONSTRAINTS,
audio: {
echoCancellation: true,
noiseSuppression: true,
autoGainControl: true,
},
});
// Released / disabled / unmounted while acquiring — this stream is stale,
// so stop it instead of leaving a live mic open, and abort the caller.
if (gen !== acquireGenRef.current) {
stream.getTracks().forEach((track) => {
streamRef.current = stream;
// Create MediaRecorder with preferred MIME type
const options: MediaRecorderOptions = {
mimeType: 'audio/webm;codecs=opus',
};
// Fallback to default if webm not supported
if (!MediaRecorder.isTypeSupported(options.mimeType!)) {
delete options.mimeType;
}
const mediaRecorder = new MediaRecorder(stream, options);
mediaRecorderRef.current = mediaRecorder;
mediaRecorder.ondataavailable = (event) => {
if (event.data.size > 0) {
chunksRef.current.push(event.data);
}
};
mediaRecorder.onstop = async () => {
// Snapshot the cancellation flag and recorded duration immediately —
// cancelRecording() clears chunks and sets cancelledRef synchronously
// before this async handler runs, so we must check it first.
const wasCancelled = cancelledRef.current;
const recordedDuration = startTimeRef.current
? (Date.now() - startTimeRef.current) / 1000
: undefined;
const webmBlob = new Blob(chunksRef.current, { type: 'audio/webm' });
// Stop all tracks now that we have the data
streamRef.current?.getTracks().forEach((track) => {
track.stop();
});
throw new Error('microphone acquisition aborted');
}
if (keepWarm) warmStreamRef.current = stream;
return stream;
})();
acquiringRef.current = acquisition;
try {
return await acquisition;
} finally {
if (acquiringRef.current === acquisition) acquiringRef.current = null;
}
}, [assertMediaDevices, keepWarm]);
streamRef.current = null;
/**
* Open the microphone ahead of the first recording so the initial dictation
* doesn't clip. No-op unless ``keepWarm`` is set. Safe to call repeatedly and
* safe to fail (e.g. permission not yet granted) — ``startRecording`` still
* surfaces a real error if capture is genuinely unavailable.
*/
const prewarm = useCallback(async () => {
if (!keepWarm) return;
try {
await acquireStream();
} catch {
// Permission missing / device busy / aborted — recording will report a
// real error if capture is genuinely unavailable.
}
}, [keepWarm, acquireStream]);
// Don't fire completion callback if the recording was cancelled
if (wasCancelled) return;
const startRecording = useCallback(
async (context?: unknown) => {
// A second chord can arrive while the first one is still waiting on
// getUserMedia. Never create overlapping MediaRecorders on the same
// coalesced stream; the original take will honor any deferred stop.
if (
startingRef.current ||
finishingRef.current ||
mediaRecorderRef.current?.state === 'recording'
)
return;
startingRef.current = true;
pendingStopRef.current = false;
// A new recording supersedes any release deferred from a prior take.
releaseAfterStopRef.current = false;
const recordingId = ++recordingCounterRef.current;
try {
setError(null);
chunksRef.current = [];
cancelledRef.current = false;
setDuration(0);
// Reuse the warm stream when present (instant); otherwise open one now.
const stream = await acquireStream();
streamRef.current = stream;
// Create MediaRecorder with preferred MIME type
const options: MediaRecorderOptions = {
mimeType: 'audio/webm;codecs=opus',
};
// Fallback to default if webm not supported
if (!MediaRecorder.isTypeSupported(options.mimeType!)) {
delete options.mimeType;
// Convert to WAV format to avoid needing ffmpeg on backend
try {
const wavBlob = await convertToWav(webmBlob);
onRecordingComplete?.(wavBlob, recordedDuration);
} catch (err) {
console.error('Error converting audio to WAV:', err);
// Fallback to original blob if conversion fails
onRecordingComplete?.(webmBlob, recordedDuration);
}
};
const mediaRecorder = new MediaRecorder(stream, options);
mediaRecorderRef.current = mediaRecorder;
mediaRecorder.onerror = (event) => {
setError('Recording error occurred');
console.error('MediaRecorder error:', event);
};
mediaRecorder.ondataavailable = (event) => {
if (event.data.size > 0) {
chunksRef.current.push(event.data);
}
};
// WebKit's MediaRecorder drops the WebM EBML header from chunks when
// started with a timeslice, so concatenated blobs fail to parse in
// both AudioContext and ffmpeg. Starting with no timeslice produces
// exactly one dataavailable on stop() with a valid container.
mediaRecorder.start();
setIsRecording(true);
startTimeRef.current = Date.now();
mediaRecorder.onstop = async () => {
// Whether this recorder is still the active one. A stale onstop (an
// older recorder stopping after a newer startRecording) must not touch
// the shared stream refs.
const isCurrent = recordingCounterRef.current === recordingId;
// Snapshot the cancellation flag and recorded duration immediately —
// cancelRecording() clears chunks and sets cancelledRef synchronously
// before this async handler runs, so we must check it first.
const wasCancelled = cancelledRef.current;
const recordedDuration = startTimeRef.current
? (Date.now() - startTimeRef.current) / 1000
: undefined;
// Start timer
timerRef.current = window.setInterval(() => {
if (startTimeRef.current) {
const elapsed = (Date.now() - startTimeRef.current) / 1000;
setDuration(elapsed);
const webmBlob = new Blob(chunksRef.current, { type: 'audio/webm' });
// Release the device unless we're keeping it warm for the next capture.
// Act on this recorder's own stream; only touch the shared refs when
// this is still the current recording.
if (keepWarm) {
if (isCurrent) {
streamRef.current = null;
// A release requested mid-recording (dictation disabled) is
// honored now that capture has finished; otherwise the warm
// stream stays open for the next take.
if (releaseAfterStopRef.current) {
releaseAfterStopRef.current = false;
releaseWarmStream();
}
}
} else {
stream.getTracks().forEach((track) => {
track.stop();
});
if (isCurrent) streamRef.current = null;
}
// All shared per-take refs have now been snapshotted and stream
// cleanup is complete. A new take may begin while WAV conversion and
// upload continue using the local values above.
finishingRef.current = false;
// Don't fire completion callback if the recording was cancelled
if (wasCancelled) return;
// Convert to WAV format to avoid needing ffmpeg on backend
try {
const wavBlob = await convertToWav(webmBlob);
onRecordingComplete?.(wavBlob, recordedDuration, context);
} catch (err) {
console.error('Error converting audio to WAV:', err);
// Fallback to original blob if conversion fails
onRecordingComplete?.(webmBlob, recordedDuration, context);
}
};
mediaRecorder.onerror = (event) => {
setError('Recording error occurred');
console.error('MediaRecorder error:', event);
};
// WebKit's MediaRecorder drops the WebM EBML header from chunks when
// started with a timeslice, so concatenated blobs fail to parse in
// both AudioContext and ffmpeg. Starting with no timeslice produces
// exactly one dataavailable on stop() with a valid container.
mediaRecorder.start();
setRecording(true);
startTimeRef.current = Date.now();
startingRef.current = false;
// A stop (chord release) that landed while the mic was still opening —
// honor it now that capture has actually begun.
if (pendingStopRef.current) {
pendingStopRef.current = false;
finishingRef.current = true;
mediaRecorder.stop();
setRecording(false);
return;
}
// Start timer
timerRef.current = window.setInterval(() => {
if (startTimeRef.current) {
const elapsed = (Date.now() - startTimeRef.current) / 1000;
setDuration(elapsed);
// Auto-stop at max duration when the caller opts in — dictation
// sessions pass undefined and run until the user releases the
// chord or hits stop; voice-clone sample recorders pass 29s to
// keep reference clips short.
if (maxDurationSeconds !== undefined && elapsed >= maxDurationSeconds) {
if (mediaRecorderRef.current && mediaRecorderRef.current.state !== 'inactive') {
finishingRef.current = true;
mediaRecorderRef.current.stop();
setRecording(false);
if (timerRef.current !== null) {
clearInterval(timerRef.current);
timerRef.current = null;
}
// Auto-stop at max duration when the caller opts in — dictation
// sessions pass undefined and run until the user releases the
// chord or hits stop; voice-clone sample recorders pass 29s to
// keep reference clips short.
if (maxDurationSeconds !== undefined && elapsed >= maxDurationSeconds) {
if (mediaRecorderRef.current && mediaRecorderRef.current.state !== 'inactive') {
mediaRecorderRef.current.stop();
setIsRecording(false);
if (timerRef.current !== null) {
clearInterval(timerRef.current);
timerRef.current = null;
}
}
}
}, 100);
} catch (err) {
const errorMessage =
err instanceof Error
? err.message
: 'Failed to access microphone. Please check permissions.';
// A fresh (non-warm) stream opened before the failure must be released
// so the mic doesn't stay lit; a warm stream is reusable, so it's kept.
if (!keepWarm) {
streamRef.current?.getTracks().forEach((track) => {
track.stop();
});
streamRef.current = null;
}
startingRef.current = false;
finishingRef.current = false;
pendingStopRef.current = false;
setError(errorMessage);
setRecording(false);
}
},
[
maxDurationSeconds,
onRecordingComplete,
acquireStream,
keepWarm,
releaseWarmStream,
setRecording,
],
);
}, 100);
} catch (err) {
const errorMessage =
err instanceof Error
? err.message
: 'Failed to access microphone. Please check permissions.';
setError(errorMessage);
setIsRecording(false);
}
}, [maxDurationSeconds, onRecordingComplete]);
const stopRecording = useCallback(() => {
// The recorder's own state is the lifecycle authority — React ``isRecording``
// lags a render behind ``mediaRecorder.start()``, so a chord release in that
// window would otherwise be dropped.
const recorder = mediaRecorderRef.current;
if (recorder && recorder.state === 'recording') {
finishingRef.current = true;
recorder.stop();
setRecording(false);
if (mediaRecorderRef.current && isRecording) {
mediaRecorderRef.current.stop();
setIsRecording(false);
if (timerRef.current !== null) {
clearInterval(timerRef.current);
timerRef.current = null;
}
} else if (startingRef.current) {
// Stop arrived before capture began (mic still opening) — defer it so
// startRecording stops as soon as the recorder goes live.
pendingStopRef.current = true;
}
}, [setRecording]);
}, [isRecording]);
const cancelRecording = useCallback(() => {
cancelledRef.current = true; // Must be set before stop() triggers onstop
const recorder = mediaRecorderRef.current;
if (recorder && recorder.state !== 'inactive') {
if (mediaRecorderRef.current) {
cancelledRef.current = true; // Must be set before stop() triggers onstop
chunksRef.current = [];
finishingRef.current = true;
recorder.stop();
setRecording(false);
mediaRecorderRef.current.stop();
setIsRecording(false);
setDuration(0);
} else if (startingRef.current) {
// Cancel during mic acquisition — stop as soon as capture begins; the
// cancelled flag suppresses the completion callback.
pendingStopRef.current = true;
}
// Keep the device warm for the next capture when opted in; otherwise stop
// the tracks so the mic is released immediately.
if (keepWarm) {
streamRef.current = null;
if (releaseAfterStopRef.current) {
releaseAfterStopRef.current = false;
releaseWarmStream();
}
} else {
streamRef.current?.getTracks().forEach((track) => {
track.stop();
});
streamRef.current = null;
}
// Stop all tracks
streamRef.current?.getTracks().forEach((track) => {
track.stop();
});
streamRef.current = null;
if (timerRef.current !== null) {
clearInterval(timerRef.current);
timerRef.current = null;
}
}, [keepWarm, releaseWarmStream, setRecording]);
}, []);
// Cleanup on unmount — always fully release the device, warm or not.
// Cleanup on unmount
useEffect(() => {
return () => {
// Invalidate any in-flight acquisition so a stream resolving after unmount
// stops itself instead of leaking a live mic.
acquireGenRef.current += 1;
if (timerRef.current !== null) {
clearInterval(timerRef.current);
}
streamRef.current?.getTracks().forEach((track) => {
track.stop();
});
warmStreamRef.current?.getTracks().forEach((track) => {
track.stop();
});
};
}, []);
@@ -446,7 +216,5 @@ export function useAudioRecording({
startRecording,
stopRecording,
cancelRecording,
prewarm,
releaseWarm: releaseWarmStream,
};
}
+23 -60
View File
@@ -54,15 +54,11 @@ const SHORT_RECORDING_MESSAGE = 'Recording too short, canceled';
export type CapturePillState = PillState | 'hidden';
export interface UseCaptureRecordingSessionOptions {
/** Keep the microphone stream open between dictations when explicitly
* enabled. Off by default so normal recorders release the device. */
keepMicWarm?: boolean;
/**
* Fired after a capture row is created on the server. Callers can use this
* to select the new capture or emit a Tauri event to a sibling window.
* ``context`` is whatever was passed to ``startRecording`` for this take.
*/
onCaptureCreated?: (capture: CaptureResponse, context?: unknown) => void;
onCaptureCreated?: (capture: CaptureResponse) => void;
/**
* Fired with the final delivered text — refined if ``auto_refine`` was on
* for this capture, raw transcript otherwise. Used by the floating
@@ -70,14 +66,12 @@ export interface UseCaptureRecordingSessionOptions {
*
* ``allowAutoPaste`` snapshots the setting at chord-start so a refine that
* lands after the user flips the toggle still uses the value the capture
* was created under. ``context`` is the value passed to ``startRecording``
* for this take, so overlapping dictations can't cross their targets.
* was created under.
*/
onFinalText?: (
text: string,
capture: CaptureResponse,
allowAutoPaste: boolean,
context?: unknown,
) => void;
}
@@ -88,14 +82,12 @@ export interface UseCaptureRecordingSessionResult {
isRecording: boolean;
isUploading: boolean;
isRefining: boolean;
startRecording: (context?: unknown) => void;
startRecording: () => void;
stopRecording: () => void;
toggleRecording: () => void;
dismissError: () => void;
uploadFile: (file: File, source: CaptureSource) => void;
refine: (captureId: string) => void;
prewarm: () => Promise<void>;
releaseWarm: () => void;
}
/**
@@ -131,13 +123,10 @@ export function useCaptureRecordingSession(
const onFinalTextRef = useRef(options.onFinalText);
onFinalTextRef.current = options.onFinalText;
// Per-capture recording context and its ``allow_auto_paste`` snapshot, keyed
// by capture id so a refine that resolves after another dictation started
// still delivers to the right target with the setting the capture was created
// under. Populated on capture-create and consumed once the final text lands.
const captureDeliveryRef = useRef<Map<string, { context: unknown; allowAutoPaste: boolean }>>(
new Map(),
);
// Snapshot of ``allow_auto_paste`` from the capture-create response —
// held so the refine onSuccess (which only sees the plain CaptureResponse)
// can still pass the original setting through to onFinalText.
const allowAutoPasteRef = useRef<boolean>(true);
const clearRestTimer = useCallback(() => {
if (restTimerRef.current !== null) {
@@ -203,34 +192,20 @@ export function useCaptureRecordingSession(
queryClient.invalidateQueries({ queryKey: ['captures'] });
broadcastUpdated(captureId);
if (pillStateRef.current === 'refining') scheduleHidePill();
const delivery = captureDeliveryRef.current.get(captureId);
captureDeliveryRef.current.delete(captureId);
const finalText = data.transcript_refined ?? data.transcript_raw;
if (finalText) {
onFinalTextRef.current?.(
finalText,
data,
delivery?.allowAutoPaste ?? true,
delivery?.context,
);
onFinalTextRef.current?.(finalText, data, allowAutoPasteRef.current);
}
},
onError: (err: Error, captureId) => {
captureDeliveryRef.current.delete(captureId);
onError: (err: Error) => {
showError(err.message || 'Refinement failed');
},
});
const uploadMutation = useMutation({
mutationFn: async ({
file,
source,
}: {
file: File;
source: CaptureSource;
context?: unknown;
}) => apiClient.createCapture(file, { source }),
onSuccess: (capture, { context }) => {
mutationFn: async ({ file, source }: { file: File; source: CaptureSource }) =>
apiClient.createCapture(file, { source }),
onSuccess: (capture) => {
queryClient.setQueryData<CaptureListResponse>(['captures'], (prev) => {
if (!prev) return prev;
if (prev.items.some((c) => c.id === capture.id)) return prev;
@@ -238,12 +213,9 @@ export function useCaptureRecordingSession(
});
queryClient.invalidateQueries({ queryKey: ['captures'] });
broadcastCreated(capture);
onCaptureCreatedRef.current?.(capture, context);
onCaptureCreatedRef.current?.(capture);
allowAutoPasteRef.current = capture.allow_auto_paste;
if (capture.auto_refine) {
captureDeliveryRef.current.set(capture.id, {
context,
allowAutoPaste: capture.allow_auto_paste,
});
setPillState('refining');
refineMutation.mutate(capture.id);
} else {
@@ -253,7 +225,6 @@ export function useCaptureRecordingSession(
capture.transcript_raw,
capture,
capture.allow_auto_paste,
context,
);
}
}
@@ -278,11 +249,8 @@ export function useCaptureRecordingSession(
startRecording: beginAudioRecording,
stopRecording,
error: recordError,
prewarm,
releaseWarm,
} = useAudioRecording({
keepWarm: options.keepMicWarm ?? false,
onRecordingComplete: (blob, recordedDuration, context) => {
onRecordingComplete: (blob, recordedDuration) => {
// Trigger-happy tap — MediaRecorder hasn't emitted a usable chunk yet
// so the blob is empty or unparseable. Surface it as a transient pill
// so the user sees their recording was recognised and canceled.
@@ -300,7 +268,7 @@ export function useCaptureRecordingSession(
const file = new File([blob], `dictation-${Date.now()}.${extension}`, {
type: blob.type,
});
uploadMutation.mutate({ file, source: 'dictation', context });
uploadMutation.mutate({ file, source: 'dictation' });
},
});
@@ -310,16 +278,13 @@ export function useCaptureRecordingSession(
}
}, [recordError, showError]);
const startRecording = useCallback(
(context?: unknown) => {
if (isRecording) return;
clearRestTimer();
setFrozenElapsedMs(0);
setPillState('recording');
beginAudioRecording(context);
},
[isRecording, beginAudioRecording, clearRestTimer],
);
const startRecording = useCallback(() => {
if (isRecording) return;
clearRestTimer();
setFrozenElapsedMs(0);
setPillState('recording');
beginAudioRecording();
}, [isRecording, beginAudioRecording, clearRestTimer]);
const toggleRecording = useCallback(() => {
if (isRecording) {
@@ -359,7 +324,5 @@ export function useCaptureRecordingSession(
dismissError,
uploadFile,
refine,
prewarm,
releaseWarm,
};
}
+1 -26
View File
@@ -1,6 +1,5 @@
import { invoke } from '@tauri-apps/api/core';
import { emit, listen } from '@tauri-apps/api/event';
import { useEffect, useRef } from 'react';
import { useEffect } from 'react';
import { useDictationReadiness } from '@/lib/hooks/useDictationReadiness';
import { useCaptureSettings } from '@/lib/hooks/useSettings';
import { usePlatform } from '@/platform/PlatformContext';
@@ -31,45 +30,21 @@ export function useChordSync() {
const { settings } = useCaptureSettings();
const { canRecord } = useDictationReadiness();
const enabled = settings?.hotkey_enabled;
const keepMicWarm = settings?.keep_mic_warm;
const pushKeys = settings?.chord_push_to_talk_keys;
const toggleKeys = settings?.chord_toggle_to_talk_keys;
// Latest warm state, so the dictate window's mount-time request can be
// answered even between the dep-driven emits below.
const shouldWarmRef = useRef(false);
// The floating dictate window holds the mic warm ahead of the first chord to
// avoid clipping, but it's a separate webview with no view of settings. Mirror
// the decision to it: warm only when dictation is armed AND the user enabled
// "keep microphone ready". Gating here is what stops the always-mounted pill
// from opening the mic — or prompting for access — when the user hasn't asked.
useEffect(() => {
if (!platform.metadata.isTauri) return;
const unlisten = listen('dictate:warm-request', () => {
emit('dictate:warm', shouldWarmRef.current).catch(() => {});
});
return () => {
unlisten.then((fn) => fn()).catch(() => {});
};
}, [platform.metadata.isTauri]);
useEffect(() => {
if (!platform.metadata.isTauri) return;
if (enabled === undefined || !pushKeys || !toggleKeys) return;
const shouldArm = enabled && canRecord;
const shouldWarm = shouldArm && (keepMicWarm ?? false);
shouldWarmRef.current = shouldWarm;
const command = shouldArm ? 'enable_hotkey' : 'disable_hotkey';
const args = shouldArm ? { pushToTalk: pushKeys, toggleToTalk: toggleKeys } : {};
invoke(command, args).catch((err) => {
console.warn(`[chord-sync] ${command} failed:`, err);
});
emit('dictate:warm', shouldWarm).catch(() => {});
}, [
platform.metadata.isTauri,
enabled,
keepMicWarm,
canRecord,
// Stringify so a referentially-new array with the same content
// doesn't fire a redundant invoke on every settings refetch.
+4 -8
View File
@@ -47,14 +47,12 @@ export function useExportGeneration() {
mutationFn: async ({ generationId, text }: { generationId: string; text: string }) => {
const blob = await apiClient.exportGeneration(generationId);
// Create safe filename from text. Append a short id so exports of
// similarly-worded generations don't collide on the same filename
// (the first 30 chars are frequently identical).
// Create safe filename from text
const safeText = text
.substring(0, 30)
.replace(/[^a-z0-9]/gi, '-')
.toLowerCase();
const filename = `generation-${safeText}-${generationId.substring(0, 8)}.voicebox.zip`;
const filename = `generation-${safeText}.voicebox.zip`;
await platform.filesystem.saveFile(filename, blob, [
{
@@ -75,14 +73,12 @@ export function useExportGenerationAudio() {
mutationFn: async ({ generationId, text }: { generationId: string; text: string }) => {
const blob = await apiClient.exportGenerationAudio(generationId);
// Create safe filename from text. Append a short id so exports of
// similarly-worded generations don't collide on the same filename
// (the first 30 chars are frequently identical).
// Create safe filename from text
const safeText = text
.substring(0, 30)
.replace(/[^a-z0-9]/gi, '-')
.toLowerCase();
const filename = `${safeText}-${generationId.substring(0, 8)}.wav`;
const filename = `${safeText}.wav`;
await platform.filesystem.saveFile(filename, blob, [
{
-71
View File
@@ -1,71 +0,0 @@
import { describe, expect, test } from 'bun:test';
import { formatDuration, formatEngineName, formatFileSize } from './format';
describe('formatDuration', () => {
test('formats zero', () => {
expect(formatDuration(0)).toBe('0:00');
});
test('pads single-digit seconds', () => {
expect(formatDuration(65)).toBe('1:05');
});
test('handles the minute boundary', () => {
expect(formatDuration(59)).toBe('0:59');
expect(formatDuration(60)).toBe('1:00');
});
test('floors fractional seconds', () => {
expect(formatDuration(89.9)).toBe('1:29');
});
test('does not roll minutes into hours', () => {
expect(formatDuration(3661)).toBe('61:01');
});
});
describe('formatFileSize', () => {
test('special-cases zero', () => {
expect(formatFileSize(0)).toBe('0 Bytes');
});
test('formats bytes below 1 KB', () => {
expect(formatFileSize(512)).toBe('512 Bytes');
expect(formatFileSize(1023)).toBe('1023 Bytes');
});
test('formats KB, MB, and GB boundaries', () => {
expect(formatFileSize(1024)).toBe('1 KB');
expect(formatFileSize(1024 ** 2)).toBe('1 MB');
expect(formatFileSize(1024 ** 3)).toBe('1 GB');
});
test('rounds to two decimal places', () => {
expect(formatFileSize(1536)).toBe('1.5 KB');
expect(formatFileSize(2_684_354_560)).toBe('2.5 GB');
expect(formatFileSize(1_234_567)).toBe('1.18 MB');
});
});
describe('formatEngineName', () => {
test('maps known engines to display names', () => {
expect(formatEngineName('luxtts')).toBe('LuxTTS');
expect(formatEngineName('chatterbox')).toBe('Chatterbox');
expect(formatEngineName('chatterbox_turbo')).toBe('Chatterbox Turbo');
});
test('defaults to Qwen when engine is undefined', () => {
expect(formatEngineName()).toBe('Qwen');
expect(formatEngineName(undefined, '1.7B')).toBe('Qwen');
});
test('appends the model size for qwen only', () => {
expect(formatEngineName('qwen', '1.7B')).toBe('Qwen 1.7B');
expect(formatEngineName('qwen')).toBe('Qwen');
expect(formatEngineName('luxtts', '1.7B')).toBe('LuxTTS');
});
test('passes unknown engines through verbatim', () => {
expect(formatEngineName('kokoro')).toBe('kokoro');
});
});
+15 -18
View File
@@ -1,5 +1,5 @@
import { formatDistance } from 'date-fns';
import { ja, zhCN, zhTW, fr } from 'date-fns/locale';
import { ja, zhCN, zhTW } from 'date-fns/locale';
import i18n from '@/i18n';
export function formatDuration(seconds: number): string {
@@ -16,35 +16,32 @@ function getDateLocale() {
return zhCN;
case 'zh-TW':
return zhTW;
case 'fr':
return fr;
default:
return undefined;
}
}
// Backend timestamps are naive UTC — append `Z` so JS doesn't parse a
// timezone-less date-time string as local time.
function parseServerDate(date: string | Date): Date {
if (typeof date !== 'string') {
return date;
}
const dateStr = date.trim();
if (!dateStr.includes('Z') && !dateStr.match(/[+-]\d{2}:\d{2}$/)) {
return new Date(`${dateStr}Z`);
}
return new Date(dateStr);
}
export function formatDate(date: string | Date): string {
return formatDistance(parseServerDate(date), new Date(), {
let dateObj: Date;
if (typeof date === 'string') {
const dateStr = date.trim();
if (!dateStr.includes('Z') && !dateStr.match(/[+-]\d{2}:\d{2}$/)) {
dateObj = new Date(`${dateStr}Z`);
} else {
dateObj = new Date(dateStr);
}
} else {
dateObj = date;
}
return formatDistance(dateObj, new Date(), {
addSuffix: true,
locale: getDateLocale(),
}).replace(/^about /i, '');
}
export function formatAbsoluteDate(date: string | Date): string {
const dateObj = parseServerDate(date);
const dateObj = typeof date === 'string' ? new Date(date) : date;
return dateObj.toLocaleString(i18n.language, {
month: 'short',
day: 'numeric',
-69
View File
@@ -1,69 +0,0 @@
import { describe, expect, test } from 'bun:test';
import { parseChangelog } from './parseChangelog';
const SAMPLE = `# Changelog
All notable changes to this project will be documented in this file.
## [0.5.0] - 2026-06-01
### Added
- Story track editor
- Cloud login
## [0.4.1]
### Fixed
- Trim clamping
## [0.4.0] - 2026-04-15
Initial public release.
[0.5.0]: https://example.com/compare/v0.4.1...v0.5.0
[0.4.1]: https://example.com/compare/v0.4.0...v0.4.1
`;
describe('parseChangelog', () => {
test('splits entries on version headings', () => {
const entries = parseChangelog(SAMPLE);
expect(entries.map((e) => e.version)).toEqual(['0.5.0', '0.4.1', '0.4.0']);
});
test('extracts the date when present and null otherwise', () => {
const entries = parseChangelog(SAMPLE);
expect(entries[0].date).toBe('2026-06-01');
expect(entries[1].date).toBeNull();
});
test('keeps the markdown body between headings', () => {
const entries = parseChangelog(SAMPLE);
expect(entries[0].body).toBe('### Added\n\n- Story track editor\n- Cloud login');
expect(entries[2].body).toBe('Initial public release.');
});
test('strips trailing link reference definitions from the last body', () => {
const entries = parseChangelog(SAMPLE);
expect(entries[2].body).toBe('Initial public release.');
expect(entries[2].body).not.toContain('example.com');
});
test('returns an empty array when no headings match', () => {
expect(parseChangelog('')).toEqual([]);
expect(parseChangelog('# Changelog\n\nNothing yet.')).toEqual([]);
});
test('handles a heading with an empty body', () => {
const entries = parseChangelog('## [1.0.0] - 2026-01-01\n');
expect(entries).toEqual([{ version: '1.0.0', date: '2026-01-01', body: '' }]);
});
test('accepts non-semver headings like Unreleased', () => {
const entries = parseChangelog('## [Unreleased]\n\n### Added\n\n- WIP\n');
expect(entries[0].version).toBe('Unreleased');
expect(entries[0].date).toBeNull();
expect(entries[0].body).toBe('### Added\n\n- WIP');
});
});
-106
View File
@@ -1,106 +0,0 @@
import { describe, expect, test } from 'bun:test';
import { MIN_CLIP_DURATION_MS, computeTrimValues } from './trim';
// A 10-second clip with no existing trims unless stated otherwise.
const DURATION = 10_000;
describe('computeTrimValues', () => {
describe('start handle', () => {
test('dragging right trims from the start', () => {
expect(computeTrimValues('start', 500, 0, 0, DURATION)).toEqual({
trim_start_ms: 500,
trim_end_ms: 0,
});
});
test('dragging left restores previously trimmed audio', () => {
expect(computeTrimValues('start', -300, 1000, 0, DURATION)).toEqual({
trim_start_ms: 700,
trim_end_ms: 0,
});
});
test('clamps at zero when restoring past the clip start', () => {
expect(computeTrimValues('start', -5000, 1000, 0, DURATION)).toEqual({
trim_start_ms: 0,
trim_end_ms: 0,
});
});
test('never trims below the minimum clip duration', () => {
const result = computeTrimValues('start', 99_999, 0, 2000, DURATION);
// Clamp lands exactly on the minimum, which the guard rejects.
expect(result).toBeNull();
});
test('rounds fractional millisecond deltas', () => {
expect(computeTrimValues('start', 100.6, 0, 0, DURATION)).toEqual({
trim_start_ms: 101,
trim_end_ms: 0,
});
});
test('preserves the untouched end trim', () => {
expect(computeTrimValues('start', 250, 0, 400, DURATION)).toEqual({
trim_start_ms: 250,
trim_end_ms: 400,
});
});
});
describe('end handle', () => {
test('dragging left trims from the end', () => {
expect(computeTrimValues('end', -500, 0, 0, DURATION)).toEqual({
trim_start_ms: 0,
trim_end_ms: 500,
});
});
test('dragging right restores previously trimmed audio', () => {
expect(computeTrimValues('end', 300, 0, 1000, DURATION)).toEqual({
trim_start_ms: 0,
trim_end_ms: 700,
});
});
test('clamps at zero when restoring past the clip end', () => {
expect(computeTrimValues('end', 5000, 0, 1000, DURATION)).toEqual({
trim_start_ms: 0,
trim_end_ms: 0,
});
});
test('never trims below the minimum clip duration', () => {
expect(computeTrimValues('end', -99_999, 3000, 0, DURATION)).toBeNull();
});
});
describe('minimum duration guard', () => {
test('rejects drags that leave less than the minimum audible clip', () => {
// 9.5s already trimmed; taking 450ms more leaves only 50ms.
expect(computeTrimValues('start', 450, 5000, 4500, DURATION)).toBeNull();
});
test('allows a drag that leaves just over the minimum', () => {
expect(computeTrimValues('start', 399, 5000, 4500, DURATION)).toEqual({
trim_start_ms: 5399,
trim_end_ms: 4500,
});
});
test('boundary: exactly the minimum remaining is rejected', () => {
// trim_start + trim_end === duration - MIN_CLIP_DURATION_MS
expect(
computeTrimValues('start', 400, 5000, 4500, DURATION)?.trim_start_ms ?? null,
).toBeNull();
expect(MIN_CLIP_DURATION_MS).toBe(100);
});
});
test('zero delta is a no-op that returns the initial trims', () => {
expect(computeTrimValues('start', 0, 1200, 800, DURATION)).toEqual({
trim_start_ms: 1200,
trim_end_ms: 800,
});
});
});
-57
View File
@@ -1,57 +0,0 @@
import type { StoryItemTrim } from '@/lib/api/types';
/** Clips are never allowed to shrink below this effective duration. */
export const MIN_CLIP_DURATION_MS = 100;
/**
* Computes new trim values for a clip while a trim handle is being dragged.
*
* `deltaMs` is the signed drag distance converted to milliseconds. Dragging
* the start handle right increases `trim_start_ms` (trims more from the
* start); dragging it left restores. The end handle mirrors this for
* `trim_end_ms`. Both values are clamped so the clip keeps at least
* MIN_CLIP_DURATION_MS of audible content.
*
* Returns null when the drag would leave less than the minimum duration.
*/
export function computeTrimValues(
side: 'start' | 'end',
deltaMs: number,
initialTrimStart: number,
initialTrimEnd: number,
originalDurationMs: number,
): StoryItemTrim | null {
let newTrimStart = initialTrimStart;
let newTrimEnd = initialTrimEnd;
if (side === 'start') {
newTrimStart = Math.round(
Math.max(
0,
Math.min(
initialTrimStart + deltaMs,
originalDurationMs - initialTrimEnd - MIN_CLIP_DURATION_MS,
),
),
);
} else {
newTrimEnd = Math.round(
Math.max(
0,
Math.min(
initialTrimEnd - deltaMs,
originalDurationMs - initialTrimStart - MIN_CLIP_DURATION_MS,
),
),
);
}
if (newTrimStart + newTrimEnd >= originalDurationMs - MIN_CLIP_DURATION_MS) {
return null;
}
return {
trim_start_ms: newTrimStart,
trim_end_ms: newTrimEnd,
};
}
-1
View File
@@ -60,7 +60,6 @@ export interface PlatformLifecycle {
stopServer(): Promise<void>;
restartServer(modelsDir?: string | null): Promise<string>;
setKeepServerRunning(keep: boolean): Promise<void>;
setBackendOverride(backend?: string | null): Promise<void>;
setupWindowCloseHandler(): Promise<void>;
subscribeToServerLogs(callback: (entry: ServerLogEntry) => void): () => void;
onServerReady?: () => void;
+1 -1
View File
@@ -25,7 +25,7 @@
"paths": {
"@/*": ["./src/*"]
},
"types": ["vite/client", "bun"]
"types": ["vite/client"]
},
"include": ["src"],
"references": [{ "path": "./tsconfig.node.json" }]
+13 -69
View File
@@ -3,8 +3,6 @@
import asyncio
import logging
import os
import re
import subprocess
import sys
from contextlib import asynccontextmanager
from pathlib import Path
@@ -39,75 +37,23 @@ logging.basicConfig(
logger = logging.getLogger(__name__)
# AMD GPU environment variables must be set before torch import
# Only set HSA_OVERRIDE_GFX_VERSION for older GPUs that need it.
# RDNA 3+ (gfx1100+) and RDNA 4 (gfx1200+) are natively supported by ROCm
# and the override can cause suboptimal performance or errors.
if not os.environ.get("HSA_OVERRIDE_GFX_VERSION"):
try:
result = subprocess.run(
["rocminfo"],
capture_output=True,
text=True,
timeout=5,
)
if result.returncode == 0:
# Collect all GPUs found in rocminfo output
gfx_versions = []
for line in result.stdout.splitlines():
line_lower = line.lower()
if "gfx" in line_lower:
match = re.search(r"(gfx\d+)", line_lower)
if match:
gfx_versions.append(match.group(1))
if gfx_versions:
# Check if any GPU needs the override (RDNA 2 and older)
# Use the oldest GPU (lowest gfx number) for the decision
try:
gfx_nums = []
for v in gfx_versions:
m = re.search(r"\d+", v)
if m:
gfx_nums.append(int(m.group()))
if gfx_nums:
oldest_num = min(gfx_nums)
oldest_gfx = gfx_versions[gfx_nums.index(oldest_num)]
if oldest_num < 1100:
os.environ["HSA_OVERRIDE_GFX_VERSION"] = "10.3.0"
logger.info(
"AMD GPU detected (%s), setting HSA_OVERRIDE_GFX_VERSION=10.3.0 for compatibility. All GPUs: %s",
oldest_gfx,
", ".join(gfx_versions),
)
else:
logger.info(
"AMD GPU detected (%s), native ROCm support available, skipping HSA_OVERRIDE_GFX_VERSION. All GPUs: %s",
oldest_gfx,
", ".join(gfx_versions),
)
except (ValueError, AttributeError) as e:
logger.info("Could not parse GPU version from rocminfo output: %s", e)
except (FileNotFoundError, subprocess.TimeoutExpired, Exception) as e:
logger.info(
"Could not detect AMD GPU via rocminfo, skipping automatic HSA_OVERRIDE_GFX_VERSION configuration: %s",
e,
)
os.environ["HSA_OVERRIDE_GFX_VERSION"] = "10.3.0"
if not os.environ.get("MIOPEN_LOG_LEVEL"):
os.environ["MIOPEN_LOG_LEVEL"] = "4"
from urllib.parse import quote
import torch
from fastapi import FastAPI
from fastapi.middleware.cors import CORSMiddleware
from urllib.parse import quote
from . import __version__, config, database
from .services import tts, transcribe, llm
from .database import get_db
from .routes import register_routers
from .services import llm, transcribe, tts
from .services.task_queue import create_background_task, init_queue
from .utils.platform_detect import get_backend_type
from .utils.progress import get_progress_manager
from .services.task_queue import create_background_task, init_queue
from .routes import register_routers
def safe_content_disposition(disposition_type: str, filename: str) -> str:
@@ -123,8 +69,8 @@ def safe_content_disposition(disposition_type: str, filename: str) -> str:
def create_app() -> FastAPI:
"""Create and configure the FastAPI application."""
from .mcp_server.context import ClientIdMiddleware
from .mcp_server.server import build_mcp_server, compose_lifespan
from .mcp_server.context import ClientIdMiddleware
# Build the MCP app up-front so we can wire its lifespan into FastAPI's —
# FastMCP's Streamable HTTP transport only works if its session manager
@@ -203,8 +149,8 @@ def _mount_frontend(application: FastAPI) -> None:
if not frontend_dir.is_dir():
return
from fastapi.responses import FileResponse
from fastapi.staticfiles import StaticFiles
from fastapi.responses import FileResponse
# Mount hashed assets (JS, CSS, images) that Vite places under /assets
assets_dir = frontend_dir / "assets"
@@ -244,9 +190,9 @@ def _get_gpu_status() -> str:
if not compatible:
label += " [UNSUPPORTED - see logs]"
return label
if hasattr(torch.backends, "mps") and torch.backends.mps.is_available():
elif hasattr(torch.backends, "mps") and torch.backends.mps.is_available():
return "MPS (Apple Silicon)"
if backend_type == "mlx":
elif backend_type == "mlx":
return "Metal (Apple Silicon via MLX)"
# Intel XPU (Arc / Data Center) via IPEX
@@ -303,7 +249,7 @@ async def _run_startup(application: FastAPI) -> None:
if result.rowcount > 0:
logger.info("Marked %d stale generation(s) as failed", result.rowcount)
from .database import Generation as DBGeneration, VoiceProfile as DBVoiceProfile
from .database import VoiceProfile as DBVoiceProfile, Generation as DBGeneration
profile_count = db.query(DBVoiceProfile).count()
generation_count = db.query(DBGeneration).count()
@@ -327,10 +273,8 @@ async def _run_startup(application: FastAPI) -> None:
logger.warning("GPU COMPATIBILITY: %s", _cuda_warning)
from .services.cuda import check_and_update_cuda_binary
from .services.rocm import check_and_update_rocm_binary
create_background_task(check_and_update_cuda_binary())
create_background_task(check_and_update_rocm_binary())
try:
progress_manager = get_progress_manager()
@@ -354,15 +298,15 @@ async def _run_shutdown() -> None:
"""Unload models on lifespan exit."""
logger.info("Voicebox server shutting down...")
try:
await tts.unload_tts_model()
tts.unload_tts_model()
except Exception:
logger.exception("Failed to unload TTS model")
try:
await transcribe.unload_whisper_model()
transcribe.unload_whisper_model()
except Exception:
logger.exception("Failed to unload Whisper model")
try:
await llm.unload_llm_model()
llm.unload_llm_model()
except Exception:
logger.exception("Failed to unload LLM model")
+30 -67
View File
@@ -10,14 +10,13 @@ and a model config registry that eliminates per-engine dispatch maps.
# import time, which wraps transformers' tokenizer load against the
# unconditional HuggingFace metadata call that otherwise raises on
# HF_HUB_OFFLINE=1 and on network failures.
from ..utils import hf_offline_patch # noqa: F401
import threading
from dataclasses import dataclass, field
from typing import Protocol
import numpy as np
from typing import Protocol, Optional, Tuple, List
from typing_extensions import runtime_checkable
from ..utils import hf_offline_patch
import numpy as np
DEFAULT_LLM_MAX_TOKENS = 512
DEFAULT_LLM_TEMPERATURE = 0.7
@@ -57,7 +56,6 @@ class ModelConfig:
model_size: str = "default"
size_mb: int = 0
needs_trim: bool = False
retries_runaway: bool = False
supports_instruct: bool = False
languages: list[str] = field(default_factory=lambda: ["en"])
@@ -78,7 +76,7 @@ class TTSBackend(Protocol):
audio_path: str,
reference_text: str,
use_cache: bool = True,
) -> tuple[dict, bool]:
) -> Tuple[dict, bool]:
"""
Create voice prompt from reference audio.
@@ -89,9 +87,9 @@ class TTSBackend(Protocol):
async def combine_voice_prompts(
self,
audio_paths: list[str],
reference_texts: list[str],
) -> tuple[np.ndarray, str]:
audio_paths: List[str],
reference_texts: List[str],
) -> Tuple[np.ndarray, str]:
"""
Combine multiple voice prompts.
@@ -105,9 +103,9 @@ class TTSBackend(Protocol):
text: str,
voice_prompt: dict,
language: str = "en",
seed: int | None = None,
instruct: str | None = None,
) -> tuple[np.ndarray, int]:
seed: Optional[int] = None,
instruct: Optional[str] = None,
) -> Tuple[np.ndarray, int]:
"""
Generate audio from text.
@@ -145,8 +143,8 @@ class STTBackend(Protocol):
async def transcribe(
self,
audio_path: str,
language: str | None = None,
model_size: str | None = None,
language: Optional[str] = None,
model_size: Optional[str] = None,
) -> str:
"""
Transcribe audio to text.
@@ -176,11 +174,11 @@ class LLMBackend(Protocol):
async def generate(
self,
prompt: str,
system: str | None = None,
system: Optional[str] = None,
max_tokens: int = DEFAULT_LLM_MAX_TOKENS,
temperature: float = DEFAULT_LLM_TEMPERATURE,
model_size: str | None = None,
examples: list[tuple[str, str]] | None = None,
model_size: Optional[str] = None,
examples: Optional[list[tuple[str, str]]] = None,
) -> str:
"""Run a single-turn chat completion and return the assistant reply.
@@ -200,11 +198,10 @@ class LLMBackend(Protocol):
# Global backend instances
_tts_backend: TTSBackend | None = None
_tts_backend: Optional[TTSBackend] = None
_tts_backends: dict[str, TTSBackend] = {}
_tts_backends_lock = threading.Lock()
_stt_backend: STTBackend | None = None
_stt_backend_lock = threading.Lock()
_stt_backend: Optional[STTBackend] = None
_llm_backends: dict[str, LLMBackend] = {}
_llm_backends_lock = threading.Lock()
@@ -235,10 +232,6 @@ def _get_qwen_model_configs() -> list[ModelConfig]:
repo_1_7b = "Qwen/Qwen3-TTS-12Hz-1.7B-Base"
repo_0_6b = "Qwen/Qwen3-TTS-12Hz-0.6B-Base"
# mlx-audio can continue after an EOS miss with silence followed by
# codec noise. Retry only the affected text as smaller chunks.
retries_runaway = backend_type == "mlx"
return [
ModelConfig(
model_name="qwen-tts-1.7B",
@@ -247,7 +240,6 @@ def _get_qwen_model_configs() -> list[ModelConfig]:
hf_repo_id=repo_1_7b,
model_size="1.7B",
size_mb=3500,
retries_runaway=retries_runaway,
supports_instruct=False, # Base model drops instruct silently
languages=["zh", "en", "ja", "ko", "de", "fr", "ru", "pt", "es", "it"],
),
@@ -258,7 +250,6 @@ def _get_qwen_model_configs() -> list[ModelConfig]:
hf_repo_id=repo_0_6b,
model_size="0.6B",
size_mb=1200,
retries_runaway=retries_runaway,
supports_instruct=False,
languages=["zh", "en", "ja", "ko", "de", "fr", "ru", "pt", "es", "it"],
),
@@ -497,7 +488,7 @@ def get_stt_model_configs() -> list[ModelConfig]:
# Lookup helpers — these replace the if/elif chains in main.py
def get_model_config(model_name: str) -> ModelConfig | None:
def get_model_config(model_name: str) -> Optional[ModelConfig]:
"""Look up a model config by model_name."""
for cfg in get_all_model_configs():
if cfg.model_name == model_name:
@@ -513,14 +504,6 @@ def engine_needs_trim(engine: str) -> bool:
return False
def engine_retries_runaway(engine: str) -> bool:
"""Whether unstable output should be retried in smaller chunks."""
for cfg in get_tts_model_configs():
if cfg.engine == engine:
return cfg.retries_runaway
return False
def engine_has_model_sizes(engine: str) -> bool:
"""Whether this engine supports multiple model sizes (only Qwen currently)."""
configs = [c for c in get_tts_model_configs() if c.engine == engine]
@@ -564,29 +547,15 @@ async def ensure_model_cached_or_raise(engine: str, model_size: str = "default")
)
async def unload_backend(backend) -> None:
"""Free a backend's model, serialized onto the MLX worker when it has one.
MLX backends expose an async ``unload`` that runs the free on the dedicated
MLX thread so it can't collide with an in-flight load/generate. Other
backends only carry the synchronous ``unload_model``.
"""
unload = getattr(backend, "unload", None)
if unload is not None:
await unload()
else:
backend.unload_model()
async def unload_model_by_config(config: ModelConfig) -> bool:
def unload_model_by_config(config: ModelConfig) -> bool:
"""Unload a model given its config. Returns True if it was loaded, False otherwise."""
from ..services import llm as llm_service, transcribe, tts
from . import get_tts_backend_for_engine
from ..services import tts, transcribe, llm as llm_service
if config.engine == "whisper":
whisper_model = transcribe.get_whisper_model()
if whisper_model.is_loaded() and whisper_model.model_size == config.model_size:
await unload_backend(whisper_model)
transcribe.unload_whisper_model()
return True
return False
@@ -594,7 +563,7 @@ async def unload_model_by_config(config: ModelConfig) -> bool:
backend = llm_service.get_llm_model()
loaded_size = getattr(backend, "_current_model_size", None) or getattr(backend, "model_size", None)
if backend.is_loaded() and loaded_size == config.model_size:
await unload_backend(backend)
backend.unload_model()
return True
return False
@@ -602,7 +571,7 @@ async def unload_model_by_config(config: ModelConfig) -> bool:
tts_model = tts.get_tts_model()
loaded_size = getattr(tts_model, "_current_model_size", None) or getattr(tts_model, "model_size", None)
if tts_model.is_loaded() and loaded_size == config.model_size:
await unload_backend(tts_model)
tts.unload_tts_model()
return True
return False
@@ -610,22 +579,22 @@ async def unload_model_by_config(config: ModelConfig) -> bool:
backend = get_tts_backend_for_engine(config.engine)
loaded_size = getattr(backend, "_current_model_size", None) or getattr(backend, "model_size", None)
if backend.is_loaded() and loaded_size == config.model_size:
await unload_backend(backend)
backend.unload_model()
return True
return False
# All other TTS engines
backend = get_tts_backend_for_engine(config.engine)
if backend.is_loaded():
await unload_backend(backend)
backend.unload_model()
return True
return False
def check_model_loaded(config: ModelConfig) -> bool:
"""Check if a model is currently loaded."""
from ..services import llm as llm_service, transcribe, tts
from . import get_tts_backend_for_engine
from ..services import tts, transcribe, llm as llm_service
try:
if config.engine == "whisper":
@@ -655,8 +624,8 @@ def check_model_loaded(config: ModelConfig) -> bool:
def get_model_load_func(config: ModelConfig):
"""Return a callable that loads/downloads the model."""
from ..services import llm as llm_service, transcribe, tts
from . import get_tts_backend_for_engine
from ..services import tts, transcribe, llm as llm_service
if config.engine == "whisper":
return lambda: transcribe.get_whisper_model().load_model(config.model_size)
@@ -755,13 +724,7 @@ def get_stt_backend() -> STTBackend:
"""
global _stt_backend
if _stt_backend is not None:
return _stt_backend
with _stt_backend_lock:
if _stt_backend is not None:
return _stt_backend
if _stt_backend is None:
backend_type = get_backend_type()
if backend_type == "mlx":
@@ -773,7 +736,7 @@ def get_stt_backend() -> STTBackend:
_stt_backend = PyTorchSTTBackend()
return _stt_backend
return _stt_backend
def get_llm_backend() -> LLMBackend:
+9 -13
View File
@@ -9,12 +9,13 @@ import logging
import platform
from contextlib import contextmanager
from pathlib import Path
from typing import Callable, List, Optional, Tuple
import numpy as np
from ..utils.audio import load_audio, normalize_audio
from ..utils.hf_progress import HFProgressTracker, create_hf_progress_callback
from ..utils.audio import normalize_audio, load_audio
from ..utils.progress import get_progress_manager
from ..utils.hf_progress import HFProgressTracker, create_hf_progress_callback
from ..utils.tasks import get_task_manager
logger = logging.getLogger(__name__)
@@ -24,7 +25,7 @@ def is_model_cached(
hf_repo: str,
*,
weight_extensions: tuple[str, ...] = (".safetensors", ".bin"),
required_files: list[str] | None = None,
required_files: Optional[list[str]] = None,
) -> bool:
"""
Check if a HuggingFace model is fully cached locally.
@@ -137,11 +138,6 @@ def check_cuda_compatibility() -> tuple[bool, str | None]:
if not torch.cuda.is_available():
return True, None
# ROCm/HIP uses the cuda frontend but has different architecture names (gfx*).
# Skip NVIDIA-specific compute capability checks on AMD hardware.
if hasattr(torch.version, "hip") and torch.version.hip:
return True, None
major, minor = torch.cuda.get_device_capability(0)
capability = f"{major}.{minor}"
device_name = torch.cuda.get_device_name(0)
@@ -200,11 +196,11 @@ def manual_seed(seed: int, device: str) -> None:
async def combine_voice_prompts(
audio_paths: list[str],
reference_texts: list[str],
audio_paths: List[str],
reference_texts: List[str],
*,
sample_rate: int | None = None,
) -> tuple[np.ndarray, str]:
sample_rate: Optional[int] = None,
) -> Tuple[np.ndarray, str]:
"""
Combine multiple reference audio samples into one.
@@ -234,7 +230,7 @@ async def combine_voice_prompts(
def model_load_progress(
model_name: str,
is_cached: bool,
filter_non_downloads: bool | None = None,
filter_non_downloads: Optional[bool] = None,
):
"""
Context manager for model loading with HF download progress tracking.
+12 -11
View File
@@ -10,16 +10,17 @@ import asyncio
import logging
import threading
from pathlib import Path
from typing import ClassVar
from typing import ClassVar, List, Optional, Tuple
import numpy as np
from . import TTSBackend
from .base import (
combine_voice_prompts as _combine_voice_prompts,
empty_device_cache,
get_torch_device,
is_model_cached,
get_torch_device,
empty_device_cache,
manual_seed,
combine_voice_prompts as _combine_voice_prompts,
model_load_progress,
patch_chatterbox_f32,
)
@@ -126,7 +127,7 @@ class ChatterboxTTSBackend:
audio_path: str,
reference_text: str,
use_cache: bool = True,
) -> tuple[dict, bool]:
) -> Tuple[dict, bool]:
"""
Create voice prompt from reference audio.
@@ -142,9 +143,9 @@ class ChatterboxTTSBackend:
async def combine_voice_prompts(
self,
audio_paths: list[str],
reference_texts: list[str],
) -> tuple[np.ndarray, str]:
audio_paths: List[str],
reference_texts: List[str],
) -> Tuple[np.ndarray, str]:
return await _combine_voice_prompts(audio_paths, reference_texts)
# Per-language generation defaults. Lower temp + higher cfg = clearer speech.
@@ -168,9 +169,9 @@ class ChatterboxTTSBackend:
text: str,
voice_prompt: dict,
language: str = "en",
seed: int | None = None,
instruct: str | None = None,
) -> tuple[np.ndarray, int]:
seed: Optional[int] = None,
instruct: Optional[str] = None,
) -> Tuple[np.ndarray, int]:
"""
Generate audio using Chatterbox Multilingual TTS.
+13 -12
View File
@@ -10,16 +10,17 @@ import asyncio
import logging
import threading
from pathlib import Path
from typing import ClassVar
from typing import ClassVar, List, Optional, Tuple
import numpy as np
from . import TTSBackend
from .base import (
combine_voice_prompts as _combine_voice_prompts,
empty_device_cache,
get_torch_device,
is_model_cached,
get_torch_device,
empty_device_cache,
manual_seed,
combine_voice_prompts as _combine_voice_prompts,
model_load_progress,
patch_chatterbox_f32,
)
@@ -80,8 +81,8 @@ class ChatterboxTurboTTSBackend:
logger.info(f"Loading Chatterbox Turbo TTS on {device}...")
import torch
from chatterbox.tts_turbo import ChatterboxTurboTTS
from huggingface_hub import snapshot_download
from chatterbox.tts_turbo import ChatterboxTurboTTS
local_path = snapshot_download(
repo_id=CHATTERBOX_TURBO_HF_REPO,
@@ -125,7 +126,7 @@ class ChatterboxTurboTTSBackend:
audio_path: str,
reference_text: str,
use_cache: bool = True,
) -> tuple[dict, bool]:
) -> Tuple[dict, bool]:
"""
Create voice prompt from reference audio.
@@ -140,9 +141,9 @@ class ChatterboxTurboTTSBackend:
async def combine_voice_prompts(
self,
audio_paths: list[str],
reference_texts: list[str],
) -> tuple[np.ndarray, str]:
audio_paths: List[str],
reference_texts: List[str],
) -> Tuple[np.ndarray, str]:
return await _combine_voice_prompts(audio_paths, reference_texts)
async def generate(
@@ -150,9 +151,9 @@ class ChatterboxTurboTTSBackend:
text: str,
voice_prompt: dict,
language: str = "en",
seed: int | None = None,
instruct: str | None = None,
) -> tuple[np.ndarray, int]:
seed: Optional[int] = None,
instruct: Optional[str] = None,
) -> Tuple[np.ndarray, int]:
"""
Generate audio using Chatterbox Turbo TTS.
+21 -30
View File
@@ -16,19 +16,20 @@ causal LM generates speech via flow-matching diffusion.
import asyncio
import logging
import threading
from typing import ClassVar
from typing import ClassVar, List, Optional, Tuple
import numpy as np
from ..utils.cache import cache_voice_prompt, get_cache_key, get_cached_voice_prompt
from . import TTSBackend
from .base import (
combine_voice_prompts as _combine_voice_prompts,
empty_device_cache,
get_torch_device,
is_model_cached,
get_torch_device,
empty_device_cache,
manual_seed,
combine_voice_prompts as _combine_voice_prompts,
model_load_progress,
)
from ..utils.cache import get_cache_key, get_cached_voice_prompt, cache_voice_prompt
logger = logging.getLogger(__name__)
@@ -145,15 +146,7 @@ class HumeTadaBackend:
)
# Determine dtype — use bf16 on CUDA/XPU for ~50% memory savings
# On ROCm/AMD, torch.cuda.is_bf16_supported() works via the HIP abstraction,
# but we wrap it defensively in case an older build lacks the symbol.
_bf16_ok = False
if device == "cuda":
try:
_bf16_ok = torch.cuda.is_bf16_supported()
except Exception:
_bf16_ok = False
if _bf16_ok:
if device == "cuda" and torch.cuda.is_bf16_supported():
model_dtype = torch.bfloat16
elif device == "xpu":
# Intel Arc (Alchemist+) supports bf16 natively
@@ -181,7 +174,7 @@ class HumeTadaBackend:
# getattr(config, "tokenizer_name", "meta-llama/Llama-3.2-1B")
# which hits the gated repo. Pre-load the config from HF,
# inject the local tokenizer path, then pass it in.
from tada.modules.tada import TadaConfig, TadaForCausalLM
from tada.modules.tada import TadaForCausalLM, TadaConfig
logger.info(f"Loading TADA {model_size} model...")
config = TadaConfig.from_pretrained(repo)
@@ -213,7 +206,7 @@ class HumeTadaBackend:
audio_path: str,
reference_text: str,
use_cache: bool = True,
) -> tuple[dict, bool]:
) -> Tuple[dict, bool]:
"""
Create voice prompt from reference audio using TADA's encoder.
@@ -233,8 +226,8 @@ class HumeTadaBackend:
return cached, True
def _encode_sync():
import soundfile as sf
import torch
import soundfile as sf
device = self._device
@@ -247,13 +240,9 @@ class HumeTadaBackend:
audio = audio.T # (samples, channels) -> (channels, samples)
audio = audio.to(device)
# Encode with forced alignment.
# Must run under inference_mode: encoder params still require
# grad by default, and an autograd graph across the DAC/Snake
# stack can balloon VRAM far past the model footprint (#890).
# Encode with forced alignment
text_arg = [reference_text] if reference_text else None
with torch.inference_mode():
prompt = self.encoder(audio, text=text_arg, sample_rate=sr)
prompt = self.encoder(audio, text=text_arg, sample_rate=sr)
# Serialize EncoderOutput to a dict of CPU tensors for caching
prompt_dict = {}
@@ -261,7 +250,9 @@ class HumeTadaBackend:
val = getattr(prompt, field_name)
if isinstance(val, torch.Tensor):
prompt_dict[field_name] = val.detach().cpu()
elif isinstance(val, (list, int, float)):
elif isinstance(val, list):
prompt_dict[field_name] = val
elif isinstance(val, (int, float)):
prompt_dict[field_name] = val
else:
prompt_dict[field_name] = val
@@ -276,9 +267,9 @@ class HumeTadaBackend:
async def combine_voice_prompts(
self,
audio_paths: list[str],
reference_texts: list[str],
) -> tuple[np.ndarray, str]:
audio_paths: List[str],
reference_texts: List[str],
) -> Tuple[np.ndarray, str]:
return await _combine_voice_prompts(audio_paths, reference_texts, sample_rate=24000)
async def generate(
@@ -286,9 +277,9 @@ class HumeTadaBackend:
text: str,
voice_prompt: dict,
language: str = "en",
seed: int | None = None,
instruct: str | None = None,
) -> tuple[np.ndarray, int]:
seed: Optional[int] = None,
instruct: Optional[str] = None,
) -> Tuple[np.ndarray, int]:
"""
Generate audio from text using HumeAI TADA.
+8 -9
View File
@@ -17,12 +17,15 @@ Languages supported (via misaki G2P):
import asyncio
import logging
import os
from typing import Optional
import numpy as np
from . import TTSBackend
from .base import (
combine_voice_prompts as _combine_voice_prompts,
get_torch_device,
combine_voice_prompts as _combine_voice_prompts,
model_load_progress,
)
@@ -119,9 +122,8 @@ class KokoroTTSBackend:
def __init__(self):
self._model = None
self._pipelines: dict = {} # lang_code -> KPipeline
self._device: str | None = None
self._device: Optional[str] = None
self.model_size = "default"
self._model_load_lock = asyncio.Lock()
def _get_device(self) -> str:
"""Select device. Kokoro supports CUDA and CPU. MPS needs fallback env var."""
@@ -155,10 +157,7 @@ class KokoroTTSBackend:
"""Load the Kokoro model."""
if self._model is not None:
return
async with self._model_load_lock:
if self._model is not None:
return
await asyncio.to_thread(self._load_model_sync)
await asyncio.to_thread(self._load_model_sync)
def _load_model_sync(self):
"""Synchronous model loading."""
@@ -240,8 +239,8 @@ class KokoroTTSBackend:
text: str,
voice_prompt: dict,
language: str = "en",
seed: int | None = None,
instruct: str | None = None,
seed: Optional[int] = None,
instruct: Optional[str] = None,
) -> tuple[np.ndarray, int]:
"""
Generate audio from text using Kokoro.
+12 -13
View File
@@ -7,18 +7,20 @@ Wraps the LuxTTS (ZipVoice) model for zero-shot voice cloning.
import asyncio
import logging
from typing import Optional, Tuple
import numpy as np
from ..utils.cache import cache_voice_prompt, get_cache_key, get_cached_voice_prompt
from . import TTSBackend
from .base import (
combine_voice_prompts as _combine_voice_prompts,
empty_device_cache,
get_torch_device,
is_model_cached,
get_torch_device,
empty_device_cache,
manual_seed,
combine_voice_prompts as _combine_voice_prompts,
model_load_progress,
)
from ..utils.cache import get_cache_key, get_cached_voice_prompt, cache_voice_prompt
logger = logging.getLogger(__name__)
@@ -33,7 +35,6 @@ class LuxTTSBackend:
self.model = None
self.model_size = "default" # LuxTTS has only one model size
self._device = None
self._model_load_lock = asyncio.Lock()
def _get_device(self) -> str:
return get_torch_device(allow_mps=True, allow_xpu=True)
@@ -60,10 +61,8 @@ class LuxTTSBackend:
"""Load the LuxTTS model."""
if self.model is not None:
return
async with self._model_load_lock:
if self.model is not None:
return
await asyncio.to_thread(self._load_model_sync)
await asyncio.to_thread(self._load_model_sync)
def _load_model_sync(self):
model_name = "luxtts"
@@ -106,7 +105,7 @@ class LuxTTSBackend:
audio_path: str,
reference_text: str,
use_cache: bool = True,
) -> tuple[dict, bool]:
) -> Tuple[dict, bool]:
"""
Create voice prompt from reference audio.
@@ -146,9 +145,9 @@ class LuxTTSBackend:
text: str,
voice_prompt: dict,
language: str = "en",
seed: int | None = None,
instruct: str | None = None,
) -> tuple[np.ndarray, int]:
seed: Optional[int] = None,
instruct: Optional[str] = None,
) -> Tuple[np.ndarray, int]:
"""
Generate audio from text using LuxTTS.
+78 -91
View File
@@ -2,24 +2,24 @@
MLX backend implementation for TTS and STT using mlx-audio.
"""
from typing import Optional, List, Tuple
import asyncio
import logging
from pathlib import Path
import numpy as np
from pathlib import Path
logger = logging.getLogger(__name__)
# PATCH: Import and apply offline patch BEFORE any huggingface_hub usage
# This prevents mlx_audio from making network requests when models are cached
from ..utils.hf_offline_patch import ensure_original_qwen_config_cached, patch_huggingface_hub_offline
from ..utils.hf_offline_patch import patch_huggingface_hub_offline, ensure_original_qwen_config_cached
patch_huggingface_hub_offline()
ensure_original_qwen_config_cached()
from ..services.mlx_thread import clear_mlx_cache, run_on_mlx_thread
from ..utils.cache import cache_voice_prompt, get_cache_key, get_cached_voice_prompt
from . import LANGUAGE_CODE_TO_NAME, WHISPER_HF_REPOS
from .base import combine_voice_prompts as _combine_voice_prompts, is_model_cached, model_load_progress
from . import TTSBackend, STTBackend, LANGUAGE_CODE_TO_NAME, WHISPER_HF_REPOS
from .base import is_model_cached, combine_voice_prompts as _combine_voice_prompts, model_load_progress
from ..utils.cache import get_cache_key, get_cached_voice_prompt, cache_voice_prompt
class MLXTTSBackend:
@@ -63,38 +63,30 @@ class MLXTTSBackend:
weight_extensions=(".safetensors", ".bin", ".npz"),
)
def _ensure_loaded_sync(self, model_size: str | None):
"""Load the model if the requested size isn't already resident.
Runs on the MLX worker thread so it stays serialized with generation.
"""
if model_size is None:
model_size = self.model_size
if self.model is not None and self._current_model_size == model_size:
return
if self.model is not None and self._current_model_size != model_size:
self.unload_model()
self._load_model_sync(model_size)
async def load_model_async(self, model_size: str | None = None):
async def load_model_async(self, model_size: Optional[str] = None):
"""
Lazy load the MLX TTS model.
Args:
model_size: Model size to load (1.7B or 0.6B)
"""
await run_on_mlx_thread(self._ensure_loaded_sync, model_size)
if model_size is None:
model_size = self.model_size
# If already loaded with correct size, return
if self.model is not None and self._current_model_size == model_size:
return
# Unload existing model if different size requested
if self.model is not None and self._current_model_size != model_size:
self.unload_model()
# Run blocking load in thread pool
await asyncio.to_thread(self._load_model_sync, model_size)
# Alias for compatibility
load_model = load_model_async
async def unload(self):
"""Free the model, serialized onto the MLX worker thread."""
await run_on_mlx_thread(self.unload_model)
def _load_model_sync(self, model_size: str):
"""Synchronous model loading."""
model_path = self._get_model_path(model_size)
@@ -118,7 +110,6 @@ class MLXTTSBackend:
del self.model
self.model = None
self._current_model_size = None
clear_mlx_cache()
logger.info("MLX TTS model unloaded")
async def create_voice_prompt(
@@ -126,7 +117,7 @@ class MLXTTSBackend:
audio_path: str,
reference_text: str,
use_cache: bool = True,
) -> tuple[dict, bool]:
) -> Tuple[dict, bool]:
"""
Create voice prompt from reference audio.
@@ -154,8 +145,9 @@ class MLXTTSBackend:
cached_audio_path = cached_prompt.get("ref_audio") or cached_prompt.get("ref_audio_path")
if cached_audio_path and Path(cached_audio_path).exists():
return cached_prompt, True
# Cached file no longer exists, invalidate cache
logger.warning("Cached audio file not found: %s, regenerating prompt", cached_audio_path)
else:
# Cached file no longer exists, invalidate cache
logger.warning("Cached audio file not found: %s, regenerating prompt", cached_audio_path)
# MLX voice prompt format - store audio path and text
# The model will process this during generation
@@ -179,9 +171,9 @@ class MLXTTSBackend:
text: str,
voice_prompt: dict,
language: str = "en",
seed: int | None = None,
instruct: str | None = None,
) -> tuple[np.ndarray, int]:
seed: Optional[int] = None,
instruct: Optional[str] = None,
) -> Tuple[np.ndarray, int]:
"""
Generate audio from text using voice prompt.
@@ -195,6 +187,8 @@ class MLXTTSBackend:
Returns:
Tuple of (audio_array, sample_rate)
"""
await self.load_model_async(None)
logger.info("Generating audio for text: %s", text)
def _generate_sync():
@@ -227,39 +221,45 @@ class MLXTTSBackend:
# mlx_audio lookups hanging when the network drops mid-inference,
# issue #462) regressed online users because libraries make
# legitimate metadata calls during generation.
# A cloning failure surfaces as a failed generation; substituting
# the model's default voice would silently break the clone the
# user asked for.
if ref_audio:
import inspect
try:
if ref_audio:
# Check if generate accepts ref_audio parameter
import inspect
sig = inspect.signature(self.model.generate)
if "ref_audio" not in sig.parameters:
raise RuntimeError(
"Loaded MLX model does not support voice cloning "
"(generate() has no ref_audio parameter)"
)
for result in self.model.generate(text, ref_audio=ref_audio, ref_text=ref_text, lang_code=lang):
audio_chunks.append(np.array(result.audio))
sample_rate = result.sample_rate
else:
sig = inspect.signature(self.model.generate)
if "ref_audio" in sig.parameters:
# Generate with voice cloning
for result in self.model.generate(text, ref_audio=ref_audio, ref_text=ref_text, lang_code=lang):
audio_chunks.append(np.array(result.audio))
sample_rate = result.sample_rate
else:
# Fallback: generate without voice cloning
for result in self.model.generate(text, lang_code=lang):
audio_chunks.append(np.array(result.audio))
sample_rate = result.sample_rate
else:
# No voice prompt, generate normally
for result in self.model.generate(text, lang_code=lang):
audio_chunks.append(np.array(result.audio))
sample_rate = result.sample_rate
except Exception as e:
# If voice cloning fails, try without it
logger.warning("Voice cloning failed, generating without voice prompt: %s", e)
for result in self.model.generate(text, lang_code=lang):
audio_chunks.append(np.array(result.audio))
sample_rate = result.sample_rate
if not audio_chunks:
raise RuntimeError("Model produced no audio")
# Concatenate all chunks
if audio_chunks:
audio = np.concatenate([np.asarray(chunk, dtype=np.float32) for chunk in audio_chunks])
else:
# Fallback: empty audio
audio = np.array([], dtype=np.float32)
audio = np.concatenate([np.asarray(chunk, dtype=np.float32) for chunk in audio_chunks])
return audio, sample_rate
# Load-if-needed and inference run as one job on the MLX worker so a
# concurrent unload or different-size load can't land between them.
def _load_and_generate():
self._ensure_loaded_sync(None)
return _generate_sync()
audio, sample_rate = await run_on_mlx_thread(_load_and_generate)
# Run blocking inference in thread pool
audio, sample_rate = await asyncio.to_thread(_generate_sync)
return audio, sample_rate
@@ -279,10 +279,12 @@ class MLXSTTBackend:
hf_repo = WHISPER_HF_REPOS.get(model_size, f"openai/whisper-{model_size}")
return is_model_cached(hf_repo, weight_extensions=(".safetensors", ".bin", ".npz"))
def _ensure_loaded_sync(self, model_size: str | None):
"""Load the model if the requested size isn't already resident.
async def load_model_async(self, model_size: Optional[str] = None):
"""
Lazy load the MLX Whisper model.
Runs on the MLX worker thread so it stays serialized with transcription.
Args:
model_size: Model size (tiny, base, small, medium, large)
"""
if model_size is None:
model_size = self.model_size
@@ -290,24 +292,12 @@ class MLXSTTBackend:
if self.model is not None and self.model_size == model_size:
return
self._load_model_sync(model_size)
async def load_model_async(self, model_size: str | None = None):
"""
Lazy load the MLX Whisper model.
Args:
model_size: Model size (tiny, base, small, medium, large)
"""
await run_on_mlx_thread(self._ensure_loaded_sync, model_size)
# Run blocking load in thread pool
await asyncio.to_thread(self._load_model_sync, model_size)
# Alias for compatibility
load_model = load_model_async
async def unload(self):
"""Free the model, serialized onto the MLX worker thread."""
await run_on_mlx_thread(self.unload_model)
def _load_model_sync(self, model_size: str):
"""Synchronous model loading."""
progress_model_name = f"whisper-{model_size}"
@@ -329,14 +319,13 @@ class MLXSTTBackend:
if self.model is not None:
del self.model
self.model = None
clear_mlx_cache()
logger.info("MLX Whisper model unloaded")
async def transcribe(
self,
audio_path: str,
language: str | None = None,
model_size: str | None = None,
language: Optional[str] = None,
model_size: Optional[str] = None,
) -> str:
"""
Transcribe audio to text.
@@ -349,6 +338,8 @@ class MLXSTTBackend:
Returns:
Transcribed text
"""
await self.load_model_async(model_size)
def _transcribe_sync():
"""Run synchronous transcription in thread pool."""
# MLX Whisper transcription using generate method
@@ -365,16 +356,12 @@ class MLXSTTBackend:
# Extract text from result
if isinstance(result, str):
return result.strip()
if isinstance(result, dict):
elif isinstance(result, dict):
return result.get("text", "").strip()
if hasattr(result, "text"):
elif hasattr(result, "text"):
return result.text.strip()
return str(result).strip()
else:
return str(result).strip()
# Load-if-needed and transcription run as one job on the MLX worker so
# a concurrent unload or load can't land between them.
def _load_and_transcribe():
self._ensure_loaded_sync(model_size)
return _transcribe_sync()
return await run_on_mlx_thread(_load_and_transcribe)
# Run blocking transcription in thread pool
return await asyncio.to_thread(_transcribe_sync)
+21 -21
View File
@@ -2,25 +2,25 @@
PyTorch backend implementation for TTS and STT.
"""
from typing import Optional, List, Tuple
import asyncio
import logging
import numpy as np
import torch
import numpy as np
logger = logging.getLogger(__name__)
from ..utils.audio import load_audio
from ..utils.cache import cache_voice_prompt, get_cache_key, get_cached_voice_prompt
from . import LANGUAGE_CODE_TO_NAME, WHISPER_HF_REPOS
from . import TTSBackend, STTBackend, LANGUAGE_CODE_TO_NAME, WHISPER_HF_REPOS
from .base import (
combine_voice_prompts as _combine_voice_prompts,
empty_device_cache,
get_torch_device,
is_model_cached,
get_torch_device,
empty_device_cache,
manual_seed,
combine_voice_prompts as _combine_voice_prompts,
model_load_progress,
)
from ..utils.cache import get_cache_key, get_cached_voice_prompt, cache_voice_prompt
from ..utils.audio import load_audio
class PyTorchTTSBackend:
@@ -63,7 +63,7 @@ class PyTorchTTSBackend:
def _is_model_cached(self, model_size: str) -> bool:
return is_model_cached(self._get_model_path(model_size))
async def load_model_async(self, model_size: str | None = None):
async def load_model_async(self, model_size: Optional[str] = None):
"""
Lazy load the TTS model with automatic downloading from HuggingFace Hub.
@@ -140,7 +140,7 @@ class PyTorchTTSBackend:
audio_path: str,
reference_text: str,
use_cache: bool = True,
) -> tuple[dict, bool]:
) -> Tuple[dict, bool]:
"""
Create voice prompt from reference audio.
@@ -165,7 +165,7 @@ class PyTorchTTSBackend:
# For PyTorch backend, the dict should contain tensors, not file paths
# So we can safely return it
return cached_prompt, True
if isinstance(cached_prompt, torch.Tensor):
elif isinstance(cached_prompt, torch.Tensor):
# Legacy cache format - convert to dict
# This shouldn't happen in practice, but handle it
return {"prompt": cached_prompt}, True
@@ -194,9 +194,9 @@ class PyTorchTTSBackend:
async def combine_voice_prompts(
self,
audio_paths: list[str],
reference_texts: list[str],
) -> tuple[np.ndarray, str]:
audio_paths: List[str],
reference_texts: List[str],
) -> Tuple[np.ndarray, str]:
return await _combine_voice_prompts(audio_paths, reference_texts)
async def generate(
@@ -204,9 +204,9 @@ class PyTorchTTSBackend:
text: str,
voice_prompt: dict,
language: str = "en",
seed: int | None = None,
instruct: str | None = None,
) -> tuple[np.ndarray, int]:
seed: Optional[int] = None,
instruct: Optional[str] = None,
) -> Tuple[np.ndarray, int]:
"""
Generate audio from text using voice prompt.
@@ -266,7 +266,7 @@ class PyTorchSTTBackend:
hf_repo = WHISPER_HF_REPOS.get(model_size, f"openai/whisper-{model_size}")
return is_model_cached(hf_repo)
async def load_model_async(self, model_size: str | None = None):
async def load_model_async(self, model_size: Optional[str] = None):
"""
Lazy load the Whisper model.
@@ -290,7 +290,7 @@ class PyTorchSTTBackend:
is_cached = self._is_model_cached(model_size)
with model_load_progress(progress_model_name, is_cached):
from transformers import WhisperForConditionalGeneration, WhisperProcessor
from transformers import WhisperProcessor, WhisperForConditionalGeneration
model_name = WHISPER_HF_REPOS.get(model_size, f"openai/whisper-{model_size}")
logger.info("Loading Whisper model %s on %s...", model_size, self.device)
@@ -317,8 +317,8 @@ class PyTorchSTTBackend:
async def transcribe(
self,
audio_path: str,
language: str | None = None,
model_size: str | None = None,
language: Optional[str] = None,
model_size: Optional[str] = None,
) -> str:
"""
Transcribe audio to text.
@@ -16,15 +16,16 @@ Languages supported: zh, en, ja, ko, de, fr, ru, pt, es, it
import asyncio
import logging
from typing import Optional
import numpy as np
import torch
from . import LANGUAGE_CODE_TO_NAME
from . import TTSBackend, LANGUAGE_CODE_TO_NAME
from .base import (
combine_voice_prompts as _combine_voice_prompts,
get_torch_device,
is_model_cached,
get_torch_device,
combine_voice_prompts as _combine_voice_prompts,
model_load_progress,
)
@@ -61,7 +62,7 @@ class QwenCustomVoiceBackend:
self.model = None
self.model_size = model_size
self.device = self._get_device()
self._current_model_size: str | None = None
self._current_model_size: Optional[str] = None
def _get_device(self) -> str:
return get_torch_device(allow_xpu=True, allow_directml=True)
@@ -74,11 +75,11 @@ class QwenCustomVoiceBackend:
raise ValueError(f"Unknown model size: {model_size}")
return QWEN_CV_HF_REPOS[model_size]
def _is_model_cached(self, model_size: str | None = None) -> bool:
def _is_model_cached(self, model_size: Optional[str] = None) -> bool:
size = model_size or self.model_size
return is_model_cached(self._get_model_path(size))
async def load_model_async(self, model_size: str | None = None) -> None:
async def load_model_async(self, model_size: Optional[str] = None) -> None:
if model_size is None:
model_size = self.model_size
@@ -163,8 +164,8 @@ class QwenCustomVoiceBackend:
text: str,
voice_prompt: dict,
language: str = "en",
seed: int | None = None,
instruct: str | None = None,
seed: Optional[int] = None,
instruct: Optional[str] = None,
) -> tuple[np.ndarray, int]:
"""
Generate audio using Qwen CustomVoice.
+27 -41
View File
@@ -9,16 +9,17 @@ and STT engines.
import asyncio
import logging
from typing import Optional
from ..services.mlx_thread import clear_mlx_cache, run_on_mlx_thread
from ..utils.hf_offline_patch import force_offline_if_cached
from . import DEFAULT_LLM_MAX_TOKENS, DEFAULT_LLM_TEMPERATURE
from . import LLMBackend, DEFAULT_LLM_MAX_TOKENS, DEFAULT_LLM_TEMPERATURE
from .base import (
empty_device_cache,
get_torch_device,
is_model_cached,
get_torch_device,
empty_device_cache,
manual_seed,
model_load_progress,
)
from ..utils.hf_offline_patch import force_offline_if_cached
logger = logging.getLogger(__name__)
@@ -42,8 +43,8 @@ def _progress_name(model_size: str) -> str:
def _build_messages(
prompt: str,
system: str | None,
examples: list[tuple[str, str]] | None = None,
system: Optional[str],
examples: Optional[list[tuple[str, str]]] = None,
) -> list[dict]:
messages: list[dict] = []
if system:
@@ -63,7 +64,7 @@ class PyTorchQwenLLMBackend:
self.model = None
self.tokenizer = None
self.model_size = model_size
self._current_model_size: str | None = None
self._current_model_size: Optional[str] = None
self.device = self._get_device()
def _get_device(self) -> str:
@@ -80,7 +81,7 @@ class PyTorchQwenLLMBackend:
def _is_model_cached(self, model_size: str) -> bool:
return is_model_cached(self._get_model_path(model_size))
async def load_model(self, model_size: str | None = None) -> None:
async def load_model(self, model_size: Optional[str] = None) -> None:
if model_size is None:
model_size = self.model_size
@@ -130,11 +131,11 @@ class PyTorchQwenLLMBackend:
async def generate(
self,
prompt: str,
system: str | None = None,
system: Optional[str] = None,
max_tokens: int = DEFAULT_LLM_MAX_TOKENS,
temperature: float = DEFAULT_LLM_TEMPERATURE,
model_size: str | None = None,
examples: list[tuple[str, str]] | None = None,
model_size: Optional[str] = None,
examples: Optional[list[tuple[str, str]]] = None,
) -> str:
await self.load_model(model_size)
return await asyncio.to_thread(
@@ -144,10 +145,10 @@ class PyTorchQwenLLMBackend:
def _generate_sync(
self,
prompt: str,
system: str | None,
system: Optional[str],
max_tokens: int,
temperature: float,
examples: list[tuple[str, str]] | None = None,
examples: Optional[list[tuple[str, str]]] = None,
) -> str:
import torch
@@ -185,7 +186,7 @@ class MLXQwenLLMBackend:
self.model = None
self.tokenizer = None
self.model_size = model_size
self._current_model_size: str | None = None
self._current_model_size: Optional[str] = None
def is_loaded(self) -> bool:
return self.model is not None
@@ -201,11 +202,7 @@ class MLXQwenLLMBackend:
weight_extensions=(".safetensors", ".bin", ".npz"),
)
def _ensure_loaded_sync(self, model_size: str | None) -> None:
"""Load the model if the requested size isn't already resident.
Runs on the MLX worker thread so it stays serialized with generation.
"""
async def load_model(self, model_size: Optional[str] = None) -> None:
if model_size is None:
model_size = self.model_size
@@ -215,14 +212,7 @@ class MLXQwenLLMBackend:
if self.model is not None and self._current_model_size != model_size:
self.unload_model()
self._load_model_sync(model_size)
async def load_model(self, model_size: str | None = None) -> None:
await run_on_mlx_thread(self._ensure_loaded_sync, model_size)
async def unload(self) -> None:
"""Free the model, serialized onto the MLX worker thread."""
await run_on_mlx_thread(self.unload_model)
await asyncio.to_thread(self._load_model_sync, model_size)
def _load_model_sync(self, model_size: str) -> None:
from mlx_lm import load as mlx_load
@@ -253,33 +243,29 @@ class MLXQwenLLMBackend:
self.model = None
self.tokenizer = None
self._current_model_size = None
clear_mlx_cache()
logger.info("Qwen3 (MLX) unloaded")
async def generate(
self,
prompt: str,
system: str | None = None,
system: Optional[str] = None,
max_tokens: int = DEFAULT_LLM_MAX_TOKENS,
temperature: float = DEFAULT_LLM_TEMPERATURE,
model_size: str | None = None,
examples: list[tuple[str, str]] | None = None,
model_size: Optional[str] = None,
examples: Optional[list[tuple[str, str]]] = None,
) -> str:
# Load-if-needed and inference run as one job on the MLX worker so a
# concurrent unload or different-size load can't land between them.
def _load_and_generate() -> str:
self._ensure_loaded_sync(model_size)
return self._generate_sync(prompt, system, max_tokens, temperature, examples)
return await run_on_mlx_thread(_load_and_generate)
await self.load_model(model_size)
return await asyncio.to_thread(
self._generate_sync, prompt, system, max_tokens, temperature, examples
)
def _generate_sync(
self,
prompt: str,
system: str | None,
system: Optional[str],
max_tokens: int,
temperature: float,
examples: list[tuple[str, str]] | None = None,
examples: Optional[list[tuple[str, str]]] = None,
) -> str:
from mlx_lm import generate as mlx_generate
from mlx_lm.sample_utils import make_sampler
+54 -249
View File
@@ -22,34 +22,24 @@ def is_apple_silicon():
return platform.system() == "Darwin" and platform.machine() == "arm64"
def build_server(cuda=False, rocm=False):
def build_server(cuda=False):
"""Build Python server as standalone binary.
Args:
cuda: If True, build with CUDA support and name the binary
voicebox-server-cuda instead of voicebox-server.
rocm: If True, build with ROCm support and name the binary
voicebox-server-rocm instead of voicebox-server.
"""
if cuda and rocm:
raise ValueError("Cannot build with both CUDA and ROCm support")
backend_dir = Path(__file__).parent
if rocm:
binary_name = "voicebox-server-rocm"
elif cuda:
binary_name = "voicebox-server-cuda"
else:
binary_name = "voicebox-server"
binary_name = "voicebox-server-cuda" if cuda else "voicebox-server"
# PyInstaller arguments
# CUDA and ROCm builds use --onedir so we can split the output into two archives:
# CUDA builds use --onedir so we can split the output into two archives:
# 1. Server core (~200-400MB) — versioned with the app
# 2. GPU libs (~2GB) — versioned independently (only redownloaded on
# GPU toolkit / torch major version changes)
# 2. CUDA libs (~2GB) — versioned independently (only redownloaded on
# CUDA toolkit / torch major version changes)
# CPU builds remain --onefile for simplicity.
pack_mode = "--onedir" if (cuda or rocm) else "--onefile"
pack_mode = "--onedir" if cuda else "--onefile"
args = [
"server.py", # Use server.py as entry point instead of main.py
pack_mode,
@@ -330,74 +320,22 @@ def build_server(cuda=False, rocm=False):
]
)
# Add CUDA/ROCm-specific hidden imports
if cuda or rocm:
variant = "ROCm" if rocm else "CUDA"
logger.info("Building with %s support", variant)
gpu_hidden = [
"--hidden-import",
"torch.cuda",
]
# cudnn is NVIDIA-specific; ROCm uses MIOpen under the abstraction layer
if cuda:
gpu_hidden.extend(
[
"--hidden-import",
"torch.backends.cudnn",
]
)
args.extend(gpu_hidden)
if rocm:
# rocm_sdk imports its backend packages dynamically via
# importlib.import_module(py_package_name), which PyInstaller's
# static analyzer cannot see. We must collect them explicitly —
# otherwise only the pure-python rocm_sdk wrapper ships and
# rocm_sdk.find_libraries crashes with UnboundLocalError at boot.
#
# The backend packages also contain the HIP/MIOpen/hipBLAS DLLs
# under bin/ (plus ~750 MB of tensile kernel files under
# bin/rocblas/library and bin/hipblaslt/library) — collect-all
# walks the tree recursively so both DLLs and kernel data are
# bundled. See rocm_sdk/_dist_info.py for the package mapping.
# Add CUDA-specific hidden imports
if cuda:
logger.info("Building with CUDA support")
args.extend(
[
"--collect-all",
"rocm_sdk",
"--collect-all",
"_rocm_sdk_core",
"--collect-all",
"_rocm_sdk_libraries_custom",
"--collect-all",
"rocm_sdk_core",
"--collect-all",
"rocm_sdk_libraries_custom",
"--hidden-import",
"_rocm_sdk_core",
"torch.cuda",
"--hidden-import",
"_rocm_sdk_libraries_custom",
"--hidden-import",
"rocm_sdk_core",
"--hidden-import",
"rocm_sdk_libraries_custom",
"--copy-metadata",
"rocm",
"--copy-metadata",
"rocm-sdk-core",
"--copy-metadata",
"rocm-sdk-libraries-custom",
# Repair rocm_sdk.find_libraries (masks UnboundLocalError
# with a readable ModuleNotFoundError on missing backends).
"--runtime-hook",
"pyi_rth_rocm_sdk.py",
"torch.backends.cudnn",
]
)
# Exclude NVIDIA CUDA packages from non-CUDA builds to keep binary small.
# When building from a venv with CUDA torch installed, PyInstaller would
# bundle ~3GB of NVIDIA shared libraries. We exclude both the Python
# modules and the binary DLLs. This applies to CPU and ROCm builds.
if not cuda:
else:
# Exclude NVIDIA CUDA packages from CPU-only builds to keep binary small.
# When building from a venv with CUDA torch installed, PyInstaller would
# bundle ~3GB of NVIDIA shared libraries. We exclude both the Python
# modules and the binary DLLs.
nvidia_packages = [
"nvidia",
"nvidia.cublas",
@@ -416,8 +354,8 @@ def build_server(cuda=False, rocm=False):
for pkg in nvidia_packages:
args.extend(["--exclude-module", pkg])
# Add MLX-specific imports if building on Apple Silicon (never for GPU builds)
if is_apple_silicon() and not cuda and not rocm:
# Add MLX-specific imports if building on Apple Silicon (never for CUDA builds)
if is_apple_silicon() and not cuda:
logger.info("Building for Apple Silicon - including MLX dependencies")
args.extend(
[
@@ -461,7 +399,7 @@ def build_server(cuda=False, rocm=False):
"mlx_lm",
]
)
elif not cuda and not rocm:
elif not cuda:
logger.info("Building for non-Apple Silicon platform - PyTorch only")
dist_dir = str(backend_dir / "dist")
@@ -482,128 +420,43 @@ def build_server(cuda=False, rocm=False):
os.chdir(backend_dir)
# For CPU builds on Windows, ensure we're using CPU-only torch.
# If CUDA or ROCm torch is installed (local dev), swap to CPU torch before
# building, then restore afterwards. This prevents PyInstaller from bundling
# GPU libraries into the CPU binary.
restore_torch = None
# If CUDA torch is installed (local dev), swap to CPU torch before building,
# then restore CUDA torch after. This prevents PyInstaller from bundling
# ~3GB of CUDA DLLs into the CPU binary.
restore_cuda = False
if not cuda and platform.system() == "Windows":
import subprocess
result = subprocess.run(
[sys.executable, "-c", "import torch; print(torch.version.cuda or '')"], capture_output=True, text=True
)
has_cuda_torch = bool(result.stdout.strip())
if has_cuda_torch:
logger.info("CUDA torch detected — installing CPU torch for CPU build...")
subprocess.run(
[
sys.executable,
"-m",
"pip",
"install",
"torch",
"torchvision",
"torchaudio",
"--index-url",
"https://download.pytorch.org/whl/cpu",
"--force-reinstall",
"-q",
],
check=True,
)
restore_cuda = True
# Run PyInstaller
try:
if not cuda and not rocm and platform.system() == "Windows":
import subprocess
cuda_result = subprocess.run(
[sys.executable, "-c", "import torch; print(torch.version.cuda or '')"], capture_output=True, text=True
)
rocm_result = subprocess.run(
[sys.executable, "-c", "import torch; print(torch.version.hip or '')"], capture_output=True, text=True
)
if cuda_result.stdout.strip():
restore_torch = "cuda"
logger.info("CUDA torch detected — installing CPU torch for CPU build...")
subprocess.run(
[
sys.executable,
"-m",
"pip",
"install",
"torch",
"torchvision",
"torchaudio",
"--index-url",
"https://download.pytorch.org/whl/cpu",
"--force-reinstall",
"--no-deps",
"-q",
],
check=True,
)
elif rocm_result.stdout.strip():
restore_torch = "rocm"
logger.info("ROCm torch detected — installing CPU torch for CPU build...")
subprocess.run(
[
sys.executable,
"-m",
"pip",
"install",
"torch",
"torchvision",
"torchaudio",
"--index-url",
"https://download.pytorch.org/whl/cpu",
"--force-reinstall",
"--no-deps",
"-q",
],
check=True,
)
# For ROCm builds on Windows, ensure ROCm torch is installed.
if rocm and platform.system() == "Windows":
import subprocess
if sys.implementation.name != "cpython" or sys.version_info[:2] != (3, 12):
raise RuntimeError(
"ROCm wheels are cp312-cp312-specific; "
f"got {sys.implementation.name} {sys.version.split()[0]}. "
"Use CPython 3.12 to build the ROCm binary."
)
result = subprocess.run(
[sys.executable, "-c", "import torch; print(torch.version.hip or '')"], capture_output=True, text=True
)
has_rocm_torch = bool(result.stdout.strip())
if not has_rocm_torch:
logger.info("ROCm torch not detected — installing ROCm torch for ROCm build...")
# Determine what to restore BEFORE overwriting the environment
cuda_result = subprocess.run(
[sys.executable, "-c", "import torch; print(torch.version.cuda or '')"],
capture_output=True,
text=True,
)
if cuda_result.stdout.strip():
restore_torch = "cuda"
else:
restore_torch = "cpu"
# Now overwrite the environment safely
subprocess.run(
[
sys.executable,
"-m",
"pip",
"install",
"https://repo.radeon.com/rocm/windows/rocm-rel-7.2.1/rocm_sdk_core-7.2.1-py3-none-win_amd64.whl",
"https://repo.radeon.com/rocm/windows/rocm-rel-7.2.1/rocm_sdk_devel-7.2.1-py3-none-win_amd64.whl",
"https://repo.radeon.com/rocm/windows/rocm-rel-7.2.1/rocm_sdk_libraries_custom-7.2.1-py3-none-win_amd64.whl",
"https://repo.radeon.com/rocm/windows/rocm-rel-7.2.1/rocm-7.2.1.tar.gz",
"--no-deps",
"-q",
],
check=True,
)
subprocess.run(
[
sys.executable,
"-m",
"pip",
"install",
"https://repo.radeon.com/rocm/windows/rocm-rel-7.2.1/torch-2.9.1%2Brocm7.2.1-cp312-cp312-win_amd64.whl",
"https://repo.radeon.com/rocm/windows/rocm-rel-7.2.1/torchaudio-2.9.1%2Brocm7.2.1-cp312-cp312-win_amd64.whl",
"https://repo.radeon.com/rocm/windows/rocm-rel-7.2.1/torchvision-0.24.1%2Brocm7.2.1-cp312-cp312-win_amd64.whl",
"--force-reinstall",
"--no-deps",
"-q",
],
check=True,
)
# Run PyInstaller
PyInstaller.__main__.run(args)
finally:
# Restore torch if we swapped it out (even on build failure)
if restore_torch == "cuda":
# Restore CUDA torch if we swapped it out (even on build failure)
if restore_cuda:
logger.info("Restoring CUDA torch...")
import subprocess
@@ -619,52 +472,10 @@ def build_server(cuda=False, rocm=False):
"--index-url",
"https://download.pytorch.org/whl/cu128",
"--force-reinstall",
"--no-deps",
"-q",
],
check=True,
)
elif restore_torch == "rocm":
logger.info("Restoring ROCm torch...")
import subprocess
subprocess.run(
[
sys.executable,
"-m",
"pip",
"install",
"https://repo.radeon.com/rocm/windows/rocm-rel-7.2.1/torch-2.9.1%2Brocm7.2.1-cp312-cp312-win_amd64.whl",
"https://repo.radeon.com/rocm/windows/rocm-rel-7.2.1/torchaudio-2.9.1%2Brocm7.2.1-cp312-cp312-win_amd64.whl",
"https://repo.radeon.com/rocm/windows/rocm-rel-7.2.1/torchvision-0.24.1%2Brocm7.2.1-cp312-cp312-win_amd64.whl",
"--force-reinstall",
"--no-deps",
"-q",
],
check=True,
)
elif restore_torch == "cpu":
logger.info("Restoring CPU torch...")
import subprocess
subprocess.run(
[
sys.executable,
"-m",
"pip",
"install",
"torch",
"torchvision",
"torchaudio",
"--index-url",
"https://download.pytorch.org/whl/cpu",
"--force-reinstall",
"--no-deps",
"-q",
],
check=True,
)
logger.info("Binary built in %s", backend_dir / "dist" / binary_name)
@@ -766,11 +577,6 @@ if __name__ == "__main__":
action="store_true",
help="Build CUDA-enabled binary (voicebox-server-cuda)",
)
parser.add_argument(
"--rocm",
action="store_true",
help="Build ROCm-enabled binary (voicebox-server-rocm) for AMD GPUs",
)
parser.add_argument(
"--shim",
action="store_true",
@@ -780,5 +586,4 @@ if __name__ == "__main__":
if cli_args.shim:
build_shim()
else:
build_server(cuda=cli_args.cuda, rocm=cli_args.rocm)
build_server(cuda=cli_args.cuda)
+6 -6
View File
@@ -6,8 +6,8 @@ without changing any importers.
"""
from .models import (
AudioChannel,
Base,
AudioChannel,
Capture,
CaptureSettings,
ChannelDeviceMapping,
@@ -24,12 +24,12 @@ from .models import (
StoryItem,
VoiceProfile,
)
from .session import SessionLocal, _db_path, engine, get_db, init_db
from .session import engine, SessionLocal, _db_path, init_db, get_db
__all__ = [
"AudioChannel",
# Models
"Base",
"AudioChannel",
"Capture",
"CaptureSettings",
"ChannelDeviceMapping",
@@ -42,13 +42,13 @@ __all__ = [
"ProfileChannelMapping",
"ProfileSample",
"Project",
"SessionLocal",
"Story",
"StoryItem",
"VoiceProfile",
"_db_path",
# Session
"engine",
"get_db",
"SessionLocal",
"_db_path",
"init_db",
"get_db",
]
+1 -8
View File
@@ -243,13 +243,6 @@ def _migrate_capture_settings(engine, inspector, tables: set[str]) -> None:
"hotkey_enabled BOOLEAN NOT NULL DEFAULT 0",
"hotkey_enabled",
)
if "keep_mic_warm" not in columns:
_add_column(
engine,
"capture_settings",
"keep_mic_warm BOOLEAN NOT NULL DEFAULT 0",
"keep_mic_warm",
)
def _migrate_mcp_bindings(engine, inspector, tables: set[str]) -> None:
@@ -303,7 +296,7 @@ def _normalize_storage_paths(engine, tables: set[str]) -> None:
"""Normalize stored file paths to be relative to the configured data dir."""
from pathlib import Path
from ..config import get_data_dir, resolve_storage_path, to_storage_path
from ..config import get_data_dir, to_storage_path, resolve_storage_path
data_dir = get_data_dir()
+2 -6
View File
@@ -1,9 +1,9 @@
"""ORM model definitions for the voicebox SQLite database."""
import uuid
from datetime import datetime
import uuid
from sqlalchemy import JSON, Boolean, Column, DateTime, Float, ForeignKey, Integer, String, Text
from sqlalchemy import Column, String, Integer, Float, DateTime, Text, ForeignKey, Boolean, JSON
from sqlalchemy.ext.declarative import declarative_base
from ..utils.capture_chords import (
@@ -210,10 +210,6 @@ class CaptureSettings(Base):
# "Voicebox would like to receive keystrokes from any application" dialog
# before they've even opened the Captures tab.
hotkey_enabled = Column(Boolean, nullable=False, default=False)
# Hold the microphone open while dictation is enabled so push-to-talk
# doesn't clip the first words. Off by default — when on, the OS mic-in-use
# indicator stays lit the whole time dictation is enabled.
keep_mic_warm = Column(Boolean, nullable=False, default=False)
# Lists of keytap key names (e.g. "MetaRight", "ControlRight"). Right-hand
# modifiers by default so they don't collide with left-hand shortcuts.
chord_push_to_talk_keys = Column(
+5 -18
View File
@@ -3,20 +3,20 @@
import logging
import uuid
from sqlalchemy import create_engine, event
from sqlalchemy import create_engine
from sqlalchemy.orm import sessionmaker
from .. import config
from .migrations import run_migrations
from .models import (
AudioChannel,
Base,
AudioChannel,
EffectPreset,
Generation,
GenerationVersion,
ProfileChannelMapping,
VoiceProfile,
)
from .migrations import run_migrations
from .seed import backfill_generation_versions, seed_builtin_presets
logger = logging.getLogger(__name__)
@@ -36,22 +36,9 @@ def init_db() -> None:
engine = create_engine(
f"sqlite:///{_db_path}",
# timeout is sqlite3's busy handler: wait up to 30s on a locked
# database instead of raising "database is locked" immediately.
connect_args={"check_same_thread": False, "timeout": 30},
connect_args={"check_same_thread": False},
)
@event.listens_for(engine, "connect")
def _set_sqlite_pragmas(dbapi_connection, _connection_record):
# WAL lets readers proceed while a writer holds the lock, which is
# the main source of lock racing between the generation worker and
# request handlers. synchronous=NORMAL is the recommended pairing
# (durable across app crashes, fsyncs only on checkpoint).
cursor = dbapi_connection.cursor()
cursor.execute("PRAGMA journal_mode=WAL")
cursor.execute("PRAGMA synchronous=NORMAL")
cursor.close()
SessionLocal = sessionmaker(autocommit=False, autoflush=False, bind=engine)
run_migrations(engine)
@@ -60,7 +47,7 @@ def init_db() -> None:
# Create default audio channel if it doesn't exist
db = SessionLocal()
try:
default_channel = db.query(AudioChannel).filter(AudioChannel.is_default).first()
default_channel = db.query(AudioChannel).filter(AudioChannel.is_default == True).first()
if not default_channel:
default_channel = AudioChannel(
id=str(uuid.uuid4()),
+1 -2
View File
@@ -5,11 +5,10 @@ entry point for development.
"""
import argparse
import uvicorn
from . import config, database
from .app import app # noqa: F401 -- re-export for uvicorn "backend.main:app"
from . import config, database
if __name__ == "__main__":
parser = argparse.ArgumentParser(description="voicebox backend server")
+3 -2
View File
@@ -11,13 +11,14 @@ import asyncio
import ipaddress
import logging
from contextvars import ContextVar
from datetime import UTC, datetime
from datetime import datetime, timezone
from starlette.middleware.base import BaseHTTPMiddleware
from starlette.requests import Request
from starlette.responses import Response
from starlette.types import ASGIApp
logger = logging.getLogger(__name__)
# Strong refs to in-flight stamp tasks so asyncio.create_task results
@@ -140,7 +141,7 @@ def _stamp_last_seen(client_id: str) -> None:
if row is None:
row = MCPClientBinding(client_id=client_id)
db.add(row)
row.last_seen_at = datetime.now(UTC)
row.last_seen_at = datetime.now(timezone.utc)
db.commit()
except Exception:
logger.debug(
+1
View File
@@ -8,6 +8,7 @@ floating pill surfaces whenever an agent is speaking.
import asyncio
from typing import Any
# Each subscriber gets its own queue. Bounded to drop oldest if a client lags.
_subscribers: set[asyncio.Queue[dict[str, Any]]] = set()
+1 -1
View File
@@ -30,7 +30,7 @@ def resolve_profile(
if client_id:
# Per-client binding. Imported lazily so this module stays importable
# even before the migration adds the table on first boot.
from ..database.models import MCPClientBinding
from ..database.models import MCPClientBinding # noqa: WPS433
binding = (
db.query(MCPClientBinding)
+2 -1
View File
@@ -9,8 +9,8 @@ binary bundled with the desktop app.
from __future__ import annotations
import logging
from collections.abc import Callable
from contextlib import AsyncExitStack, asynccontextmanager
from collections.abc import Callable
from fastapi import FastAPI
from fastmcp import FastMCP
@@ -18,6 +18,7 @@ from fastmcp import FastMCP
from .context import ClientIdMiddleware
from .tools import register_tools
logger = logging.getLogger(__name__)

Some files were not shown because too many files have changed in this diff Show More