Compare commits

..
Author SHA1 Message Date
Alex SummerandGitHub 51f49dea19 fix(docs): update quick start guide to reflect correct terminology for voice profiles (#963) 2026-07-26 23:32:02 -07:00
80610d880e fix(ui): open FloatingGenerateBox selects upward to prevent clipping (fixes #928) (#936)
The floating generate box is fixed at the bottom of the viewport, so
all of its Select dropdowns (voice profile, language, engine, effects)
opened downward into — or beyond — the window edge. Add side="top" to
each SelectContent so the menus appear above their trigger instead.

Co-authored-by: Claude Sonnet 4.6 <[email protected]>
2026-07-26 23:31:59 -07:00
Sai Sridhar TarraandGitHub 397051ba44 fix(key_codes): add Function key arm so macOS fn can be bound to a chord (#950)
key_from_str() had no arm for "Function", so it fell through to
None. Since build_chord propagates that as a hard Err via ?, binding
any chord containing fn made build_chord_bindings fail entirely —
HotkeyMonitor was never spawned, silently killing both push-to-talk
and toggle-to-talk until the chord was reverted.

Every other layer (keytap's macOS key tap, Key::Function itself, the
frontend's canonicalKeyFromEvent/displayLabelForKey) already handles
fn — only this string-to-Key bridge was missing the arm.

Fixes #941
2026-07-26 23:31:54 -07:00
1ba935e83b fix(export): disambiguate export filenames with generation id (#956)
Export filenames were derived from only the first 30 characters of the
generation text. Generations with similar wording (a common workflow when
iterating on the same line) produced identical filenames, so exports
collided on disk — the browser appended " (1)"/" (2)" and users ended up
opening audio that didn't match the expected filename.

Append the first 8 chars of the generation id to the .wav and .voicebox.zip
export filenames, in both the backend Content-Disposition headers and the
frontend save-file hooks.

Co-authored-by: Claude Opus 4.8 <[email protected]>
2026-07-26 23:31:51 -07:00
6a6f4643da fix(backend): guard avatar upload against a missing filename (#954)
`UploadFile.filename` can be None, and `Path(None)` raises TypeError. On the
avatar endpoint this happens before the try/except, so a filename-less upload
surfaces as an unhandled 500 instead of a clean response. Every other upload
handler already guards this with `file.filename or ""` (add_profile_sample,
transcription, generations); apply the same guard here.

Co-authored-by: Claude Opus 4.8 (1M context) <[email protected]>
2026-07-26 23:31:48 -07:00
44ef8daba3 fix(ui): parse naive-UTC timestamps consistently in formatAbsoluteDate (#953)
* fix(ui): parse naive-UTC timestamps consistently in formatAbsoluteDate

Backend timestamps are naive UTC (Python `datetime.utcnow()`) and are
serialized without a timezone suffix. `formatDate` already normalizes
these by appending `Z` before parsing, but `formatAbsoluteDate` called
`new Date(date)` directly. Per the ES spec, a timezone-less date-time
string is parsed as local time, so absolute timestamps were shown off by
the viewer's UTC offset (e.g. +9h in JST) — and disagreed with the
relative time rendered by `formatDate` for the same value (visible in the
Captures detail panel, which uses both on `capture.created_at`).

Extract the normalization into a shared `parseServerDate` helper and use
it in both formatters.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>

* docs(format): clarify parseServerDate comment on date-only vs date-time parsing

ECMAScript parses date-only strings ("2026-07-23") as UTC but timezone-less
date-time strings ("2026-07-23T10:00:00") as local time. The backend emits the
latter, which is the case this helper normalizes. Corrects the comment per PR
review feedback.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>

* docs(format): trim parseServerDate comment to match surrounding style

Reduce the multi-line explanation to a single why-comment consistent with
other utils comments.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <[email protected]>
2026-07-26 23:31:45 -07:00
AhmedIrfanandGitHub 68ece25a80 fix(mcp): add model_size parameter to voicebox.speak (#895)
The MCP voicebox.speak tool built its GenerationRequest without a
model_size, so every agent-triggered generation fell back to the schema
default ("1.7B"). There was no way to reach the 0.6B Qwen variant (or
TADA's 1B/3B) through MCP, and callers paid a model reload whenever the
requested size differed from what was already loaded.

Thread an optional model_size through voicebox.speak and the _speak
helper into GenerationRequest, mirroring the REST /generate surface.
Omitting it passes None, which generate_speech normalizes to the engine
default, so existing callers are unaffected.

Add backend/tests/test_mcp_speak.py covering the forwarded value, the
omitted-default path, and rejection of an invalid size.

Fixes #884
2026-07-26 23:31:41 -07:00
XariannandGitHub 1db0fdf645 fix(rocm): add MIOpen stability env vars to docker-compose.rocm.yml (#865)
Add three environment variables to prevent miopenStatusUnknownError and
system stuttering during inference on RDNA4 GPUs:

- MIOPEN_USER_DB_PATH: redirect MIOpen kernel cache to writable, persistent dir
- MIOPEN_CUSTOM_CACHE_DIR: same, for custom operator cache
- MIOPEN_FIND_MODE=FAST: use heuristic kernel selection instead of exhaustive
  benchmarking, which fails on RDNA4 with ptr: 0 size: 0 workspace warnings

MIOPEN_FIND_MODE=FAST does not affect output quality. All MIOpen kernel
variants produce the same numerical result; fast mode selects a known-good
kernel using heuristics instead of benchmarking every variant on the GPU.

Tested on RX 9070 (gfx1201) with ROCm 7.2 and PyTorch 2.12.1+rocm7.2.

Hardware note: tested on Ryzen 7 9800X3D + RX 9070 with Gigabyte B650M DS3H
motherboard. The exhaustive benchmarking failures may be related to IOMMU
behavior on this platform. This system was affected by an IOMMU bug patched
upstream in kernel 6.19.10, which may be a contributing factor. May not
affect all RDNA4 systems. MIOPEN_FIND_MODE=FAST is a safe default regardless.

Depends on PR #862 which fixes the broken ROCm Docker build.
2026-07-26 23:31:37 -07:00
Sai Sridhar TarraandGitHub 2a001fd63f fix(docker): normalize CRLF line endings on Windows checkouts (#951)
A Windows Git checkout with checkout-time CRLF conversion enabled
produces CRLF working-tree copies of package.json and
scripts/rocm-entrypoint.sh, breaking the Docker build two ways:

- The frontend stage's `sed -i -z 's/,\n  ]/…/'` is LF-anchored, so
  it doesn't match against \r\n and leaves an invalid trailing comma
  in package.json, which then fails JSON parsing in the vite build.
- The final stage copies rocm-entrypoint.sh straight from the build
  context; with a CRLF shebang the container reports the misleading
  "no such file or directory" for an entrypoint that plainly exists,
  because Linux can't resolve "/bin/sh\r" as an interpreter.

Add .gitattributes forcing LF for both files at checkout time, plus a
sed normalization step in each Dockerfile stage for resilience with
clones that predate the .gitattributes rule.

Fixes #915
2026-07-26 23:31:33 -07:00
Sai Sridhar TarraandGitHub e5813304ef fix(linux-audio): select monitor device by name instead of setting PULSE_SOURCE (#949)
std::env::set_var is not thread-safe on Unix (unsafe as of Rust 2024
edition) and calling it from a spawned capture thread while other
threads (tokio runtime, webview, Tauri plugins) may read the
environment is a data race risk. It also never got unset, so the
monitor source would leak into any later cpal/ALSA init in the same
process.

Replace the env-var indirection with direct device selection: when
pactl reports a monitor source name, search cpal's input device
enumeration for an exact match. Fall back to a substring match on
'monitor' (the original pactl-unavailable path), then the host's
default input device. This is the 'pass the source name directly to
cpal' option from the issue - no env mutation, no leakage between
capture sessions, and it still re-detects the current default sink's
monitor on every start_capture call.

Fixes #471
2026-07-26 23:31:30 -07:00
a5773807a5 fix(transcription): transcode uploads to WAV before STT (#957)
The /transcribe endpoint passed the raw uploaded file straight to the STT
backend (mlx_audio.stt -> miniaudio), which only decodes WAV/FLAC/MP3/Vorbis.
Browser recordings arrive as WebM/Opus (Chrome/Firefox MediaRecorder), so
web-mode dictation failed with 500 "unsupported file format". The Tauri app
was unaffected because WebKit produces MP4.

librosa already fully decodes the upload to compute duration (falling back to
audioread/ffmpeg for exotic containers), so re-encode that PCM to a temp WAV
and hand it to Whisper. WAV inputs pass through unchanged; the temp file is
cleaned up in the finally block.

Co-authored-by: Claude Opus 4.8 <[email protected]>
2026-07-26 23:31:27 -07:00
ed54347e81 Fix runaway MLX Qwen audio chunks (#964)
* fix runaway MLX Qwen audio chunks

* test: tighten runaway retry coverage

---------

Co-authored-by: huanghua01 <[email protected]>
2026-07-26 23:31:23 -07:00
624f6a2140 fix(tada): run voice-prompt encode under torch.inference_mode (#955)
Encoder.eval() alone still builds an autograd graph because parameters
require grad by default. On 8GB GPUs that ballooned TADA encode VRAM far
past the model footprint (issue 890). Wrap the encode forward in
inference_mode and add a unit test that asserts the flag is set.

Co-authored-by: fooSynaptic <[email protected]>
2026-07-26 23:31:20 -07:00
Kyle BuxtonandGitHub 669f85024f fix(macos): set Command flag on Cmd-down event so Electron apps paste (#952)
The macOS auto-paste sequence in `send_paste` posted the Cmd-down
CGEvent with flags = 0, setting the Command flag only on the V events.

On real hardware the Cmd keyDown (a flagsChanged event) already carries
kCGEventFlagMaskCommand, and Chromium/Electron builds its tracked
modifier state from that flag. With flags = 0 the tracker stays at
"Command up", so the following V matches neither the Cmd+V accelerator
(tracker says no modifier) nor plain-text insertion (the V event's own
flags say Command is held) — Electron drops it silently, producing no
paste and no stray "v". AppKit reads the V event's own modifier flags
and pastes regardless, which is why native apps (Notes, TextEdit,
Warp) worked while Electron targets (Slack, VS Code, VS Code Insiders)
silently no-op'd.

Setting kCGEventFlagMaskCommand on the Cmd-down event makes the
flagsChanged event well-formed; Chromium then registers Command=down
and Cmd+V matches. Likely fixes #762 and #643.
2026-07-26 23:31:17 -07:00
52f8d8dd38 Fix voice sample validation on Python 3.13 (fixes #852) (#853)
* Fix voice sample validation on Python 3.13

Python 3.13 removed audioop from the standard library, which broke reference
audio validation when adding voice samples. Add the audioop-lts backport for
3.13+ installs and bundle audioop in PyInstaller builds on the same versions.

* style(tests): satisfy Ruff import ordering

---------

Co-authored-by: Jamie Pine <[email protected]>
2026-07-20 22:35:23 -07:00
fb1e16d2ce fix(backend): return 404 instead of 500 for audio of failed generations (#893)
* fix(backend): return 404 instead of 500 for audio of failed generations

A failed generation stores an empty audio_path. resolve_storage_path("")
resolved to the data directory itself, which exists, so the route's 404
guard passed and FileResponse raised RuntimeError ("File at path .../data
is not a file"), surfacing as a 500.

- resolve_storage_path now returns None for empty paths
- audio routes check is_file() instead of exists() so directories never
  reach FileResponse
- GET /audio/{generation_id} reports "Generation failed; no audio
  available" when the generation status is failed

Co-Authored-By: Claude Fable 5 <[email protected]>

* fix(backend): reject empty Path objects in resolve_storage_path

Path("") is truthy, so the previous `if not path` guard only caught
None and empty strings. Callers such as database/migrations.py pass
Path objects, so an empty Path could still resolve to the data dir.
Check None separately and reject paths with no parts.

Also add regression tests asserting the version and sample audio
endpoints 404 when a stored path resolves to an existing directory
(guards the is_file() checks against regressing to exists()).

Addresses CodeRabbit review on PR #893.

Co-Authored-By: Claude Fable 5 <[email protected]>

* style(tests): drop parentheses on pytest.fixture decorator (ruff PT001)

Co-Authored-By: Claude Fable 5 <[email protected]>

* style(tests): satisfy Ruff naming rule

---------

Co-authored-by: Claude Fable 5 <[email protected]>
Co-authored-by: Jamie Pine <[email protected]>
2026-07-20 22:35:04 -07:00
f750596364 fix(setup): install mlx-lm and mlx-audio in setup-python on Apple Silicon (#892)
* fix(setup): install mlx-lm and mlx-audio in setup-python on Apple Silicon

The dev setup installed requirements-mlx.txt but not mlx-audio/mlx-lm
themselves, so POST /transcribe failed on a fresh Apple Silicon setup
with "No module named 'mlx_audio'" (then "No module named 'mlx_lm'").
The release workflow already installs both with --no-deps (they declare
transformers>=5.x, conflicting with our <=4.57.x cap); mirror that in
the setup-python recipe with the same pins.

Co-Authored-By: Claude Fable 5 <[email protected]>

* test: add MLX smoke test for the --no-deps mlx-audio/mlx-lm install

mlx-audio and mlx-lm are installed --no-deps, so a missing transitive
dependency only surfaces at import time. Add a pytest-discoverable
smoke test (skipped off Apple Silicon) covering the exact entry points
the backend uses: mlx_audio.tts.load, mlx_audio.stt.load (which also
exercises the miniaudio dep from issue #505), mlx_lm.load/generate,
and a basic mlx.core op.

Co-Authored-By: Claude Fable 5 <[email protected]>

---------

Co-authored-by: Claude Fable 5 <[email protected]>
2026-07-20 22:26:46 -07:00
XariannandGitHub 91cd6df108 fix(rocm): unset empty HSA_OVERRIDE_GFX_VERSION before torch loads (#864)
Docker compose sets HSA_OVERRIDE_GFX_VERSION=${HSA_OVERRIDE_GFX_VERSION:-}
which results in an empty string when not provided. An empty string is
not the same as unset - ROCm treats it as 'force-empty' and no GPU is
detected, even natively supported ones (e.g. gfx1201 / RX 9070 on ROCm 7.2).

Pop the env var when it is empty, before torch loads, so ROCm auto-detects
the GPU correctly.

Tested on RX 9070 (gfx1201) with ROCm 7.2 and PyTorch 2.12.1+rocm7.2.
2026-07-20 22:26:25 -07:00
484a39ad9f fix(build): build voicebox-mcp shim sidecar on Windows (#794)
The Windows `build-server` just recipe only built and copied the
voicebox-server sidecar, omitting the voicebox-mcp stdio shim that the
Unix scripts/build-server.sh builds via `build_binary.py --shim`.

As a result `just build` on Windows produced only one sidecar and the
Tauri bundle step failed with:

    resource path `binaries\voicebox-mcp-<triple>.exe` doesn't exist

Build and copy the shim sidecar after the server, mirroring
build-server.sh. Hoist the triple/binaries-dir setup ahead of both
builds so the shim step reuses them.

Co-authored-by: namu.shin <[email protected]>
Co-authored-by: Claude Opus 4.8 (1M context) <[email protected]>
2026-07-20 22:26:04 -07:00
3bfcbdc819 fix: the justfile syntax errror, GPU information cannot be read (#669)
Co-authored-by: xor_s <[email protected]>
2026-07-20 22:25:43 -07:00
Shabeer VPKandGitHub 190bc5e8a8 Update .dockerignore (#861)
whitelist ROCm entrypoint
2026-07-20 22:25:20 -07:00
jitendra kumar sainiandGitHub 80af641b61 docs: fix incorrect app identifier in CONTRIBUTING.md (#863)
The CUDA backend path used com.voicebox.app, but the actual Tauri identifier is sh.voicebox.app (as in tauri.conf.json and all other docs).
2026-07-20 21:19:03 -07:00
neuron-tech-aiandGitHub 6936789a88 Batch story item counts in list_stories to eliminate N+1 (#663)
list_stories() previously executed one COUNT(story_items) query per
story in a Python loop. With N stories that is N+1 round-trips to
SQLite regardless of list length. Replace with a single aggregated
GROUP BY query that fetches all counts at once, then populate each
StoryResponse from a dict lookup.
2026-07-20 21:14:46 -07:00
youtsuhoandGitHub f3eca34d33 fix: gate macOS-only keyboard_layout symbols behind cfg (#831)
* fix: gate macOS-only keyboard_layout symbols behind cfg to suppress dead_code warnings

* chore: sync bun.lock with package.json
2026-07-20 21:07:33 -07:00
30db291b01 fix(kokoro): add missing male Mandarin voices (#788)
Co-authored-by: Siddharth Chintawar <[email protected]>
Co-authored-by: Cursor <[email protected]>
2026-07-20 20:51:06 -07:00
Andrew BarnesandGitHub 71b51366bc Fix CUDA downloads on unsupported platforms (#770)
* Fix CUDA downloads on unsupported platforms

* fix: align CUDA status nullability

* fix: require CUDA download support flag
2026-07-20 19:22:18 -07:00
258b92c9c0 fix(offline): remove process-global offline guard from Qwen3 LLM loads (#924)
force_offline_if_cached flips HF_HUB_OFFLINE (env + huggingface_hub
constant + transformers._is_offline_mode) process-wide for the duration
of a cached LLM load, silently switching every concurrent model
download/load on other threads to offline mode. With default capture
settings (whisper-turbo STT + Qwen3 refinement + auto_refine) a first
run downloads several models concurrently, and a poisoned fetch
surfaces as "Can't load feature extractor..." (whisper) or
"Unrecognized model ... model_type" (Qwen3) rather than anything
mentioning offline mode.

These are the last two call sites of the guard — the same pattern was
deliberately removed app-wide in #524/#530 after identical failures,
and the 0.5.0 LLM backend reintroduced it. LLM loads now run with the
process's default HF_HUB_OFFLINE state, matching every other backend
(issue #462 precedent).

Fixes #841


Claude-Session: https://claude.ai/code/session_011iwL9AyeAWgz2jpgcHxJpC

Co-authored-by: Claude Fable 5 <[email protected]>
2026-07-20 18:47:10 -07:00
e6cf50c7f7 feat(i18n): add Korean (ko) locale with 559 translation keys (#814)
* feat(i18n): add Korean (ko) locale with 559 translation keys

* fix(i18n): complete Korean translations for current UI

---------

Co-authored-by: Jamie Pine <[email protected]>
2026-07-20 15:19:39 -07:00
2dc3b075d5 feat(i18n): add Spanish (es) locale (#798)
Adds Spanish as a UI display language, matching the existing
4-locale pattern (en, ja, zh-CN, zh-TW) with full key parity.

- app/src/i18n/locales/es/translation.json: 832 strings across 18
  namespaces, translated from the en master. Keys, {{interpolation}}
  placeholders, <code>/<path>/<link>/<strong> tags and _one/_other
  plurals preserved. Brand/model names (Whisper, Qwen3, CUDA, MCP…)
  left untranslated by design.
- app/src/i18n/index.ts: register `es` in SUPPORTED_LANGUAGES and
  resources; the language switcher and LanguageCode derive automatically.
- app/src/lib/utils/format.ts: wire the date-fns `es` locale for
  relative-date formatting.

Verified: key parity 832/832 (no missing/extra, placeholders & tags
intact), biome check clean, app+web typecheck pass, build:web succeeds.

Co-authored-by: Claude Opus 4.8 (1M context) <[email protected]>
Co-authored-by: Jamie Pine <[email protected]>
2026-07-20 13:15:57 -07:00
albanobattistellaandGitHub 05d90790f8 Add Italian translation (#904)
* Add Italian translation

* Add Italian language support to i18n
2026-07-20 13:14:31 -07:00
4a6b5da793 fix(linux): skip click-through toggle on dictate pill to prevent startup crash (#906)
The dictate pill window is built hidden at setup and the frontend emits
dictate:hide as soon as it mounts. The handler calls
set_ignore_cursor_events(true) on a window GTK has never realized, and
tao's CursorIgnoreEvents path unwraps the missing GdkWindow
(tao-0.34.5 event_loop.rs:449), panicking inside a glib dispatch that
cannot unwind — the process aborts within seconds of launch on Linux.

The click-through toggle exists as a macOS workaround for transparent
always-on-top NSWindows lingering as invisible click targets; it was
never needed on Linux. Gate all three call sites so Linux never toggles
it: the true/false pair stays balanced (never set, never unset), and
macOS/Windows builds are unchanged.

Co-authored-by: Claude Fable 5 <[email protected]>
2026-07-20 12:40:10 -07:00
2c9d02af62 fix(models): stop reporting errored downloads as still downloading (#926)
TaskManager.error_download() intentionally keeps a failed task in the
active list (status="error") so /tasks/active can surface the error
and retry UI — but /models/status derived its "downloading" flag from
the same unfiltered list. One failed download therefore showed the
model as downloading:true / downloaded:false for the life of the
process, masking the model's real cache state (even a fully valid
on-disk cache) until an app restart. Likely behind endless-spinner
reports like #181 and the restart-fixes-it pattern in #883.

Add TaskManager.get_pending_downloads() (downloading/extracting only)
and use it in /models/status; /tasks/active behavior is unchanged.

Fixes #925


Claude-Session: https://claude.ai/code/session_011iwL9AyeAWgz2jpgcHxJpC

Co-authored-by: Claude Fable 5 <[email protected]>
2026-07-20 12:39:58 -07:00
Elem OghenekaroandGitHub b680097dfb Fix: keep the uploaded file extension when transcribing (#903)
/transcribe wrote every upload to a temp file named .wav regardless of its
real format. librosa picks its decoder from the extension, so any non-wav
upload failed with "could not open/decode file" even though the format is
one the app handles elsewhere.

profiles.py already solves this for voice samples by keeping the uploaded
extension when it is one of the audio types it accepts, and falling back to
.wav otherwise. Same approach here, same set. The fallback means an unknown
or missing extension behaves exactly as it does today.
2026-07-20 12:39:46 -07:00
199 changed files with 7767 additions and 2926 deletions
-4
View File
@@ -37,7 +37,3 @@ replace = "version": "{new_version}"
[bumpversion:file:backend/__init__.py]
search = __version__ = "{current_version}"
replace = __version__ = "{new_version}"
[bumpversion:file:backend/pyproject.toml]
search = version = "{current_version}"
replace = version = "{new_version}"
+2 -1
View File
@@ -8,7 +8,8 @@ tauri/
landing/
docs/
mlx-test/
scripts/
scripts/*
!scripts/rocm-entrypoint.sh
# Dependencies & build artifacts (rebuilt in Docker)
node_modules/
-66
View File
@@ -6,10 +6,6 @@ on:
branches:
- main
concurrency:
group: ci-${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
frontend-quality:
runs-on: ubuntu-latest
@@ -23,70 +19,8 @@ jobs:
- name: Install dependencies
run: bun install --frozen-lockfile
- name: Lint (biome)
run: bun run lint
- name: Typecheck app + web
run: bun run typecheck
- name: Unit tests
run: bun run test
- name: Build web smoke test
run: bun run build:web
backend-quality:
# macOS arm64 matches the primary user platform and lets the MLX-path
# tests run instead of being skipped.
runs-on: macos-14
steps:
- uses: actions/checkout@v4
- name: Setup Python
uses: actions/setup-python@v5
with:
python-version: "3.12"
- name: Cache pip downloads
uses: actions/cache@v4
with:
path: ~/Library/Caches/pip
key: pip-${{ runner.os }}-${{ hashFiles('backend/requirements*.txt', 'justfile') }}
restore-keys: pip-${{ runner.os }}-
- name: Install just
run: brew install just
- name: Install backend dependencies
run: just setup-python
- name: Lint (ruff)
run: venv/bin/ruff check .
working-directory: backend
- name: Run tests
run: venv/bin/python -m pytest tests -q
working-directory: backend
rust-quality:
runs-on: macos-14
steps:
- uses: actions/checkout@v4
- name: Rust cache
uses: Swatinem/rust-cache@v2
with:
workspaces: tauri/src-tauri
- name: Stub sidecar binaries
# tauri-build validates externalBin paths; real sidecars are only
# produced by the release pipeline.
run: |
mkdir -p tauri/src-tauri/binaries tauri/dist
touch tauri/src-tauri/binaries/voicebox-server-aarch64-apple-darwin
touch tauri/src-tauri/binaries/voicebox-mcp-aarch64-apple-darwin
- name: Cargo check
run: cargo check --manifest-path tauri/src-tauri/Cargo.toml
BIN
View File
Binary file not shown.
+25 -23
View File
@@ -18,9 +18,9 @@ Thank you for your interest in contributing to Voicebox! This document provides
curl -fsSL https://bun.sh/install | bash
```
- **[Python 3.12+](https://python.org)** - For backend development
- **[Python 3.11+](https://python.org)** - For backend development
```bash
python --version # Should be 3.12 or higher
python --version # Should be 3.11 or higher
```
- **[Rust](https://rustup.rs)** - For Tauri desktop app (installed automatically by Tauri CLI)
@@ -115,6 +115,14 @@ just build-server
This makes PyInstaller use your local qwen-tts version instead of the pip-installed package.
### Generate OpenAPI Client
After starting the backend server:
```bash
./scripts/generate-api.sh
```
This downloads the OpenAPI schema and generates the TypeScript client in `app/src/lib/api/`
### Convert Assets to Web Formats
To optimize images and videos for the web, run:
@@ -204,7 +212,7 @@ export const ProfileCard = (props) => { ... }
- Follow PEP 8 style guide
- Use type hints
- Use async/await for I/O operations
- Format and lint with ruff (configured in `backend/pyproject.toml`)
- Format with Black (if configured)
```python
# Good
@@ -234,12 +242,9 @@ voicebox/
│ ├── lib/ # Utilities and API client
│ └── hooks/ # React hooks
├── backend/ # Python FastAPI server
│ ├── main.py # Entry point (FastAPI app assembled in app.py)
│ ├── routes/ # API routers, one per domain
│ ├── services/ # Business logic (generation, transcription, profiles, ...)
│ ├── backends/ # TTS engine implementations
│ ├── database/ # SQLAlchemy models, sessions, migrations
│ └── tests/ # pytest suite
│ ├── main.py # API routes
│ ├── tts.py # Voice synthesis
│ └── ...
├── tauri/ # Desktop app wrapper
│ └── src-tauri/ # Rust backend
└── scripts/ # Build scripts
@@ -284,26 +289,23 @@ voicebox/
When adding new API endpoints:
1. **Add the route to the relevant router in `backend/routes/`** (new routers get registered in `backend/routes/__init__.py`)
1. **Add route in `backend/main.py`**
2. **Create Pydantic models in `backend/models.py`**
3. **Implement business logic in appropriate module**
4. **Update OpenAPI schema** (automatic with FastAPI)
5. **Update the TypeScript client** — add matching types to `app/src/lib/api/types.ts` and a method to `app/src/lib/api/client.ts`
5. **Regenerate TypeScript client:**
```bash
bun run generate:api
```
6. **Update `backend/README.md`** with endpoint documentation
## Testing
Backend tests live in `backend/tests/` and run with pytest:
Currently, testing is primarily manual. When adding tests:
```bash
cd backend
venv/bin/python -m pytest tests
```
CI runs the backend test suite on every PR, along with frontend lint and typecheck (`bun run lint`, `bun run typecheck`) and a `cargo check` of the Tauri app (see `.github/workflows/ci.yml`). Add backend tests alongside your changes where it makes sense.
- **Frontend**: Vitest for React component tests (coverage is still sparse — contributions welcome)
- **E2E**: Playwright for end-to-end tests (future)
- **Backend**: Use pytest for Python tests
- **Frontend**: Use Vitest for React component tests
- **E2E**: Use Playwright for end-to-end tests (future)
## Pull Request Process
@@ -361,7 +363,7 @@ See [docs/content/docs/overview/troubleshooting.mdx](docs/content/docs/overview/
**Quick fixes:**
- **Backend won't start:** Check Python version (3.12+), ensure venv is activated, install dependencies
- **Backend won't start:** Check Python version (3.11+), ensure venv is activated, install dependencies
- **Tauri build fails:** Ensure Rust is installed, clean build with `cd tauri/src-tauri && cargo clean`
- **OpenAPI client generation fails:** Ensure backend is running, check `curl http://localhost:17493/openapi.json`
@@ -377,7 +379,7 @@ See [docs/content/docs/overview/troubleshooting.mdx](docs/content/docs/overview/
- [README.md](README.md) - Project overview
- [backend/README.md](backend/README.md) - API documentation
- [docs/PROJECT_STATUS.md](docs/PROJECT_STATUS.md) - Living engineering roadmap: architecture, shipped vs in-flight work, prioritized open issues, candidate TTS engines under evaluation, architectural bottlenecks. Keep this updated when you ship significant features, close or backlog a model integration, or identify new bottlenecks.
- [docs/content/docs/developer/autoupdater.mdx](docs/content/docs/developer/autoupdater.mdx) - Auto-updater setup (published at [voicebox.sh docs](https://voicebox.sh/docs/developer/autoupdater))
- [docs/AUTOUPDATER_QUICKSTART.md](docs/AUTOUPDATER_QUICKSTART.md) - Auto-updater setup
- [SECURITY.md](SECURITY.md) - Security policy
- [CHANGELOG.md](CHANGELOG.md) - Version history
+1 -1
View File
@@ -64,7 +64,7 @@ RUN pip install --no-cache-dir --prefix=/install -r requirements.txt
RUN pip install --no-cache-dir --prefix=/install --no-deps chatterbox-tts
RUN pip install --no-cache-dir --prefix=/install --no-deps hume-tada
RUN pip install --no-cache-dir --prefix=/install \
git+https://github.com/QwenLM/Qwen3-TTS.git@022e286b98fbec7e1e916cb940cdf532cd9f488e
git+https://github.com/QwenLM/Qwen3-TTS.git
# === Stage 3: Runtime ===
+2 -1
View File
@@ -270,7 +270,8 @@ Use cases: agent dev loops (dictate a question, hear the answer in a cloned voic
| Platform | Backend | Notes |
| ------------------------ | -------------- | ---------------------------------------------- |
| macOS (Apple Silicon) | MLX (Metal) | 4-5x faster via Neural Engine |
| Windows / Linux (NVIDIA) | PyTorch (CUDA) | Auto-downloads CUDA binary from within the app |
| Windows (NVIDIA) | PyTorch (CUDA) | Auto-downloads CUDA binary from within the app |
| Linux (NVIDIA) | PyTorch (CUDA) | Use a local/remote Python backend with CUDA PyTorch |
| Linux (AMD) | PyTorch (ROCm) | Auto-configures HSA_OVERRIDE_GFX_VERSION |
| Windows (any GPU) | DirectML | Universal Windows GPU support |
| Intel Arc | IPEX/XPU | Intel discrete GPU acceleration |
+4 -4
View File
@@ -6,8 +6,8 @@ We release patches for security vulnerabilities. Which versions are eligible for
| Version | Supported |
| ------- | ------------------ |
| 0.5.x | :white_check_mark: |
| < 0.5 | :x: |
| 0.3.x | :white_check_mark: |
| < 0.3 | :x: |
## Reporting a Vulnerability
@@ -39,7 +39,7 @@ We will:
### For Developers
- **Dependencies** - Keep all dependencies up to date
- **CI checks** - Every PR must pass typecheck, lint, backend tests, and `cargo check` before merging
- **Code review** - All PRs require review before merging
- **Secrets** - Never commit API keys or signing keys
- **Signing** - All releases are cryptographically signed
@@ -82,7 +82,7 @@ Timeline may vary based on severity and complexity.
## Security Updates
Security updates will be:
- Released as patch versions (e.g., 0.5.1)
- Released as patch versions (e.g., 0.3.2)
- Documented in CHANGELOG.md
- Announced via GitHub releases
- Automatically delivered via auto-updater
+1 -1
View File
@@ -6,7 +6,7 @@
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>voicebox</title>
<script>
(() => {
(function () {
try {
var theme = 'system';
var raw = localStorage.getItem('voicebox-ui');
-2
View File
@@ -7,7 +7,6 @@
"dev": "vite",
"build": "vite build",
"typecheck": "tsc -p tsconfig.json --noEmit",
"test": "bun test",
"preview": "vite preview",
"lint": "biome lint src",
"lint:fix": "biome lint --write src",
@@ -61,7 +60,6 @@
},
"devDependencies": {
"@tailwindcss/vite": "^4.1.18",
"@types/bun": "^1.3.4",
"@types/react": "^18.3.0",
"@types/react-dom": "^18.3.0",
"@vitejs/plugin-react": "^4.3.0",
@@ -35,17 +35,19 @@ export function DictateWindow() {
};
}, []);
// Mirrored from the main window: true only when dictation is armed and the
// user opted into keeping the microphone ready.
const [micWarm, setMicWarm] = useState(false);
// Snapshot of the focused UI element at chord-start, shipped over from
// Rust on the ``dictate:start`` payload. Held in a ref so it survives
// the 1–2 s transcribe + refine window — the paste only fires once the
// final text comes back.
const focusRef = useRef<FocusSnapshot | null>(null);
const session = useCaptureRecordingSession({
keepMicWarm: micWarm,
onFinalText: async (text, _capture, allowAutoPaste, context) => {
// Focus is the snapshot taken at chord-start and threaded through as this
// take's context, so it survives the 1–2 s transcribe + refine window and
// overlapping dictations can't paste into each other's target.
const focus = context as FocusSnapshot | null;
onFinalText: async (text, _capture, allowAutoPaste) => {
const focus = focusRef.current;
// Consume-once: a second chord before this fires would overwrite
// focusRef, but nulling it here guards against the late-arriving
// refine-result firing a paste after the user has moved on.
focusRef.current = null;
if (!allowAutoPaste) return;
if (!focus || !text.trim()) return;
try {
@@ -70,41 +72,23 @@ export function DictateWindow() {
sessionRef.current = session;
useEffect(() => {
let disposed = false;
const unlistens: UnlistenFn[] = [];
const registrations = [
const unlistens: Promise<UnlistenFn>[] = [];
unlistens.push(
listen<{ focus: FocusSnapshot | null }>('dictate:start', (event) => {
sessionRef.current.startRecording(event.payload?.focus ?? null);
focusRef.current = event.payload?.focus ?? null;
sessionRef.current.startRecording();
}),
);
unlistens.push(
listen('dictate:stop', () => {
// Forward stops that arrive while getUserMedia is still resolving.
sessionRef.current.stopRecording();
if (sessionRef.current.isRecording) sessionRef.current.stopRecording();
}),
listen<boolean>('dictate:warm', (event) => {
setMicWarm(Boolean(event.payload));
}),
];
Promise.all(registrations)
.then((registered) => {
if (disposed) {
for (const unlisten of registered) unlisten();
return;
}
unlistens.push(...registered);
emit('dictate:warm-request').catch(() => {});
})
.catch((err) => console.warn('[dictate] event listener registration failed:', err));
);
return () => {
disposed = true;
for (const unlisten of unlistens) unlisten();
for (const p of unlistens) p.then((fn) => fn()).catch(() => {});
};
}, []);
useEffect(() => {
if (micWarm) void session.prewarm();
else session.releaseWarm();
}, [micWarm, session.prewarm, session.releaseWarm]);
// --- Agent-speak cycle ---------------------------------------------------
const [speaking, setSpeaking] = useState<{
@@ -138,7 +138,6 @@ export function CapturesPage() {
const allowAutoPaste = settings?.allow_auto_paste ?? true;
const defaultVoiceId = settings?.default_playback_voice_id ?? null;
const hotkeyEnabled = settings?.hotkey_enabled ?? false;
const keepMicWarm = settings?.keep_mic_warm ?? false;
const pushToTalkKeys = settings?.chord_push_to_talk_keys ?? defaultChordKeys('push');
const toggleToTalkKeys = settings?.chord_toggle_to_talk_keys ?? defaultChordKeys('toggle');
@@ -222,22 +221,6 @@ export function CapturesPage() {
<InputMonitoringNotice enabled={hotkeyEnabled} />
</div>
<SettingRow
title={t('settings.captures.dictation.keepMicWarm.title')}
description={t('settings.captures.dictation.keepMicWarm.description')}
htmlFor="keepMicWarm"
action={
<Toggle
id="keepMicWarm"
checked={keepMicWarm}
disabled={!hotkeyEnabled}
onCheckedChange={(v) => {
update({ keep_mic_warm: v });
}}
/>
}
/>
<SettingRow
title={t('settings.captures.dictation.pushToTalk.title')}
description={t('settings.captures.dictation.pushToTalk.description')}
@@ -38,7 +38,6 @@ import {
useUpdateStoryItemVolume,
} from '@/lib/hooks/useStories';
import { cn } from '@/lib/utils/cn';
import { computeTrimValues } from '@/lib/utils/trim';
import { useGenerationStore } from '@/stores/generationStore';
import { useStoryStore } from '@/stores/storyStore';
@@ -601,17 +600,41 @@ export function StoryTrackEditor({ storyId, items }: StoryTrackEditorProps) {
const deltaMs = pixelsToMs(deltaX); // Signed delta in milliseconds
const { item, initialTrimStart, initialTrimEnd } = trimStartItemRef.current;
const newTrimValues = computeTrimValues(
trimSide,
deltaMs,
initialTrimStart,
initialTrimEnd,
item.duration * 1000,
);
if (!newTrimValues) return;
const originalDurationMs = item.duration * 1000;
let newTrimStart = initialTrimStart;
let newTrimEnd = initialTrimEnd;
if (trimSide === 'start') {
// Moving right increases trim_start (trims more from start)
// Moving left decreases trim_start (restores from start)
newTrimStart = Math.round(
Math.max(
0,
Math.min(initialTrimStart + deltaMs, originalDurationMs - initialTrimEnd - 100),
),
);
} else {
// Moving right decreases trim_end (restores from end)
// Moving left increases trim_end (trims more from end)
newTrimEnd = Math.round(
Math.max(
0,
Math.min(initialTrimEnd - deltaMs, originalDurationMs - initialTrimStart - 100),
),
);
}
// Validate that we don't exceed duration
if (newTrimStart + newTrimEnd >= originalDurationMs - 100) {
return; // Don't allow trimming to less than 100ms
}
// Update temporary trim values for visual feedback
setTempTrimValues(newTrimValues);
setTempTrimValues({
trim_start_ms: newTrimStart,
trim_end_ms: newTrimEnd,
});
},
[trimmingItem, trimSide, trimStartX, pixelsToMs],
);
+10 -1
View File
@@ -2,19 +2,25 @@ import i18n from 'i18next';
import LanguageDetector from 'i18next-browser-languagedetector';
import { initReactI18next } from 'react-i18next';
import en from './locales/en/translation.json';
import es from './locales/es/translation.json';
import fr from './locales/fr/translation.json';
import it from './locales/it/translation.json';
import ja from './locales/ja/translation.json';
import ko from './locales/ko/translation.json';
import ptBR from './locales/pt-BR/translation.json';
import zhCN from './locales/zh-CN/translation.json';
import zhTW from './locales/zh-TW/translation.json';
import fr from './locales/fr/translation.json';
export const SUPPORTED_LANGUAGES = [
{ code: 'en', label: 'English' },
{ code: 'es', label: 'Español' },
{ code: 'pt-BR', label: 'Português (Brasil)' },
{ code: 'ja', label: '日本語' },
{ code: 'ko', label: '한국어' },
{ code: 'zh-CN', label: '简体中文' },
{ code: 'zh-TW', label: '繁體中文' },
{ code: 'fr', label: 'Français' },
{ code: 'it', label: 'Italiano' },
] as const;
export type LanguageCode = (typeof SUPPORTED_LANGUAGES)[number]['code'];
@@ -25,11 +31,14 @@ i18n
.init({
resources: {
en: { translation: en },
es: { translation: es },
'pt-BR': { translation: ptBR },
ja: { translation: ja },
ko: { translation: ko },
'zh-CN': { translation: zhCN },
'zh-TW': { translation: zhTW },
fr: { translation: fr },
it: { translation: it },
},
fallbackLng: 'en',
supportedLngs: SUPPORTED_LANGUAGES.map((l) => l.code),
-4
View File
@@ -887,10 +887,6 @@
"title": "Global shortcut",
"description": "Hold the shortcut to record from anywhere on your machine. Release to transcribe."
},
"keepMicWarm": {
"title": "Keep microphone ready",
"description": "Hold the microphone open while dictation is enabled so the first words are never clipped. The macOS microphone indicator stays lit while it's on."
},
"pushToTalk": {
"title": "Push-to-talk shortcut",
"description": "Hold these keys anywhere on your system to record. Release to stop and transcribe.",
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
+15 -1
View File
@@ -1,5 +1,4 @@
import type { LanguageCode } from '@/lib/constants/languages';
import { formatErrorDetail } from '@/lib/api/errors';
import { useServerStore } from '@/stores/serverStore';
import type {
ActiveTasksResponse,
@@ -56,6 +55,21 @@ import type {
CloudStatus,
} from './types';
function formatErrorDetail(detail: unknown, fallback: string): string {
if (typeof detail === 'string') return detail;
if (Array.isArray(detail)) {
return detail
.map((e: Record<string, unknown>) => e.msg || e.message || JSON.stringify(e))
.join('; ');
}
if (detail && typeof detail === 'object') {
const obj = detail as Record<string, unknown>;
if (typeof obj.message === 'string') return obj.message;
return JSON.stringify(detail);
}
return fallback;
}
class ApiClient {
private getBaseUrl(): string {
const serverUrl = useServerStore.getState().serverUrl;
+25
View File
@@ -0,0 +1,25 @@
/* generated using openapi-typescript-codegen -- do not edit */
/* istanbul ignore file */
/* tslint:disable */
/* eslint-disable */
import type { ApiRequestOptions } from './ApiRequestOptions';
import type { ApiResult } from './ApiResult';
export class ApiError extends Error {
public readonly url: string;
public readonly status: number;
public readonly statusText: string;
public readonly body: any;
public readonly request: ApiRequestOptions;
constructor(request: ApiRequestOptions, response: ApiResult, message: string) {
super(message);
this.name = 'ApiError';
this.url = response.url;
this.status = response.status;
this.statusText = response.statusText;
this.body = response.body;
this.request = request;
}
}
+17
View File
@@ -0,0 +1,17 @@
/* generated using openapi-typescript-codegen -- do not edit */
/* istanbul ignore file */
/* tslint:disable */
/* eslint-disable */
export type ApiRequestOptions = {
readonly method: 'GET' | 'PUT' | 'POST' | 'DELETE' | 'OPTIONS' | 'HEAD' | 'PATCH';
readonly url: string;
readonly path?: Record<string, any>;
readonly cookies?: Record<string, any>;
readonly headers?: Record<string, any>;
readonly query?: Record<string, any>;
readonly formData?: Record<string, any>;
readonly body?: any;
readonly mediaType?: string;
readonly responseHeader?: string;
readonly errors?: Record<number, string>;
};
+11
View File
@@ -0,0 +1,11 @@
/* generated using openapi-typescript-codegen -- do not edit */
/* istanbul ignore file */
/* tslint:disable */
/* eslint-disable */
export type ApiResult = {
readonly url: string;
readonly ok: boolean;
readonly status: number;
readonly statusText: string;
readonly body: any;
};
+130
View File
@@ -0,0 +1,130 @@
/* generated using openapi-typescript-codegen -- do not edit */
/* istanbul ignore file */
/* tslint:disable */
/* eslint-disable */
export class CancelError extends Error {
constructor(message: string) {
super(message);
this.name = 'CancelError';
}
public get isCancelled(): boolean {
return true;
}
}
export interface OnCancel {
readonly isResolved: boolean;
readonly isRejected: boolean;
readonly isCancelled: boolean;
(cancelHandler: () => void): void;
}
export class CancelablePromise<T> implements Promise<T> {
#isResolved: boolean;
#isRejected: boolean;
#isCancelled: boolean;
readonly #cancelHandlers: (() => void)[];
readonly #promise: Promise<T>;
#resolve?: (value: T | PromiseLike<T>) => void;
#reject?: (reason?: any) => void;
constructor(
executor: (
resolve: (value: T | PromiseLike<T>) => void,
reject: (reason?: any) => void,
onCancel: OnCancel,
) => void,
) {
this.#isResolved = false;
this.#isRejected = false;
this.#isCancelled = false;
this.#cancelHandlers = [];
this.#promise = new Promise<T>((resolve, reject) => {
this.#resolve = resolve;
this.#reject = reject;
const onResolve = (value: T | PromiseLike<T>): void => {
if (this.#isResolved || this.#isRejected || this.#isCancelled) {
return;
}
this.#isResolved = true;
if (this.#resolve) this.#resolve(value);
};
const onReject = (reason?: any): void => {
if (this.#isResolved || this.#isRejected || this.#isCancelled) {
return;
}
this.#isRejected = true;
if (this.#reject) this.#reject(reason);
};
const onCancel = (cancelHandler: () => void): void => {
if (this.#isResolved || this.#isRejected || this.#isCancelled) {
return;
}
this.#cancelHandlers.push(cancelHandler);
};
Object.defineProperty(onCancel, 'isResolved', {
get: (): boolean => this.#isResolved,
});
Object.defineProperty(onCancel, 'isRejected', {
get: (): boolean => this.#isRejected,
});
Object.defineProperty(onCancel, 'isCancelled', {
get: (): boolean => this.#isCancelled,
});
return executor(onResolve, onReject, onCancel as OnCancel);
});
}
get [Symbol.toStringTag]() {
return 'Cancellable Promise';
}
public then<TResult1 = T, TResult2 = never>(
onFulfilled?: ((value: T) => TResult1 | PromiseLike<TResult1>) | null,
onRejected?: ((reason: any) => TResult2 | PromiseLike<TResult2>) | null,
): Promise<TResult1 | TResult2> {
return this.#promise.then(onFulfilled, onRejected);
}
public catch<TResult = never>(
onRejected?: ((reason: any) => TResult | PromiseLike<TResult>) | null,
): Promise<T | TResult> {
return this.#promise.catch(onRejected);
}
public finally(onFinally?: (() => void) | null): Promise<T> {
return this.#promise.finally(onFinally);
}
public cancel(): void {
if (this.#isResolved || this.#isRejected || this.#isCancelled) {
return;
}
this.#isCancelled = true;
if (this.#cancelHandlers.length) {
try {
for (const cancelHandler of this.#cancelHandlers) {
cancelHandler();
}
} catch (error) {
console.warn('Cancellation threw an error', error);
return;
}
}
this.#cancelHandlers.length = 0;
if (this.#reject) this.#reject(new CancelError('Request aborted'));
}
public get isCancelled(): boolean {
return this.#isCancelled;
}
}
+32
View File
@@ -0,0 +1,32 @@
/* generated using openapi-typescript-codegen -- do not edit */
/* istanbul ignore file */
/* tslint:disable */
/* eslint-disable */
import type { ApiRequestOptions } from './ApiRequestOptions';
type Resolver<T> = (options: ApiRequestOptions) => Promise<T>;
type Headers = Record<string, string>;
export type OpenAPIConfig = {
BASE: string;
VERSION: string;
WITH_CREDENTIALS: boolean;
CREDENTIALS: 'include' | 'omit' | 'same-origin';
TOKEN?: string | Resolver<string> | undefined;
USERNAME?: string | Resolver<string> | undefined;
PASSWORD?: string | Resolver<string> | undefined;
HEADERS?: Headers | Resolver<Headers> | undefined;
ENCODE_PATH?: ((path: string) => string) | undefined;
};
export const OpenAPI: OpenAPIConfig = {
BASE: '',
VERSION: '0.1.0',
WITH_CREDENTIALS: false,
CREDENTIALS: 'include',
TOKEN: undefined,
USERNAME: undefined,
PASSWORD: undefined,
HEADERS: undefined,
ENCODE_PATH: undefined,
};
+341
View File
@@ -0,0 +1,341 @@
/* generated using openapi-typescript-codegen -- do not edit */
/* istanbul ignore file */
/* tslint:disable */
/* eslint-disable */
import { ApiError } from './ApiError';
import type { ApiRequestOptions } from './ApiRequestOptions';
import type { ApiResult } from './ApiResult';
import { CancelablePromise } from './CancelablePromise';
import type { OnCancel } from './CancelablePromise';
import type { OpenAPIConfig } from './OpenAPI';
export const isDefined = <T>(
value: T | null | undefined,
): value is Exclude<T, null | undefined> => {
return value !== undefined && value !== null;
};
export const isString = (value: any): value is string => {
return typeof value === 'string';
};
export const isStringWithValue = (value: any): value is string => {
return isString(value) && value !== '';
};
export const isBlob = (value: any): value is Blob => {
return (
typeof value === 'object' &&
typeof value.type === 'string' &&
typeof value.stream === 'function' &&
typeof value.arrayBuffer === 'function' &&
typeof value.constructor === 'function' &&
typeof value.constructor.name === 'string' &&
/^(Blob|File)$/.test(value.constructor.name) &&
/^(Blob|File)$/.test(value[Symbol.toStringTag])
);
};
export const isFormData = (value: any): value is FormData => {
return value instanceof FormData;
};
export const base64 = (str: string): string => {
try {
return btoa(str);
} catch (err) {
// @ts-ignore
return Buffer.from(str).toString('base64');
}
};
export const getQueryString = (params: Record<string, any>): string => {
const qs: string[] = [];
const append = (key: string, value: any) => {
qs.push(`${encodeURIComponent(key)}=${encodeURIComponent(String(value))}`);
};
const process = (key: string, value: any) => {
if (isDefined(value)) {
if (Array.isArray(value)) {
value.forEach((v) => {
process(key, v);
});
} else if (typeof value === 'object') {
Object.entries(value).forEach(([k, v]) => {
process(`${key}[${k}]`, v);
});
} else {
append(key, value);
}
}
};
Object.entries(params).forEach(([key, value]) => {
process(key, value);
});
if (qs.length > 0) {
return `?${qs.join('&')}`;
}
return '';
};
const getUrl = (config: OpenAPIConfig, options: ApiRequestOptions): string => {
const encoder = config.ENCODE_PATH || encodeURI;
const path = options.url
.replace('{api-version}', config.VERSION)
.replace(/{(.*?)}/g, (substring: string, group: string) => {
if (options.path?.hasOwnProperty(group)) {
return encoder(String(options.path[group]));
}
return substring;
});
const url = `${config.BASE}${path}`;
if (options.query) {
return `${url}${getQueryString(options.query)}`;
}
return url;
};
export const getFormData = (options: ApiRequestOptions): FormData | undefined => {
if (options.formData) {
const formData = new FormData();
const process = (key: string, value: any) => {
if (isString(value) || isBlob(value)) {
formData.append(key, value);
} else {
formData.append(key, JSON.stringify(value));
}
};
Object.entries(options.formData)
.filter(([_, value]) => isDefined(value))
.forEach(([key, value]) => {
if (Array.isArray(value)) {
value.forEach((v) => process(key, v));
} else {
process(key, value);
}
});
return formData;
}
return undefined;
};
type Resolver<T> = (options: ApiRequestOptions) => Promise<T>;
export const resolve = async <T>(
options: ApiRequestOptions,
resolver?: T | Resolver<T>,
): Promise<T | undefined> => {
if (typeof resolver === 'function') {
return (resolver as Resolver<T>)(options);
}
return resolver;
};
export const getHeaders = async (
config: OpenAPIConfig,
options: ApiRequestOptions,
): Promise<Headers> => {
const [token, username, password, additionalHeaders] = await Promise.all([
resolve(options, config.TOKEN),
resolve(options, config.USERNAME),
resolve(options, config.PASSWORD),
resolve(options, config.HEADERS),
]);
const headers = Object.entries({
Accept: 'application/json',
...additionalHeaders,
...options.headers,
})
.filter(([_, value]) => isDefined(value))
.reduce(
(headers, [key, value]) => ({
...headers,
[key]: String(value),
}),
{} as Record<string, string>,
);
if (isStringWithValue(token)) {
headers['Authorization'] = `Bearer ${token}`;
}
if (isStringWithValue(username) && isStringWithValue(password)) {
const credentials = base64(`${username}:${password}`);
headers['Authorization'] = `Basic ${credentials}`;
}
if (options.body !== undefined) {
if (options.mediaType) {
headers['Content-Type'] = options.mediaType;
} else if (isBlob(options.body)) {
headers['Content-Type'] = options.body.type || 'application/octet-stream';
} else if (isString(options.body)) {
headers['Content-Type'] = 'text/plain';
} else if (!isFormData(options.body)) {
headers['Content-Type'] = 'application/json';
}
}
return new Headers(headers);
};
export const getRequestBody = (options: ApiRequestOptions): any => {
if (options.body !== undefined) {
if (options.mediaType?.includes('/json')) {
return JSON.stringify(options.body);
} else if (isString(options.body) || isBlob(options.body) || isFormData(options.body)) {
return options.body;
} else {
return JSON.stringify(options.body);
}
}
return undefined;
};
export const sendRequest = async (
config: OpenAPIConfig,
options: ApiRequestOptions,
url: string,
body: any,
formData: FormData | undefined,
headers: Headers,
onCancel: OnCancel,
): Promise<Response> => {
const controller = new AbortController();
const request: RequestInit = {
headers,
body: body ?? formData,
method: options.method,
signal: controller.signal,
};
if (config.WITH_CREDENTIALS) {
request.credentials = config.CREDENTIALS;
}
onCancel(() => controller.abort());
return await fetch(url, request);
};
export const getResponseHeader = (
response: Response,
responseHeader?: string,
): string | undefined => {
if (responseHeader) {
const content = response.headers.get(responseHeader);
if (isString(content)) {
return content;
}
}
return undefined;
};
export const getResponseBody = async (response: Response): Promise<any> => {
if (response.status !== 204) {
try {
const contentType = response.headers.get('Content-Type');
if (contentType) {
const jsonTypes = ['application/json', 'application/problem+json'];
const isJSON = jsonTypes.some((type) => contentType.toLowerCase().startsWith(type));
if (isJSON) {
return await response.json();
} else {
return await response.text();
}
}
} catch (error) {
console.error(error);
}
}
return undefined;
};
export const catchErrorCodes = (options: ApiRequestOptions, result: ApiResult): void => {
const errors: Record<number, string> = {
400: 'Bad Request',
401: 'Unauthorized',
403: 'Forbidden',
404: 'Not Found',
500: 'Internal Server Error',
502: 'Bad Gateway',
503: 'Service Unavailable',
...options.errors,
};
const error = errors[result.status];
if (error) {
throw new ApiError(options, result, error);
}
if (!result.ok) {
const errorStatus = result.status ?? 'unknown';
const errorStatusText = result.statusText ?? 'unknown';
const errorBody = (() => {
try {
return JSON.stringify(result.body, null, 2);
} catch (e) {
return undefined;
}
})();
throw new ApiError(
options,
result,
`Generic Error: status: ${errorStatus}; status text: ${errorStatusText}; body: ${errorBody}`,
);
}
};
/**
* Request method
* @param config The OpenAPI configuration object
* @param options The request options from the service
* @returns CancelablePromise<T>
* @throws ApiError
*/
export const request = <T>(
config: OpenAPIConfig,
options: ApiRequestOptions,
): CancelablePromise<T> => {
return new CancelablePromise(async (resolve, reject, onCancel) => {
try {
const url = getUrl(config, options);
const formData = getFormData(options);
const body = getRequestBody(options);
const headers = await getHeaders(config, options);
if (!onCancel.isCancelled) {
const response = await sendRequest(config, options, url, body, formData, headers, onCancel);
const responseBody = await getResponseBody(response);
const responseHeader = getResponseHeader(response, options.responseHeader);
const result: ApiResult = {
url,
ok: response.ok,
status: response.status,
statusText: response.statusText,
body: responseHeader ?? responseBody,
};
catchErrorCodes(options, result);
resolve(result.body);
}
} catch (error) {
reject(error);
}
});
};
-58
View File
@@ -1,58 +0,0 @@
import { describe, expect, test } from 'bun:test';
import { formatErrorDetail } from './errors';
const FALLBACK = 'HTTP error! status: 500';
describe('formatErrorDetail', () => {
test('returns string details as-is', () => {
expect(formatErrorDetail('Profile not found', FALLBACK)).toBe('Profile not found');
});
test('returns empty string details as-is (not the fallback)', () => {
expect(formatErrorDetail('', FALLBACK)).toBe('');
});
test('joins FastAPI validation error arrays on msg', () => {
const detail = [
{ loc: ['body', 'text'], msg: 'field required', type: 'value_error.missing' },
{ loc: ['body', 'seed'], msg: 'value is not a valid integer', type: 'type_error.integer' },
];
expect(formatErrorDetail(detail, FALLBACK)).toBe(
'field required; value is not a valid integer',
);
});
test('falls back to message key within array entries', () => {
expect(formatErrorDetail([{ message: 'boom' }], FALLBACK)).toBe('boom');
});
test('stringifies array entries with neither msg nor message', () => {
expect(formatErrorDetail([{ code: 42 }], FALLBACK)).toBe('{"code":42}');
});
test('returns empty string for an empty array', () => {
expect(formatErrorDetail([], FALLBACK)).toBe('');
});
test('uses message property of object details', () => {
expect(formatErrorDetail({ message: 'engine offline' }, FALLBACK)).toBe('engine offline');
});
test('stringifies objects without a string message', () => {
expect(formatErrorDetail({ message: 42, hint: 'x' }, FALLBACK)).toBe(
'{"message":42,"hint":"x"}',
);
expect(formatErrorDetail({ error: 'nested' }, FALLBACK)).toBe('{"error":"nested"}');
});
test('falls back for null, undefined, and primitives', () => {
expect(formatErrorDetail(null, FALLBACK)).toBe(FALLBACK);
expect(formatErrorDetail(undefined, FALLBACK)).toBe(FALLBACK);
expect(formatErrorDetail(404, FALLBACK)).toBe(FALLBACK);
expect(formatErrorDetail(true, FALLBACK)).toBe(FALLBACK);
});
test('preserves unicode in messages', () => {
expect(formatErrorDetail('模型未加载 🎙️', FALLBACK)).toBe('模型未加载 🎙️');
});
});
-21
View File
@@ -1,21 +0,0 @@
/**
* Normalizes a FastAPI error `detail` payload into a human-readable message.
*
* FastAPI returns `detail` as a plain string for HTTPException, an array of
* validation error objects for 422 responses, or an arbitrary object for
* custom handlers. Anything unrecognized falls back to the provided default.
*/
export function formatErrorDetail(detail: unknown, fallback: string): string {
if (typeof detail === 'string') return detail;
if (Array.isArray(detail)) {
return detail
.map((e: Record<string, unknown>) => e.msg || e.message || JSON.stringify(e))
.join('; ');
}
if (detail && typeof detail === 'object') {
const obj = detail as Record<string, unknown>;
if (typeof obj.message === 'string') return obj.message;
return JSON.stringify(detail);
}
return fallback;
}
+44
View File
@@ -0,0 +1,44 @@
/* generated using openapi-typescript-codegen -- do not edit */
/* istanbul ignore file */
/* tslint:disable */
/* eslint-disable */
export { ApiError } from './core/ApiError';
export { CancelablePromise, CancelError } from './core/CancelablePromise';
export { OpenAPI } from './core/OpenAPI';
export type { OpenAPIConfig } from './core/OpenAPI';
export type { Body_add_profile_sample_profiles__profile_id__samples_post } from './models/Body_add_profile_sample_profiles__profile_id__samples_post';
export type { Body_transcribe_audio_transcribe_post } from './models/Body_transcribe_audio_transcribe_post';
export type { GenerationRequest } from './models/GenerationRequest';
export type { GenerationResponse } from './models/GenerationResponse';
export type { HealthResponse } from './models/HealthResponse';
export type { HistoryListResponse } from './models/HistoryListResponse';
export type { HistoryResponse } from './models/HistoryResponse';
export type { HTTPValidationError } from './models/HTTPValidationError';
export type { ModelDownloadRequest } from './models/ModelDownloadRequest';
export type { ModelStatus } from './models/ModelStatus';
export type { ModelStatusListResponse } from './models/ModelStatusListResponse';
export type { ProfileSampleResponse } from './models/ProfileSampleResponse';
export type { TranscriptionResponse } from './models/TranscriptionResponse';
export type { ValidationError } from './models/ValidationError';
export type { VoiceProfileCreate } from './models/VoiceProfileCreate';
export type { VoiceProfileResponse } from './models/VoiceProfileResponse';
export { $Body_add_profile_sample_profiles__profile_id__samples_post } from './schemas/$Body_add_profile_sample_profiles__profile_id__samples_post';
export { $Body_transcribe_audio_transcribe_post } from './schemas/$Body_transcribe_audio_transcribe_post';
export { $GenerationRequest } from './schemas/$GenerationRequest';
export { $GenerationResponse } from './schemas/$GenerationResponse';
export { $HealthResponse } from './schemas/$HealthResponse';
export { $HistoryListResponse } from './schemas/$HistoryListResponse';
export { $HistoryResponse } from './schemas/$HistoryResponse';
export { $HTTPValidationError } from './schemas/$HTTPValidationError';
export { $ModelDownloadRequest } from './schemas/$ModelDownloadRequest';
export { $ModelStatus } from './schemas/$ModelStatus';
export { $ModelStatusListResponse } from './schemas/$ModelStatusListResponse';
export { $ProfileSampleResponse } from './schemas/$ProfileSampleResponse';
export { $TranscriptionResponse } from './schemas/$TranscriptionResponse';
export { $ValidationError } from './schemas/$ValidationError';
export { $VoiceProfileCreate } from './schemas/$VoiceProfileCreate';
export { $VoiceProfileResponse } from './schemas/$VoiceProfileResponse';
export { DefaultService } from './services/DefaultService';
@@ -0,0 +1,8 @@
/* generated using openapi-typescript-codegen -- do not edit */
/* istanbul ignore file */
/* tslint:disable */
/* eslint-disable */
export type Body_add_profile_sample_profiles__profile_id__samples_post = {
file: Blob;
reference_text: string;
};
@@ -0,0 +1,8 @@
/* generated using openapi-typescript-codegen -- do not edit */
/* istanbul ignore file */
/* tslint:disable */
/* eslint-disable */
export type Body_transcribe_audio_transcribe_post = {
file: Blob;
language?: string | null;
};
@@ -0,0 +1,15 @@
/* generated using openapi-typescript-codegen -- do not edit */
/* istanbul ignore file */
/* tslint:disable */
/* eslint-disable */
/**
* Request model for voice generation.
*/
export type GenerationRequest = {
profile_id: string;
text: string;
language?: string;
seed?: number | null;
model_size?: string | null;
instruct?: string | null;
};
@@ -0,0 +1,18 @@
/* generated using openapi-typescript-codegen -- do not edit */
/* istanbul ignore file */
/* tslint:disable */
/* eslint-disable */
/**
* Response model for voice generation.
*/
export type GenerationResponse = {
id: string;
profile_id: string;
text: string;
language: string;
audio_path: string;
duration: number;
seed: number | null;
instruct: string | null;
created_at: string;
};
@@ -0,0 +1,8 @@
/* generated using openapi-typescript-codegen -- do not edit */
/* istanbul ignore file */
/* tslint:disable */
/* eslint-disable */
import type { ValidationError } from './ValidationError';
export type HTTPValidationError = {
detail?: Array<ValidationError>;
};
+15
View File
@@ -0,0 +1,15 @@
/* generated using openapi-typescript-codegen -- do not edit */
/* istanbul ignore file */
/* tslint:disable */
/* eslint-disable */
/**
* Response model for health check.
*/
export type HealthResponse = {
status: string;
model_loaded: boolean;
model_downloaded?: boolean | null;
model_size?: string | null;
gpu_available: boolean;
vram_used_mb?: number | null;
};
@@ -0,0 +1,12 @@
/* generated using openapi-typescript-codegen -- do not edit */
/* istanbul ignore file */
/* tslint:disable */
/* eslint-disable */
import type { HistoryResponse } from './HistoryResponse';
/**
* Response model for history list.
*/
export type HistoryListResponse = {
items: Array<HistoryResponse>;
total: number;
};
+19
View File
@@ -0,0 +1,19 @@
/* generated using openapi-typescript-codegen -- do not edit */
/* istanbul ignore file */
/* tslint:disable */
/* eslint-disable */
/**
* Response model for history entry (includes profile name).
*/
export type HistoryResponse = {
id: string;
profile_id: string;
profile_name: string;
text: string;
language: string;
audio_path: string;
duration: number;
seed: number | null;
instruct: string | null;
created_at: string;
};
@@ -0,0 +1,10 @@
/* generated using openapi-typescript-codegen -- do not edit */
/* istanbul ignore file */
/* tslint:disable */
/* eslint-disable */
/**
* Request model for triggering model download.
*/
export type ModelDownloadRequest = {
model_name: string;
};
+15
View File
@@ -0,0 +1,15 @@
/* generated using openapi-typescript-codegen -- do not edit */
/* istanbul ignore file */
/* tslint:disable */
/* eslint-disable */
/**
* Response model for model status.
*/
export type ModelStatus = {
model_name: string;
display_name: string;
downloaded: boolean;
downloading?: boolean; // True if download is in progress
size_mb?: number | null;
loaded?: boolean;
};
@@ -0,0 +1,11 @@
/* generated using openapi-typescript-codegen -- do not edit */
/* istanbul ignore file */
/* tslint:disable */
/* eslint-disable */
import type { ModelStatus } from './ModelStatus';
/**
* Response model for model status list.
*/
export type ModelStatusListResponse = {
models: Array<ModelStatus>;
};
@@ -0,0 +1,13 @@
/* generated using openapi-typescript-codegen -- do not edit */
/* istanbul ignore file */
/* tslint:disable */
/* eslint-disable */
/**
* Response model for profile sample.
*/
export type ProfileSampleResponse = {
id: string;
profile_id: string;
audio_path: string;
reference_text: string;
};
@@ -0,0 +1,11 @@
/* generated using openapi-typescript-codegen -- do not edit */
/* istanbul ignore file */
/* tslint:disable */
/* eslint-disable */
/**
* Response model for transcription.
*/
export type TranscriptionResponse = {
text: string;
duration: number;
};
@@ -0,0 +1,9 @@
/* generated using openapi-typescript-codegen -- do not edit */
/* istanbul ignore file */
/* tslint:disable */
/* eslint-disable */
export type ValidationError = {
loc: Array<string | number>;
msg: string;
type: string;
};
@@ -0,0 +1,12 @@
/* generated using openapi-typescript-codegen -- do not edit */
/* istanbul ignore file */
/* tslint:disable */
/* eslint-disable */
/**
* Request model for creating a voice profile.
*/
export type VoiceProfileCreate = {
name: string;
description?: string | null;
language?: string;
};
@@ -0,0 +1,15 @@
/* generated using openapi-typescript-codegen -- do not edit */
/* istanbul ignore file */
/* tslint:disable */
/* eslint-disable */
/**
* Response model for voice profile.
*/
export type VoiceProfileResponse = {
id: string;
name: string;
description: string | null;
language: string;
created_at: string;
updated_at: string;
};
@@ -0,0 +1,17 @@
/* generated using openapi-typescript-codegen -- do not edit */
/* istanbul ignore file */
/* tslint:disable */
/* eslint-disable */
export const $Body_add_profile_sample_profiles__profile_id__samples_post = {
properties: {
file: {
type: 'binary',
isRequired: true,
format: 'binary',
},
reference_text: {
type: 'string',
isRequired: true,
},
},
} as const;
@@ -0,0 +1,24 @@
/* generated using openapi-typescript-codegen -- do not edit */
/* istanbul ignore file */
/* tslint:disable */
/* eslint-disable */
export const $Body_transcribe_audio_transcribe_post = {
properties: {
file: {
type: 'binary',
isRequired: true,
format: 'binary',
},
language: {
type: 'any-of',
contains: [
{
type: 'string',
},
{
type: 'null',
},
],
},
},
} as const;
@@ -0,0 +1,46 @@
/* generated using openapi-typescript-codegen -- do not edit */
/* istanbul ignore file */
/* tslint:disable */
/* eslint-disable */
export const $GenerationRequest = {
description: `Request model for voice generation.`,
properties: {
profile_id: {
type: 'string',
isRequired: true,
},
text: {
type: 'string',
isRequired: true,
maxLength: 5000,
minLength: 1,
},
language: {
type: 'string',
pattern: '^(en|zh)$',
},
seed: {
type: 'any-of',
contains: [
{
type: 'number',
},
{
type: 'null',
},
],
},
model_size: {
type: 'any-of',
contains: [
{
type: 'string',
pattern: '^(1\\.7B|0\\.6B)$',
},
{
type: 'null',
},
],
},
},
} as const;
@@ -0,0 +1,50 @@
/* generated using openapi-typescript-codegen -- do not edit */
/* istanbul ignore file */
/* tslint:disable */
/* eslint-disable */
export const $GenerationResponse = {
description: `Response model for voice generation.`,
properties: {
id: {
type: 'string',
isRequired: true,
},
profile_id: {
type: 'string',
isRequired: true,
},
text: {
type: 'string',
isRequired: true,
},
language: {
type: 'string',
isRequired: true,
},
audio_path: {
type: 'string',
isRequired: true,
},
duration: {
type: 'number',
isRequired: true,
},
seed: {
type: 'any-of',
contains: [
{
type: 'number',
},
{
type: 'null',
},
],
isRequired: true,
},
created_at: {
type: 'string',
isRequired: true,
format: 'date-time',
},
},
} as const;
@@ -0,0 +1,14 @@
/* generated using openapi-typescript-codegen -- do not edit */
/* istanbul ignore file */
/* tslint:disable */
/* eslint-disable */
export const $HTTPValidationError = {
properties: {
detail: {
type: 'array',
contains: {
type: 'ValidationError',
},
},
},
} as const;
@@ -0,0 +1,54 @@
/* generated using openapi-typescript-codegen -- do not edit */
/* istanbul ignore file */
/* tslint:disable */
/* eslint-disable */
export const $HealthResponse = {
description: `Response model for health check.`,
properties: {
status: {
type: 'string',
isRequired: true,
},
model_loaded: {
type: 'boolean',
isRequired: true,
},
model_downloaded: {
type: 'any-of',
contains: [
{
type: 'boolean',
},
{
type: 'null',
},
],
},
model_size: {
type: 'any-of',
contains: [
{
type: 'string',
},
{
type: 'null',
},
],
},
gpu_available: {
type: 'boolean',
isRequired: true,
},
vram_used_mb: {
type: 'any-of',
contains: [
{
type: 'number',
},
{
type: 'null',
},
],
},
},
} as const;
@@ -0,0 +1,20 @@
/* generated using openapi-typescript-codegen -- do not edit */
/* istanbul ignore file */
/* tslint:disable */
/* eslint-disable */
export const $HistoryListResponse = {
description: `Response model for history list.`,
properties: {
items: {
type: 'array',
contains: {
type: 'HistoryResponse',
},
isRequired: true,
},
total: {
type: 'number',
isRequired: true,
},
},
} as const;
@@ -0,0 +1,54 @@
/* generated using openapi-typescript-codegen -- do not edit */
/* istanbul ignore file */
/* tslint:disable */
/* eslint-disable */
export const $HistoryResponse = {
description: `Response model for history entry (includes profile name).`,
properties: {
id: {
type: 'string',
isRequired: true,
},
profile_id: {
type: 'string',
isRequired: true,
},
profile_name: {
type: 'string',
isRequired: true,
},
text: {
type: 'string',
isRequired: true,
},
language: {
type: 'string',
isRequired: true,
},
audio_path: {
type: 'string',
isRequired: true,
},
duration: {
type: 'number',
isRequired: true,
},
seed: {
type: 'any-of',
contains: [
{
type: 'number',
},
{
type: 'null',
},
],
isRequired: true,
},
created_at: {
type: 'string',
isRequired: true,
format: 'date-time',
},
},
} as const;
@@ -0,0 +1,13 @@
/* generated using openapi-typescript-codegen -- do not edit */
/* istanbul ignore file */
/* tslint:disable */
/* eslint-disable */
export const $ModelDownloadRequest = {
description: `Request model for triggering model download.`,
properties: {
model_name: {
type: 'string',
isRequired: true,
},
},
} as const;
+35
View File
@@ -0,0 +1,35 @@
/* generated using openapi-typescript-codegen -- do not edit */
/* istanbul ignore file */
/* tslint:disable */
/* eslint-disable */
export const $ModelStatus = {
description: `Response model for model status.`,
properties: {
model_name: {
type: 'string',
isRequired: true,
},
display_name: {
type: 'string',
isRequired: true,
},
downloaded: {
type: 'boolean',
isRequired: true,
},
size_mb: {
type: 'any-of',
contains: [
{
type: 'number',
},
{
type: 'null',
},
],
},
loaded: {
type: 'boolean',
},
},
} as const;
@@ -0,0 +1,16 @@
/* generated using openapi-typescript-codegen -- do not edit */
/* istanbul ignore file */
/* tslint:disable */
/* eslint-disable */
export const $ModelStatusListResponse = {
description: `Response model for model status list.`,
properties: {
models: {
type: 'array',
contains: {
type: 'ModelStatus',
},
isRequired: true,
},
},
} as const;
@@ -0,0 +1,25 @@
/* generated using openapi-typescript-codegen -- do not edit */
/* istanbul ignore file */
/* tslint:disable */
/* eslint-disable */
export const $ProfileSampleResponse = {
description: `Response model for profile sample.`,
properties: {
id: {
type: 'string',
isRequired: true,
},
profile_id: {
type: 'string',
isRequired: true,
},
audio_path: {
type: 'string',
isRequired: true,
},
reference_text: {
type: 'string',
isRequired: true,
},
},
} as const;
@@ -0,0 +1,17 @@
/* generated using openapi-typescript-codegen -- do not edit */
/* istanbul ignore file */
/* tslint:disable */
/* eslint-disable */
export const $TranscriptionResponse = {
description: `Response model for transcription.`,
properties: {
text: {
type: 'string',
isRequired: true,
},
duration: {
type: 'number',
isRequired: true,
},
},
} as const;
@@ -0,0 +1,31 @@
/* generated using openapi-typescript-codegen -- do not edit */
/* istanbul ignore file */
/* tslint:disable */
/* eslint-disable */
export const $ValidationError = {
properties: {
loc: {
type: 'array',
contains: {
type: 'any-of',
contains: [
{
type: 'string',
},
{
type: 'number',
},
],
},
isRequired: true,
},
msg: {
type: 'string',
isRequired: true,
},
type: {
type: 'string',
isRequired: true,
},
},
} as const;
@@ -0,0 +1,31 @@
/* generated using openapi-typescript-codegen -- do not edit */
/* istanbul ignore file */
/* tslint:disable */
/* eslint-disable */
export const $VoiceProfileCreate = {
description: `Request model for creating a voice profile.`,
properties: {
name: {
type: 'string',
isRequired: true,
maxLength: 100,
minLength: 1,
},
description: {
type: 'any-of',
contains: [
{
type: 'string',
maxLength: 500,
},
{
type: 'null',
},
],
},
language: {
type: 'string',
pattern: '^(en|zh)$',
},
},
} as const;
@@ -0,0 +1,43 @@
/* generated using openapi-typescript-codegen -- do not edit */
/* istanbul ignore file */
/* tslint:disable */
/* eslint-disable */
export const $VoiceProfileResponse = {
description: `Response model for voice profile.`,
properties: {
id: {
type: 'string',
isRequired: true,
},
name: {
type: 'string',
isRequired: true,
},
description: {
type: 'any-of',
contains: [
{
type: 'string',
},
{
type: 'null',
},
],
isRequired: true,
},
language: {
type: 'string',
isRequired: true,
},
created_at: {
type: 'string',
isRequired: true,
format: 'date-time',
},
updated_at: {
type: 'string',
isRequired: true,
format: 'date-time',
},
},
} as const;
+459
View File
@@ -0,0 +1,459 @@
/* generated using openapi-typescript-codegen -- do not edit */
/* istanbul ignore file */
/* tslint:disable */
/* eslint-disable */
import type { Body_add_profile_sample_profiles__profile_id__samples_post } from '../models/Body_add_profile_sample_profiles__profile_id__samples_post';
import type { Body_transcribe_audio_transcribe_post } from '../models/Body_transcribe_audio_transcribe_post';
import type { GenerationRequest } from '../models/GenerationRequest';
import type { GenerationResponse } from '../models/GenerationResponse';
import type { HealthResponse } from '../models/HealthResponse';
import type { HistoryListResponse } from '../models/HistoryListResponse';
import type { HistoryResponse } from '../models/HistoryResponse';
import type { ModelDownloadRequest } from '../models/ModelDownloadRequest';
import type { ModelStatusListResponse } from '../models/ModelStatusListResponse';
import type { ProfileSampleResponse } from '../models/ProfileSampleResponse';
import type { TranscriptionResponse } from '../models/TranscriptionResponse';
import type { VoiceProfileCreate } from '../models/VoiceProfileCreate';
import type { VoiceProfileResponse } from '../models/VoiceProfileResponse';
import type { CancelablePromise } from '../core/CancelablePromise';
import { OpenAPI } from '../core/OpenAPI';
import { request as __request } from '../core/request';
export class DefaultService {
/**
* Root
* Root endpoint.
* @returns any Successful Response
* @throws ApiError
*/
public static rootGet(): CancelablePromise<any> {
return __request(OpenAPI, {
method: 'GET',
url: '/',
});
}
/**
* Health
* Health check endpoint.
* @returns HealthResponse Successful Response
* @throws ApiError
*/
public static healthHealthGet(): CancelablePromise<HealthResponse> {
return __request(OpenAPI, {
method: 'GET',
url: '/health',
});
}
/**
* List Profiles
* List all voice profiles.
* @returns VoiceProfileResponse Successful Response
* @throws ApiError
*/
public static listProfilesProfilesGet(): CancelablePromise<Array<VoiceProfileResponse>> {
return __request(OpenAPI, {
method: 'GET',
url: '/profiles',
});
}
/**
* Create Profile
* Create a new voice profile.
* @returns VoiceProfileResponse Successful Response
* @throws ApiError
*/
public static createProfileProfilesPost({
requestBody,
}: {
requestBody: VoiceProfileCreate;
}): CancelablePromise<VoiceProfileResponse> {
return __request(OpenAPI, {
method: 'POST',
url: '/profiles',
body: requestBody,
mediaType: 'application/json',
errors: {
422: `Validation Error`,
},
});
}
/**
* Get Profile
* Get a voice profile by ID.
* @returns VoiceProfileResponse Successful Response
* @throws ApiError
*/
public static getProfileProfilesProfileIdGet({
profileId,
}: {
profileId: string;
}): CancelablePromise<VoiceProfileResponse> {
return __request(OpenAPI, {
method: 'GET',
url: '/profiles/{profile_id}',
path: {
profile_id: profileId,
},
errors: {
422: `Validation Error`,
},
});
}
/**
* Update Profile
* Update a voice profile.
* @returns VoiceProfileResponse Successful Response
* @throws ApiError
*/
public static updateProfileProfilesProfileIdPut({
profileId,
requestBody,
}: {
profileId: string;
requestBody: VoiceProfileCreate;
}): CancelablePromise<VoiceProfileResponse> {
return __request(OpenAPI, {
method: 'PUT',
url: '/profiles/{profile_id}',
path: {
profile_id: profileId,
},
body: requestBody,
mediaType: 'application/json',
errors: {
422: `Validation Error`,
},
});
}
/**
* Delete Profile
* Delete a voice profile.
* @returns any Successful Response
* @throws ApiError
*/
public static deleteProfileProfilesProfileIdDelete({
profileId,
}: {
profileId: string;
}): CancelablePromise<any> {
return __request(OpenAPI, {
method: 'DELETE',
url: '/profiles/{profile_id}',
path: {
profile_id: profileId,
},
errors: {
422: `Validation Error`,
},
});
}
/**
* Add Profile Sample
* Add a sample to a voice profile.
* @returns ProfileSampleResponse Successful Response
* @throws ApiError
*/
public static addProfileSampleProfilesProfileIdSamplesPost({
profileId,
formData,
}: {
profileId: string;
formData: Body_add_profile_sample_profiles__profile_id__samples_post;
}): CancelablePromise<ProfileSampleResponse> {
return __request(OpenAPI, {
method: 'POST',
url: '/profiles/{profile_id}/samples',
path: {
profile_id: profileId,
},
formData: formData,
mediaType: 'multipart/form-data',
errors: {
422: `Validation Error`,
},
});
}
/**
* Get Profile Samples
* Get all samples for a profile.
* @returns ProfileSampleResponse Successful Response
* @throws ApiError
*/
public static getProfileSamplesProfilesProfileIdSamplesGet({
profileId,
}: {
profileId: string;
}): CancelablePromise<Array<ProfileSampleResponse>> {
return __request(OpenAPI, {
method: 'GET',
url: '/profiles/{profile_id}/samples',
path: {
profile_id: profileId,
},
errors: {
422: `Validation Error`,
},
});
}
/**
* Delete Profile Sample
* Delete a profile sample.
* @returns any Successful Response
* @throws ApiError
*/
public static deleteProfileSampleProfilesSamplesSampleIdDelete({
sampleId,
}: {
sampleId: string;
}): CancelablePromise<any> {
return __request(OpenAPI, {
method: 'DELETE',
url: '/profiles/samples/{sample_id}',
path: {
sample_id: sampleId,
},
errors: {
422: `Validation Error`,
},
});
}
/**
* Generate Speech
* Generate speech from text using a voice profile.
* @returns GenerationResponse Successful Response
* @throws ApiError
*/
public static generateSpeechGeneratePost({
requestBody,
}: {
requestBody: GenerationRequest;
}): CancelablePromise<GenerationResponse> {
return __request(OpenAPI, {
method: 'POST',
url: '/generate',
body: requestBody,
mediaType: 'application/json',
errors: {
422: `Validation Error`,
},
});
}
/**
* List History
* List generation history with optional filters.
* @returns HistoryListResponse Successful Response
* @throws ApiError
*/
public static listHistoryHistoryGet({
profileId,
search,
limit = 50,
offset,
}: {
profileId?: string | null;
search?: string | null;
limit?: number;
offset?: number;
}): CancelablePromise<HistoryListResponse> {
return __request(OpenAPI, {
method: 'GET',
url: '/history',
query: {
profile_id: profileId,
search: search,
limit: limit,
offset: offset,
},
errors: {
422: `Validation Error`,
},
});
}
/**
* Get Generation
* Get a generation by ID.
* @returns HistoryResponse Successful Response
* @throws ApiError
*/
public static getGenerationHistoryGenerationIdGet({
generationId,
}: {
generationId: string;
}): CancelablePromise<HistoryResponse> {
return __request(OpenAPI, {
method: 'GET',
url: '/history/{generation_id}',
path: {
generation_id: generationId,
},
errors: {
422: `Validation Error`,
},
});
}
/**
* Delete Generation
* Delete a generation.
* @returns any Successful Response
* @throws ApiError
*/
public static deleteGenerationHistoryGenerationIdDelete({
generationId,
}: {
generationId: string;
}): CancelablePromise<any> {
return __request(OpenAPI, {
method: 'DELETE',
url: '/history/{generation_id}',
path: {
generation_id: generationId,
},
errors: {
422: `Validation Error`,
},
});
}
/**
* Get Stats
* Get generation statistics.
* @returns any Successful Response
* @throws ApiError
*/
public static getStatsHistoryStatsGet(): CancelablePromise<any> {
return __request(OpenAPI, {
method: 'GET',
url: '/history/stats',
});
}
/**
* Transcribe Audio
* Transcribe audio file to text.
* @returns TranscriptionResponse Successful Response
* @throws ApiError
*/
public static transcribeAudioTranscribePost({
formData,
}: {
formData: Body_transcribe_audio_transcribe_post;
}): CancelablePromise<TranscriptionResponse> {
return __request(OpenAPI, {
method: 'POST',
url: '/transcribe',
formData: formData,
mediaType: 'multipart/form-data',
errors: {
422: `Validation Error`,
},
});
}
/**
* Get Audio
* Serve generated audio file.
* @returns any Successful Response
* @throws ApiError
*/
public static getAudioAudioGenerationIdGet({
generationId,
}: {
generationId: string;
}): CancelablePromise<any> {
return __request(OpenAPI, {
method: 'GET',
url: '/audio/{generation_id}',
path: {
generation_id: generationId,
},
errors: {
422: `Validation Error`,
},
});
}
/**
* Load Model
* Manually load TTS model.
* @returns any Successful Response
* @throws ApiError
*/
public static loadModelModelsLoadPost({
modelSize = '1.7B',
}: {
modelSize?: string;
}): CancelablePromise<any> {
return __request(OpenAPI, {
method: 'POST',
url: '/models/load',
query: {
model_size: modelSize,
},
errors: {
422: `Validation Error`,
},
});
}
/**
* Unload Model
* Unload TTS model to free memory.
* @returns any Successful Response
* @throws ApiError
*/
public static unloadModelModelsUnloadPost(): CancelablePromise<any> {
return __request(OpenAPI, {
method: 'POST',
url: '/models/unload',
});
}
/**
* Get Model Progress
* Get model download progress via Server-Sent Events.
* @returns any Successful Response
* @throws ApiError
*/
public static getModelProgressModelsProgressModelNameGet({
modelName,
}: {
modelName: string;
}): CancelablePromise<any> {
return __request(OpenAPI, {
method: 'GET',
url: '/models/progress/{model_name}',
path: {
model_name: modelName,
},
errors: {
422: `Validation Error`,
},
});
}
/**
* Get Model Status
* Get status of all available models.
* @returns ModelStatusListResponse Successful Response
* @throws ApiError
*/
public static getModelStatusModelsStatusGet(): CancelablePromise<ModelStatusListResponse> {
return __request(OpenAPI, {
method: 'GET',
url: '/models/status',
});
}
/**
* Trigger Model Download
* Trigger download of a specific model.
* @returns any Successful Response
* @throws ApiError
*/
public static triggerModelDownloadModelsDownloadPost({
requestBody,
}: {
requestBody: ModelDownloadRequest;
}): CancelablePromise<any> {
return __request(OpenAPI, {
method: 'POST',
url: '/models/download',
body: requestBody,
mediaType: 'application/json',
errors: {
422: `Validation Error`,
},
});
}
}
+4 -5
View File
@@ -213,10 +213,6 @@ export interface CaptureSettings {
/** Whether the global keyboard hotkey is armed. Off by default — turning
* this on triggers the macOS Input Monitoring TCC prompt. */
hotkey_enabled: boolean;
/** Hold the mic open while dictation is enabled so push-to-talk doesn't clip
* the first words. Off by default — when on, the OS mic indicator stays lit
* the whole time dictation is enabled. */
keep_mic_warm: boolean;
/** keytap key names. Defaults are platform-specific right-hand modifiers. */
chord_push_to_talk_keys: string[];
/** keytap key names. Toggle adds Space to the platform-specific PTT chord. */
@@ -291,7 +287,10 @@ export interface CudaDownloadProgress {
export interface CudaStatus {
available: boolean; // CUDA binary exists on disk
active: boolean; // Currently running the CUDA binary
binary_path?: string;
binary_path: string | null;
cuda_libs_version: string | null;
download_supported: boolean; // Platform has a matching release asset
unsupported_reason: string | null;
downloading: boolean; // Download in progress
download_progress?: CudaDownloadProgress;
}
-70
View File
@@ -1,70 +0,0 @@
import { describe, expect, test } from 'bun:test';
import {
ALL_LANGUAGES,
ENGINE_LANGUAGES,
LANGUAGE_CODES,
LANGUAGE_OPTIONS,
getLanguageOptionsForEngine,
} from './languages';
describe('ENGINE_LANGUAGES', () => {
test('every engine maps only to codes defined in ALL_LANGUAGES', () => {
for (const [engine, codes] of Object.entries(ENGINE_LANGUAGES)) {
for (const code of codes) {
expect(ALL_LANGUAGES[code], `${engine} references unknown code "${code}"`).toBeDefined();
}
}
});
test('no engine lists a language twice', () => {
for (const [engine, codes] of Object.entries(ENGINE_LANGUAGES)) {
expect(new Set(codes).size, `${engine} has duplicate codes`).toBe(codes.length);
}
});
test('every engine supports at least English', () => {
for (const codes of Object.values(ENGINE_LANGUAGES)) {
expect(codes).toContain('en');
}
});
test('English-only engines list exactly one language', () => {
expect(ENGINE_LANGUAGES.luxtts).toEqual(['en']);
expect(ENGINE_LANGUAGES.chatterbox_turbo).toEqual(['en']);
});
test('qwen and qwen_custom_voice support the same languages', () => {
expect(ENGINE_LANGUAGES.qwen_custom_voice).toEqual(ENGINE_LANGUAGES.qwen);
});
});
describe('getLanguageOptionsForEngine', () => {
test('builds value/label pairs from ALL_LANGUAGES', () => {
expect(getLanguageOptionsForEngine('luxtts')).toEqual([{ value: 'en', label: 'English' }]);
});
test('preserves the engine declaration order', () => {
const values = getLanguageOptionsForEngine('qwen').map((o) => o.value);
expect(values).toEqual([...ENGINE_LANGUAGES.qwen]);
});
test('falls back to qwen languages for unknown engines', () => {
expect(getLanguageOptionsForEngine('does-not-exist')).toEqual(
getLanguageOptionsForEngine('qwen'),
);
});
});
describe('language option exports', () => {
test('LANGUAGE_CODES covers every ALL_LANGUAGES key exactly once', () => {
const codes: string[] = [...LANGUAGE_CODES].sort();
expect(codes).toEqual(Object.keys(ALL_LANGUAGES).sort());
expect(new Set(LANGUAGE_CODES).size).toBe(LANGUAGE_CODES.length);
});
test('LANGUAGE_OPTIONS labels match ALL_LANGUAGES', () => {
for (const option of LANGUAGE_OPTIONS) {
expect(option.label).toBe(ALL_LANGUAGES[option.value]);
}
});
});
+139 -371
View File
@@ -4,45 +4,12 @@ import { convertToWav } from '@/lib/utils/audio';
interface UseAudioRecordingOptions {
maxDurationSeconds?: number;
// ``context`` is whatever was handed to ``startRecording`` for this take,
// threaded back untouched so callers can correlate the result with the
// recording it came from (the dictate window pairs it with the focus
// snapshot captured at chord-start).
onRecordingComplete?: (blob: Blob, duration?: number, context?: unknown) => void;
/**
* Keep the microphone ``MediaStream`` open between recordings instead of
* tearing it down on every stop. This is what removes the "first words get
* clipped" problem on push-to-talk dictation: ``getUserMedia`` on macOS can
* take several hundred ms — up to a second cold — to hand back a stream, and
* ``MediaRecorder`` only starts capturing *after* it resolves, so everything
* spoken in that window is lost. With a warm stream already open, the next
* ``startRecording`` skips ``getUserMedia`` entirely.
*
* Off by default: the voice-clone sample recorders release the device
* immediately, and the dictation session only opts in when the user enables
* the "keep microphone ready" setting. While on, the warm stream stays open —
* and the OS mic-in-use indicator stays lit — until it's explicitly released
* (dictation disabled or the setting turned off), so the trade-off is visible
* and user-controlled rather than a background mic that's always warm.
*/
keepWarm?: boolean;
onRecordingComplete?: (blob: Blob, duration?: number) => void;
}
// Audio constraints for capture. Kept identical to the previous inline value so
// this change is purely about *when* the stream is opened, not *how*.
const AUDIO_CONSTRAINTS: MediaTrackConstraints = {
echoCancellation: true,
noiseSuppression: true,
autoGainControl: true,
};
const streamHasLiveAudio = (stream: MediaStream | null): stream is MediaStream =>
!!stream && stream.getAudioTracks().some((t) => t.readyState === 'live');
export function useAudioRecording({
maxDurationSeconds,
onRecordingComplete,
keepWarm = false,
}: UseAudioRecordingOptions = {}) {
const platform = usePlatform();
const [isRecording, setIsRecording] = useState(false);
@@ -50,392 +17,195 @@ export function useAudioRecording({
const [error, setError] = useState<string | null>(null);
const mediaRecorderRef = useRef<MediaRecorder | null>(null);
const chunksRef = useRef<Blob[]>([]);
// The stream currently backing the MediaRecorder. When ``keepWarm`` is set
// this is the same object as ``warmStreamRef`` and is *not* torn down on
// stop; otherwise it's stopped as soon as the recording completes.
const streamRef = useRef<MediaStream | null>(null);
// Persistent pre-opened stream reused across recordings when ``keepWarm``.
const warmStreamRef = useRef<MediaStream | null>(null);
const timerRef = useRef<number | null>(null);
const startTimeRef = useRef<number | null>(null);
const cancelledRef = useRef<boolean>(false);
// Mirror of ``isRecording`` for reads inside callbacks that would otherwise
// close over a stale render.
const isRecordingRef = useRef(false);
// A ``getUserMedia`` call in flight, shared so concurrent acquirers (prewarm
// plus an immediate chord) coalesce onto one stream instead of each opening —
// and orphaning — their own.
const acquiringRef = useRef<Promise<MediaStream> | null>(null);
// True from ``startRecording`` entry until the recorder is actually running
// (or has failed), so a stop that arrives mid-acquisition can be deferred.
const startingRef = useRef(false);
// True from MediaRecorder.stop() until onstop has snapshotted the take's
// shared refs. React state and MediaRecorder.state both flip before onstop,
// so without this gate a rapid next chord can clear chunks/duration/cancel
// state out from under the recorder that is still finalising.
const finishingRef = useRef(false);
const pendingStopRef = useRef(false);
// Bumped per recording so a stale recorder's ``onstop`` can tell it's no
// longer the active one before it touches the shared stream refs.
const recordingCounterRef = useRef(0);
// Bumped whenever the warm stream is released/aborted so a ``getUserMedia``
// still in flight can tell its result is stale and stop it instead of
// adopting a live mic after disable/unmount.
const acquireGenRef = useRef(0);
// Set when a release is requested mid-recording; the onstop path performs the
// deferred release once capture finishes rather than yanking the device now.
const releaseAfterStopRef = useRef(false);
// Keeps the ref in lockstep with the state so the synchronous stop path reads
// a fresh value without waiting for a rerender.
const setRecording = useCallback((next: boolean) => {
isRecordingRef.current = next;
setIsRecording(next);
}, []);
const startRecording = useCallback(async () => {
try {
setError(null);
chunksRef.current = [];
cancelledRef.current = false;
setDuration(0);
const releaseWarmStream = useCallback(() => {
// Invalidate any getUserMedia still in flight so its stream is stopped on
// resolve rather than adopted as the warm stream.
acquireGenRef.current += 1;
// Don't tear the device out from under an active/starting recording — the
// warm stream is the one backing it; defer to the onstop path instead.
if (isRecordingRef.current || startingRef.current) {
releaseAfterStopRef.current = true;
return;
}
warmStreamRef.current?.getTracks().forEach((track) => {
track.stop();
});
warmStreamRef.current = null;
}, []);
// Assert that getUserMedia is reachable, mirroring the previous inline guard
// (Tauri webviews occasionally expose ``navigator.mediaDevices`` a beat late).
const assertMediaDevices = useCallback(async () => {
if (typeof navigator === 'undefined') {
throw new Error('Navigator API is not available. This might be a Tauri configuration issue.');
}
if (!navigator.mediaDevices || !navigator.mediaDevices.getUserMedia) {
await new Promise((resolve) => setTimeout(resolve, 100));
if (!navigator.mediaDevices || !navigator.mediaDevices.getUserMedia) {
throw new Error(
platform.metadata.isTauri
? 'Microphone access is not available. Please ensure:\n1. The app has microphone permissions in System Settings (macOS: System Settings > Privacy & Security > Microphone)\n2. You restart the app after granting permissions\n3. You are using Tauri v2 with a webview that supports getUserMedia'
: 'Microphone access is not available. Please ensure you are using a secure context (HTTPS or localhost) and that your browser has microphone permissions enabled.',
);
// Check if getUserMedia is available
// In Tauri, navigator.mediaDevices might not be available immediately
if (typeof navigator === 'undefined') {
const errorMsg =
'Navigator API is not available. This might be a Tauri configuration issue.';
setError(errorMsg);
throw new Error(errorMsg);
}
}
}, [platform.metadata.isTauri]);
// Return a live capture stream, reusing the warm one when available so the
// hot path (chord-down → record) never waits on getUserMedia.
const acquireStream = useCallback(async (): Promise<MediaStream> => {
// Captured separately so it stays typed as the full stream after the live
// check narrows ``warmStreamRef.current`` itself.
const existing = warmStreamRef.current;
if (streamHasLiveAudio(warmStreamRef.current)) {
return warmStreamRef.current;
}
// Coalesce concurrent acquirers onto one getUserMedia call so prewarm and
// an immediate chord can't open two streams.
if (acquiringRef.current) return acquiringRef.current;
// A dead warm stream (device unplugged / tracks ended) — drop it and reopen.
if (existing) {
existing.getTracks().forEach((track) => {
track.stop();
});
warmStreamRef.current = null;
}
const gen = acquireGenRef.current;
const acquisition = (async () => {
await assertMediaDevices();
if (!navigator.mediaDevices || !navigator.mediaDevices.getUserMedia) {
// Try waiting a bit for Tauri webview to initialize
await new Promise((resolve) => setTimeout(resolve, 100));
if (!navigator.mediaDevices || !navigator.mediaDevices.getUserMedia) {
console.error('MediaDevices check:', {
hasNavigator: typeof navigator !== 'undefined',
hasMediaDevices: !!navigator?.mediaDevices,
hasGetUserMedia: !!navigator?.mediaDevices?.getUserMedia,
isTauri: platform.metadata.isTauri,
});
const errorMsg = platform.metadata.isTauri
? 'Microphone access is not available. Please ensure:\n1. The app has microphone permissions in System Settings (macOS: System Settings > Privacy & Security > Microphone)\n2. You restart the app after granting permissions\n3. You are using Tauri v2 with a webview that supports getUserMedia'
: 'Microphone access is not available. Please ensure you are using a secure context (HTTPS or localhost) and that your browser has microphone permissions enabled.';
setError(errorMsg);
throw new Error(errorMsg);
}
}
// Request microphone access
const stream = await navigator.mediaDevices.getUserMedia({
audio: AUDIO_CONSTRAINTS,
audio: {
echoCancellation: true,
noiseSuppression: true,
autoGainControl: true,
},
});
// Released / disabled / unmounted while acquiring — this stream is stale,
// so stop it instead of leaving a live mic open, and abort the caller.
if (gen !== acquireGenRef.current) {
stream.getTracks().forEach((track) => {
streamRef.current = stream;
// Create MediaRecorder with preferred MIME type
const options: MediaRecorderOptions = {
mimeType: 'audio/webm;codecs=opus',
};
// Fallback to default if webm not supported
if (!MediaRecorder.isTypeSupported(options.mimeType!)) {
delete options.mimeType;
}
const mediaRecorder = new MediaRecorder(stream, options);
mediaRecorderRef.current = mediaRecorder;
mediaRecorder.ondataavailable = (event) => {
if (event.data.size > 0) {
chunksRef.current.push(event.data);
}
};
mediaRecorder.onstop = async () => {
// Snapshot the cancellation flag and recorded duration immediately —
// cancelRecording() clears chunks and sets cancelledRef synchronously
// before this async handler runs, so we must check it first.
const wasCancelled = cancelledRef.current;
const recordedDuration = startTimeRef.current
? (Date.now() - startTimeRef.current) / 1000
: undefined;
const webmBlob = new Blob(chunksRef.current, { type: 'audio/webm' });
// Stop all tracks now that we have the data
streamRef.current?.getTracks().forEach((track) => {
track.stop();
});
throw new Error('microphone acquisition aborted');
}
if (keepWarm) warmStreamRef.current = stream;
return stream;
})();
acquiringRef.current = acquisition;
try {
return await acquisition;
} finally {
if (acquiringRef.current === acquisition) acquiringRef.current = null;
}
}, [assertMediaDevices, keepWarm]);
streamRef.current = null;
/**
* Open the microphone ahead of the first recording so the initial dictation
* doesn't clip. No-op unless ``keepWarm`` is set. Safe to call repeatedly and
* safe to fail (e.g. permission not yet granted) — ``startRecording`` still
* surfaces a real error if capture is genuinely unavailable.
*/
const prewarm = useCallback(async () => {
if (!keepWarm) return;
try {
await acquireStream();
} catch {
// Permission missing / device busy / aborted — recording will report a
// real error if capture is genuinely unavailable.
}
}, [keepWarm, acquireStream]);
// Don't fire completion callback if the recording was cancelled
if (wasCancelled) return;
const startRecording = useCallback(
async (context?: unknown) => {
// A second chord can arrive while the first one is still waiting on
// getUserMedia. Never create overlapping MediaRecorders on the same
// coalesced stream; the original take will honor any deferred stop.
if (
startingRef.current ||
finishingRef.current ||
mediaRecorderRef.current?.state === 'recording'
)
return;
startingRef.current = true;
pendingStopRef.current = false;
// A new recording supersedes any release deferred from a prior take.
releaseAfterStopRef.current = false;
const recordingId = ++recordingCounterRef.current;
try {
setError(null);
chunksRef.current = [];
cancelledRef.current = false;
setDuration(0);
// Reuse the warm stream when present (instant); otherwise open one now.
const stream = await acquireStream();
streamRef.current = stream;
// Create MediaRecorder with preferred MIME type
const options: MediaRecorderOptions = {
mimeType: 'audio/webm;codecs=opus',
};
// Fallback to default if webm not supported
if (!MediaRecorder.isTypeSupported(options.mimeType!)) {
delete options.mimeType;
// Convert to WAV format to avoid needing ffmpeg on backend
try {
const wavBlob = await convertToWav(webmBlob);
onRecordingComplete?.(wavBlob, recordedDuration);
} catch (err) {
console.error('Error converting audio to WAV:', err);
// Fallback to original blob if conversion fails
onRecordingComplete?.(webmBlob, recordedDuration);
}
};
const mediaRecorder = new MediaRecorder(stream, options);
mediaRecorderRef.current = mediaRecorder;
mediaRecorder.onerror = (event) => {
setError('Recording error occurred');
console.error('MediaRecorder error:', event);
};
mediaRecorder.ondataavailable = (event) => {
if (event.data.size > 0) {
chunksRef.current.push(event.data);
}
};
// WebKit's MediaRecorder drops the WebM EBML header from chunks when
// started with a timeslice, so concatenated blobs fail to parse in
// both AudioContext and ffmpeg. Starting with no timeslice produces
// exactly one dataavailable on stop() with a valid container.
mediaRecorder.start();
setIsRecording(true);
startTimeRef.current = Date.now();
mediaRecorder.onstop = async () => {
// Whether this recorder is still the active one. A stale onstop (an
// older recorder stopping after a newer startRecording) must not touch
// the shared stream refs.
const isCurrent = recordingCounterRef.current === recordingId;
// Snapshot the cancellation flag and recorded duration immediately —
// cancelRecording() clears chunks and sets cancelledRef synchronously
// before this async handler runs, so we must check it first.
const wasCancelled = cancelledRef.current;
const recordedDuration = startTimeRef.current
? (Date.now() - startTimeRef.current) / 1000
: undefined;
// Start timer
timerRef.current = window.setInterval(() => {
if (startTimeRef.current) {
const elapsed = (Date.now() - startTimeRef.current) / 1000;
setDuration(elapsed);
const webmBlob = new Blob(chunksRef.current, { type: 'audio/webm' });
// Release the device unless we're keeping it warm for the next capture.
// Act on this recorder's own stream; only touch the shared refs when
// this is still the current recording.
if (keepWarm) {
if (isCurrent) {
streamRef.current = null;
// A release requested mid-recording (dictation disabled) is
// honored now that capture has finished; otherwise the warm
// stream stays open for the next take.
if (releaseAfterStopRef.current) {
releaseAfterStopRef.current = false;
releaseWarmStream();
}
}
} else {
stream.getTracks().forEach((track) => {
track.stop();
});
if (isCurrent) streamRef.current = null;
}
// All shared per-take refs have now been snapshotted and stream
// cleanup is complete. A new take may begin while WAV conversion and
// upload continue using the local values above.
finishingRef.current = false;
// Don't fire completion callback if the recording was cancelled
if (wasCancelled) return;
// Convert to WAV format to avoid needing ffmpeg on backend
try {
const wavBlob = await convertToWav(webmBlob);
onRecordingComplete?.(wavBlob, recordedDuration, context);
} catch (err) {
console.error('Error converting audio to WAV:', err);
// Fallback to original blob if conversion fails
onRecordingComplete?.(webmBlob, recordedDuration, context);
}
};
mediaRecorder.onerror = (event) => {
setError('Recording error occurred');
console.error('MediaRecorder error:', event);
};
// WebKit's MediaRecorder drops the WebM EBML header from chunks when
// started with a timeslice, so concatenated blobs fail to parse in
// both AudioContext and ffmpeg. Starting with no timeslice produces
// exactly one dataavailable on stop() with a valid container.
mediaRecorder.start();
setRecording(true);
startTimeRef.current = Date.now();
startingRef.current = false;
// A stop (chord release) that landed while the mic was still opening —
// honor it now that capture has actually begun.
if (pendingStopRef.current) {
pendingStopRef.current = false;
finishingRef.current = true;
mediaRecorder.stop();
setRecording(false);
return;
}
// Start timer
timerRef.current = window.setInterval(() => {
if (startTimeRef.current) {
const elapsed = (Date.now() - startTimeRef.current) / 1000;
setDuration(elapsed);
// Auto-stop at max duration when the caller opts in — dictation
// sessions pass undefined and run until the user releases the
// chord or hits stop; voice-clone sample recorders pass 29s to
// keep reference clips short.
if (maxDurationSeconds !== undefined && elapsed >= maxDurationSeconds) {
if (mediaRecorderRef.current && mediaRecorderRef.current.state !== 'inactive') {
finishingRef.current = true;
mediaRecorderRef.current.stop();
setRecording(false);
if (timerRef.current !== null) {
clearInterval(timerRef.current);
timerRef.current = null;
}
// Auto-stop at max duration when the caller opts in — dictation
// sessions pass undefined and run until the user releases the
// chord or hits stop; voice-clone sample recorders pass 29s to
// keep reference clips short.
if (maxDurationSeconds !== undefined && elapsed >= maxDurationSeconds) {
if (mediaRecorderRef.current && mediaRecorderRef.current.state !== 'inactive') {
mediaRecorderRef.current.stop();
setIsRecording(false);
if (timerRef.current !== null) {
clearInterval(timerRef.current);
timerRef.current = null;
}
}
}
}, 100);
} catch (err) {
const errorMessage =
err instanceof Error
? err.message
: 'Failed to access microphone. Please check permissions.';
// A fresh (non-warm) stream opened before the failure must be released
// so the mic doesn't stay lit; a warm stream is reusable, so it's kept.
if (!keepWarm) {
streamRef.current?.getTracks().forEach((track) => {
track.stop();
});
streamRef.current = null;
}
startingRef.current = false;
finishingRef.current = false;
pendingStopRef.current = false;
setError(errorMessage);
setRecording(false);
}
},
[
maxDurationSeconds,
onRecordingComplete,
acquireStream,
keepWarm,
releaseWarmStream,
setRecording,
],
);
}, 100);
} catch (err) {
const errorMessage =
err instanceof Error
? err.message
: 'Failed to access microphone. Please check permissions.';
setError(errorMessage);
setIsRecording(false);
}
}, [maxDurationSeconds, onRecordingComplete]);
const stopRecording = useCallback(() => {
// The recorder's own state is the lifecycle authority — React ``isRecording``
// lags a render behind ``mediaRecorder.start()``, so a chord release in that
// window would otherwise be dropped.
const recorder = mediaRecorderRef.current;
if (recorder && recorder.state === 'recording') {
finishingRef.current = true;
recorder.stop();
setRecording(false);
if (mediaRecorderRef.current && isRecording) {
mediaRecorderRef.current.stop();
setIsRecording(false);
if (timerRef.current !== null) {
clearInterval(timerRef.current);
timerRef.current = null;
}
} else if (startingRef.current) {
// Stop arrived before capture began (mic still opening) — defer it so
// startRecording stops as soon as the recorder goes live.
pendingStopRef.current = true;
}
}, [setRecording]);
}, [isRecording]);
const cancelRecording = useCallback(() => {
cancelledRef.current = true; // Must be set before stop() triggers onstop
const recorder = mediaRecorderRef.current;
if (recorder && recorder.state !== 'inactive') {
if (mediaRecorderRef.current) {
cancelledRef.current = true; // Must be set before stop() triggers onstop
chunksRef.current = [];
finishingRef.current = true;
recorder.stop();
setRecording(false);
mediaRecorderRef.current.stop();
setIsRecording(false);
setDuration(0);
} else if (startingRef.current) {
// Cancel during mic acquisition — stop as soon as capture begins; the
// cancelled flag suppresses the completion callback.
pendingStopRef.current = true;
}
// Keep the device warm for the next capture when opted in; otherwise stop
// the tracks so the mic is released immediately.
if (keepWarm) {
streamRef.current = null;
if (releaseAfterStopRef.current) {
releaseAfterStopRef.current = false;
releaseWarmStream();
}
} else {
streamRef.current?.getTracks().forEach((track) => {
track.stop();
});
streamRef.current = null;
}
// Stop all tracks
streamRef.current?.getTracks().forEach((track) => {
track.stop();
});
streamRef.current = null;
if (timerRef.current !== null) {
clearInterval(timerRef.current);
timerRef.current = null;
}
}, [keepWarm, releaseWarmStream, setRecording]);
}, []);
// Cleanup on unmount — always fully release the device, warm or not.
// Cleanup on unmount
useEffect(() => {
return () => {
// Invalidate any in-flight acquisition so a stream resolving after unmount
// stops itself instead of leaking a live mic.
acquireGenRef.current += 1;
if (timerRef.current !== null) {
clearInterval(timerRef.current);
}
streamRef.current?.getTracks().forEach((track) => {
track.stop();
});
warmStreamRef.current?.getTracks().forEach((track) => {
track.stop();
});
};
}, []);
@@ -446,7 +216,5 @@ export function useAudioRecording({
startRecording,
stopRecording,
cancelRecording,
prewarm,
releaseWarm: releaseWarmStream,
};
}
+23 -60
View File
@@ -54,15 +54,11 @@ const SHORT_RECORDING_MESSAGE = 'Recording too short, canceled';
export type CapturePillState = PillState | 'hidden';
export interface UseCaptureRecordingSessionOptions {
/** Keep the microphone stream open between dictations when explicitly
* enabled. Off by default so normal recorders release the device. */
keepMicWarm?: boolean;
/**
* Fired after a capture row is created on the server. Callers can use this
* to select the new capture or emit a Tauri event to a sibling window.
* ``context`` is whatever was passed to ``startRecording`` for this take.
*/
onCaptureCreated?: (capture: CaptureResponse, context?: unknown) => void;
onCaptureCreated?: (capture: CaptureResponse) => void;
/**
* Fired with the final delivered text — refined if ``auto_refine`` was on
* for this capture, raw transcript otherwise. Used by the floating
@@ -70,14 +66,12 @@ export interface UseCaptureRecordingSessionOptions {
*
* ``allowAutoPaste`` snapshots the setting at chord-start so a refine that
* lands after the user flips the toggle still uses the value the capture
* was created under. ``context`` is the value passed to ``startRecording``
* for this take, so overlapping dictations can't cross their targets.
* was created under.
*/
onFinalText?: (
text: string,
capture: CaptureResponse,
allowAutoPaste: boolean,
context?: unknown,
) => void;
}
@@ -88,14 +82,12 @@ export interface UseCaptureRecordingSessionResult {
isRecording: boolean;
isUploading: boolean;
isRefining: boolean;
startRecording: (context?: unknown) => void;
startRecording: () => void;
stopRecording: () => void;
toggleRecording: () => void;
dismissError: () => void;
uploadFile: (file: File, source: CaptureSource) => void;
refine: (captureId: string) => void;
prewarm: () => Promise<void>;
releaseWarm: () => void;
}
/**
@@ -131,13 +123,10 @@ export function useCaptureRecordingSession(
const onFinalTextRef = useRef(options.onFinalText);
onFinalTextRef.current = options.onFinalText;
// Per-capture recording context and its ``allow_auto_paste`` snapshot, keyed
// by capture id so a refine that resolves after another dictation started
// still delivers to the right target with the setting the capture was created
// under. Populated on capture-create and consumed once the final text lands.
const captureDeliveryRef = useRef<Map<string, { context: unknown; allowAutoPaste: boolean }>>(
new Map(),
);
// Snapshot of ``allow_auto_paste`` from the capture-create response —
// held so the refine onSuccess (which only sees the plain CaptureResponse)
// can still pass the original setting through to onFinalText.
const allowAutoPasteRef = useRef<boolean>(true);
const clearRestTimer = useCallback(() => {
if (restTimerRef.current !== null) {
@@ -203,34 +192,20 @@ export function useCaptureRecordingSession(
queryClient.invalidateQueries({ queryKey: ['captures'] });
broadcastUpdated(captureId);
if (pillStateRef.current === 'refining') scheduleHidePill();
const delivery = captureDeliveryRef.current.get(captureId);
captureDeliveryRef.current.delete(captureId);
const finalText = data.transcript_refined ?? data.transcript_raw;
if (finalText) {
onFinalTextRef.current?.(
finalText,
data,
delivery?.allowAutoPaste ?? true,
delivery?.context,
);
onFinalTextRef.current?.(finalText, data, allowAutoPasteRef.current);
}
},
onError: (err: Error, captureId) => {
captureDeliveryRef.current.delete(captureId);
onError: (err: Error) => {
showError(err.message || 'Refinement failed');
},
});
const uploadMutation = useMutation({
mutationFn: async ({
file,
source,
}: {
file: File;
source: CaptureSource;
context?: unknown;
}) => apiClient.createCapture(file, { source }),
onSuccess: (capture, { context }) => {
mutationFn: async ({ file, source }: { file: File; source: CaptureSource }) =>
apiClient.createCapture(file, { source }),
onSuccess: (capture) => {
queryClient.setQueryData<CaptureListResponse>(['captures'], (prev) => {
if (!prev) return prev;
if (prev.items.some((c) => c.id === capture.id)) return prev;
@@ -238,12 +213,9 @@ export function useCaptureRecordingSession(
});
queryClient.invalidateQueries({ queryKey: ['captures'] });
broadcastCreated(capture);
onCaptureCreatedRef.current?.(capture, context);
onCaptureCreatedRef.current?.(capture);
allowAutoPasteRef.current = capture.allow_auto_paste;
if (capture.auto_refine) {
captureDeliveryRef.current.set(capture.id, {
context,
allowAutoPaste: capture.allow_auto_paste,
});
setPillState('refining');
refineMutation.mutate(capture.id);
} else {
@@ -253,7 +225,6 @@ export function useCaptureRecordingSession(
capture.transcript_raw,
capture,
capture.allow_auto_paste,
context,
);
}
}
@@ -278,11 +249,8 @@ export function useCaptureRecordingSession(
startRecording: beginAudioRecording,
stopRecording,
error: recordError,
prewarm,
releaseWarm,
} = useAudioRecording({
keepWarm: options.keepMicWarm ?? false,
onRecordingComplete: (blob, recordedDuration, context) => {
onRecordingComplete: (blob, recordedDuration) => {
// Trigger-happy tap — MediaRecorder hasn't emitted a usable chunk yet
// so the blob is empty or unparseable. Surface it as a transient pill
// so the user sees their recording was recognised and canceled.
@@ -300,7 +268,7 @@ export function useCaptureRecordingSession(
const file = new File([blob], `dictation-${Date.now()}.${extension}`, {
type: blob.type,
});
uploadMutation.mutate({ file, source: 'dictation', context });
uploadMutation.mutate({ file, source: 'dictation' });
},
});
@@ -310,16 +278,13 @@ export function useCaptureRecordingSession(
}
}, [recordError, showError]);
const startRecording = useCallback(
(context?: unknown) => {
if (isRecording) return;
clearRestTimer();
setFrozenElapsedMs(0);
setPillState('recording');
beginAudioRecording(context);
},
[isRecording, beginAudioRecording, clearRestTimer],
);
const startRecording = useCallback(() => {
if (isRecording) return;
clearRestTimer();
setFrozenElapsedMs(0);
setPillState('recording');
beginAudioRecording();
}, [isRecording, beginAudioRecording, clearRestTimer]);
const toggleRecording = useCallback(() => {
if (isRecording) {
@@ -359,7 +324,5 @@ export function useCaptureRecordingSession(
dismissError,
uploadFile,
refine,
prewarm,
releaseWarm,
};
}
+1 -26
View File
@@ -1,6 +1,5 @@
import { invoke } from '@tauri-apps/api/core';
import { emit, listen } from '@tauri-apps/api/event';
import { useEffect, useRef } from 'react';
import { useEffect } from 'react';
import { useDictationReadiness } from '@/lib/hooks/useDictationReadiness';
import { useCaptureSettings } from '@/lib/hooks/useSettings';
import { usePlatform } from '@/platform/PlatformContext';
@@ -31,45 +30,21 @@ export function useChordSync() {
const { settings } = useCaptureSettings();
const { canRecord } = useDictationReadiness();
const enabled = settings?.hotkey_enabled;
const keepMicWarm = settings?.keep_mic_warm;
const pushKeys = settings?.chord_push_to_talk_keys;
const toggleKeys = settings?.chord_toggle_to_talk_keys;
// Latest warm state, so the dictate window's mount-time request can be
// answered even between the dep-driven emits below.
const shouldWarmRef = useRef(false);
// The floating dictate window holds the mic warm ahead of the first chord to
// avoid clipping, but it's a separate webview with no view of settings. Mirror
// the decision to it: warm only when dictation is armed AND the user enabled
// "keep microphone ready". Gating here is what stops the always-mounted pill
// from opening the mic — or prompting for access — when the user hasn't asked.
useEffect(() => {
if (!platform.metadata.isTauri) return;
const unlisten = listen('dictate:warm-request', () => {
emit('dictate:warm', shouldWarmRef.current).catch(() => {});
});
return () => {
unlisten.then((fn) => fn()).catch(() => {});
};
}, [platform.metadata.isTauri]);
useEffect(() => {
if (!platform.metadata.isTauri) return;
if (enabled === undefined || !pushKeys || !toggleKeys) return;
const shouldArm = enabled && canRecord;
const shouldWarm = shouldArm && (keepMicWarm ?? false);
shouldWarmRef.current = shouldWarm;
const command = shouldArm ? 'enable_hotkey' : 'disable_hotkey';
const args = shouldArm ? { pushToTalk: pushKeys, toggleToTalk: toggleKeys } : {};
invoke(command, args).catch((err) => {
console.warn(`[chord-sync] ${command} failed:`, err);
});
emit('dictate:warm', shouldWarm).catch(() => {});
}, [
platform.metadata.isTauri,
enabled,
keepMicWarm,
canRecord,
// Stringify so a referentially-new array with the same content
// doesn't fire a redundant invoke on every settings refetch.
-71
View File
@@ -1,71 +0,0 @@
import { describe, expect, test } from 'bun:test';
import { formatDuration, formatEngineName, formatFileSize } from './format';
describe('formatDuration', () => {
test('formats zero', () => {
expect(formatDuration(0)).toBe('0:00');
});
test('pads single-digit seconds', () => {
expect(formatDuration(65)).toBe('1:05');
});
test('handles the minute boundary', () => {
expect(formatDuration(59)).toBe('0:59');
expect(formatDuration(60)).toBe('1:00');
});
test('floors fractional seconds', () => {
expect(formatDuration(89.9)).toBe('1:29');
});
test('does not roll minutes into hours', () => {
expect(formatDuration(3661)).toBe('61:01');
});
});
describe('formatFileSize', () => {
test('special-cases zero', () => {
expect(formatFileSize(0)).toBe('0 Bytes');
});
test('formats bytes below 1 KB', () => {
expect(formatFileSize(512)).toBe('512 Bytes');
expect(formatFileSize(1023)).toBe('1023 Bytes');
});
test('formats KB, MB, and GB boundaries', () => {
expect(formatFileSize(1024)).toBe('1 KB');
expect(formatFileSize(1024 ** 2)).toBe('1 MB');
expect(formatFileSize(1024 ** 3)).toBe('1 GB');
});
test('rounds to two decimal places', () => {
expect(formatFileSize(1536)).toBe('1.5 KB');
expect(formatFileSize(2_684_354_560)).toBe('2.5 GB');
expect(formatFileSize(1_234_567)).toBe('1.18 MB');
});
});
describe('formatEngineName', () => {
test('maps known engines to display names', () => {
expect(formatEngineName('luxtts')).toBe('LuxTTS');
expect(formatEngineName('chatterbox')).toBe('Chatterbox');
expect(formatEngineName('chatterbox_turbo')).toBe('Chatterbox Turbo');
});
test('defaults to Qwen when engine is undefined', () => {
expect(formatEngineName()).toBe('Qwen');
expect(formatEngineName(undefined, '1.7B')).toBe('Qwen');
});
test('appends the model size for qwen only', () => {
expect(formatEngineName('qwen', '1.7B')).toBe('Qwen 1.7B');
expect(formatEngineName('qwen')).toBe('Qwen');
expect(formatEngineName('luxtts', '1.7B')).toBe('LuxTTS');
});
test('passes unknown engines through verbatim', () => {
expect(formatEngineName('kokoro')).toBe('kokoro');
});
});
+3 -1
View File
@@ -1,5 +1,5 @@
import { formatDistance } from 'date-fns';
import { ja, zhCN, zhTW, fr } from 'date-fns/locale';
import { es, fr, ja, zhCN, zhTW } from 'date-fns/locale';
import i18n from '@/i18n';
export function formatDuration(seconds: number): string {
@@ -10,6 +10,8 @@ export function formatDuration(seconds: number): string {
function getDateLocale() {
switch (i18n.language) {
case 'es':
return es;
case 'ja':
return ja;
case 'zh-CN':
-69
View File
@@ -1,69 +0,0 @@
import { describe, expect, test } from 'bun:test';
import { parseChangelog } from './parseChangelog';
const SAMPLE = `# Changelog
All notable changes to this project will be documented in this file.
## [0.5.0] - 2026-06-01
### Added
- Story track editor
- Cloud login
## [0.4.1]
### Fixed
- Trim clamping
## [0.4.0] - 2026-04-15
Initial public release.
[0.5.0]: https://example.com/compare/v0.4.1...v0.5.0
[0.4.1]: https://example.com/compare/v0.4.0...v0.4.1
`;
describe('parseChangelog', () => {
test('splits entries on version headings', () => {
const entries = parseChangelog(SAMPLE);
expect(entries.map((e) => e.version)).toEqual(['0.5.0', '0.4.1', '0.4.0']);
});
test('extracts the date when present and null otherwise', () => {
const entries = parseChangelog(SAMPLE);
expect(entries[0].date).toBe('2026-06-01');
expect(entries[1].date).toBeNull();
});
test('keeps the markdown body between headings', () => {
const entries = parseChangelog(SAMPLE);
expect(entries[0].body).toBe('### Added\n\n- Story track editor\n- Cloud login');
expect(entries[2].body).toBe('Initial public release.');
});
test('strips trailing link reference definitions from the last body', () => {
const entries = parseChangelog(SAMPLE);
expect(entries[2].body).toBe('Initial public release.');
expect(entries[2].body).not.toContain('example.com');
});
test('returns an empty array when no headings match', () => {
expect(parseChangelog('')).toEqual([]);
expect(parseChangelog('# Changelog\n\nNothing yet.')).toEqual([]);
});
test('handles a heading with an empty body', () => {
const entries = parseChangelog('## [1.0.0] - 2026-01-01\n');
expect(entries).toEqual([{ version: '1.0.0', date: '2026-01-01', body: '' }]);
});
test('accepts non-semver headings like Unreleased', () => {
const entries = parseChangelog('## [Unreleased]\n\n### Added\n\n- WIP\n');
expect(entries[0].version).toBe('Unreleased');
expect(entries[0].date).toBeNull();
expect(entries[0].body).toBe('### Added\n\n- WIP');
});
});
-106
View File
@@ -1,106 +0,0 @@
import { describe, expect, test } from 'bun:test';
import { MIN_CLIP_DURATION_MS, computeTrimValues } from './trim';
// A 10-second clip with no existing trims unless stated otherwise.
const DURATION = 10_000;
describe('computeTrimValues', () => {
describe('start handle', () => {
test('dragging right trims from the start', () => {
expect(computeTrimValues('start', 500, 0, 0, DURATION)).toEqual({
trim_start_ms: 500,
trim_end_ms: 0,
});
});
test('dragging left restores previously trimmed audio', () => {
expect(computeTrimValues('start', -300, 1000, 0, DURATION)).toEqual({
trim_start_ms: 700,
trim_end_ms: 0,
});
});
test('clamps at zero when restoring past the clip start', () => {
expect(computeTrimValues('start', -5000, 1000, 0, DURATION)).toEqual({
trim_start_ms: 0,
trim_end_ms: 0,
});
});
test('never trims below the minimum clip duration', () => {
const result = computeTrimValues('start', 99_999, 0, 2000, DURATION);
// Clamp lands exactly on the minimum, which the guard rejects.
expect(result).toBeNull();
});
test('rounds fractional millisecond deltas', () => {
expect(computeTrimValues('start', 100.6, 0, 0, DURATION)).toEqual({
trim_start_ms: 101,
trim_end_ms: 0,
});
});
test('preserves the untouched end trim', () => {
expect(computeTrimValues('start', 250, 0, 400, DURATION)).toEqual({
trim_start_ms: 250,
trim_end_ms: 400,
});
});
});
describe('end handle', () => {
test('dragging left trims from the end', () => {
expect(computeTrimValues('end', -500, 0, 0, DURATION)).toEqual({
trim_start_ms: 0,
trim_end_ms: 500,
});
});
test('dragging right restores previously trimmed audio', () => {
expect(computeTrimValues('end', 300, 0, 1000, DURATION)).toEqual({
trim_start_ms: 0,
trim_end_ms: 700,
});
});
test('clamps at zero when restoring past the clip end', () => {
expect(computeTrimValues('end', 5000, 0, 1000, DURATION)).toEqual({
trim_start_ms: 0,
trim_end_ms: 0,
});
});
test('never trims below the minimum clip duration', () => {
expect(computeTrimValues('end', -99_999, 3000, 0, DURATION)).toBeNull();
});
});
describe('minimum duration guard', () => {
test('rejects drags that leave less than the minimum audible clip', () => {
// 9.5s already trimmed; taking 450ms more leaves only 50ms.
expect(computeTrimValues('start', 450, 5000, 4500, DURATION)).toBeNull();
});
test('allows a drag that leaves just over the minimum', () => {
expect(computeTrimValues('start', 399, 5000, 4500, DURATION)).toEqual({
trim_start_ms: 5399,
trim_end_ms: 4500,
});
});
test('boundary: exactly the minimum remaining is rejected', () => {
// trim_start + trim_end === duration - MIN_CLIP_DURATION_MS
expect(
computeTrimValues('start', 400, 5000, 4500, DURATION)?.trim_start_ms ?? null,
).toBeNull();
expect(MIN_CLIP_DURATION_MS).toBe(100);
});
});
test('zero delta is a no-op that returns the initial trims', () => {
expect(computeTrimValues('start', 0, 1200, 800, DURATION)).toEqual({
trim_start_ms: 1200,
trim_end_ms: 800,
});
});
});
-57
View File
@@ -1,57 +0,0 @@
import type { StoryItemTrim } from '@/lib/api/types';
/** Clips are never allowed to shrink below this effective duration. */
export const MIN_CLIP_DURATION_MS = 100;
/**
* Computes new trim values for a clip while a trim handle is being dragged.
*
* `deltaMs` is the signed drag distance converted to milliseconds. Dragging
* the start handle right increases `trim_start_ms` (trims more from the
* start); dragging it left restores. The end handle mirrors this for
* `trim_end_ms`. Both values are clamped so the clip keeps at least
* MIN_CLIP_DURATION_MS of audible content.
*
* Returns null when the drag would leave less than the minimum duration.
*/
export function computeTrimValues(
side: 'start' | 'end',
deltaMs: number,
initialTrimStart: number,
initialTrimEnd: number,
originalDurationMs: number,
): StoryItemTrim | null {
let newTrimStart = initialTrimStart;
let newTrimEnd = initialTrimEnd;
if (side === 'start') {
newTrimStart = Math.round(
Math.max(
0,
Math.min(
initialTrimStart + deltaMs,
originalDurationMs - initialTrimEnd - MIN_CLIP_DURATION_MS,
),
),
);
} else {
newTrimEnd = Math.round(
Math.max(
0,
Math.min(
initialTrimEnd - deltaMs,
originalDurationMs - initialTrimStart - MIN_CLIP_DURATION_MS,
),
),
);
}
if (newTrimStart + newTrimEnd >= originalDurationMs - MIN_CLIP_DURATION_MS) {
return null;
}
return {
trim_start_ms: newTrimStart,
trim_end_ms: newTrimEnd,
};
}
+1 -1
View File
@@ -25,7 +25,7 @@
"paths": {
"@/*": ["./src/*"]
},
"types": ["vite/client", "bun"]
"types": ["vite/client"]
},
"include": ["src"],
"references": [{ "path": "./tsconfig.node.json" }]
+19 -13
View File
@@ -38,6 +38,13 @@ logging.basicConfig(
logger = logging.getLogger(__name__)
# An empty HSA_OVERRIDE_GFX_VERSION poisons the ROCm HSA runtime. It is
# treated as "force-empty" and no GPU is detected, even natively supported
# ones (e.g. gfx1201 / RX 9070 on ROCm 7.2). docker-compose can't
# conditionally omit an env var, so we clean it up here before torch loads.
if not os.environ.get("HSA_OVERRIDE_GFX_VERSION"):
os.environ.pop("HSA_OVERRIDE_GFX_VERSION", None)
# AMD GPU environment variables must be set before torch import
# Only set HSA_OVERRIDE_GFX_VERSION for older GPUs that need it.
# RDNA 3+ (gfx1100+) and RDNA 4 (gfx1200+) are natively supported by ROCm
@@ -95,19 +102,18 @@ if not os.environ.get("HSA_OVERRIDE_GFX_VERSION"):
if not os.environ.get("MIOPEN_LOG_LEVEL"):
os.environ["MIOPEN_LOG_LEVEL"] = "4"
from urllib.parse import quote
import torch
from fastapi import FastAPI
from fastapi.middleware.cors import CORSMiddleware
from urllib.parse import quote
from . import __version__, config, database
from .services import tts, transcribe, llm
from .database import get_db
from .routes import register_routers
from .services import llm, transcribe, tts
from .services.task_queue import create_background_task, init_queue
from .utils.platform_detect import get_backend_type
from .utils.progress import get_progress_manager
from .services.task_queue import create_background_task, init_queue
from .routes import register_routers
def safe_content_disposition(disposition_type: str, filename: str) -> str:
@@ -123,8 +129,8 @@ def safe_content_disposition(disposition_type: str, filename: str) -> str:
def create_app() -> FastAPI:
"""Create and configure the FastAPI application."""
from .mcp_server.context import ClientIdMiddleware
from .mcp_server.server import build_mcp_server, compose_lifespan
from .mcp_server.context import ClientIdMiddleware
# Build the MCP app up-front so we can wire its lifespan into FastAPI's —
# FastMCP's Streamable HTTP transport only works if its session manager
@@ -203,8 +209,8 @@ def _mount_frontend(application: FastAPI) -> None:
if not frontend_dir.is_dir():
return
from fastapi.responses import FileResponse
from fastapi.staticfiles import StaticFiles
from fastapi.responses import FileResponse
# Mount hashed assets (JS, CSS, images) that Vite places under /assets
assets_dir = frontend_dir / "assets"
@@ -244,9 +250,9 @@ def _get_gpu_status() -> str:
if not compatible:
label += " [UNSUPPORTED - see logs]"
return label
if hasattr(torch.backends, "mps") and torch.backends.mps.is_available():
elif hasattr(torch.backends, "mps") and torch.backends.mps.is_available():
return "MPS (Apple Silicon)"
if backend_type == "mlx":
elif backend_type == "mlx":
return "Metal (Apple Silicon via MLX)"
# Intel XPU (Arc / Data Center) via IPEX
@@ -303,7 +309,7 @@ async def _run_startup(application: FastAPI) -> None:
if result.rowcount > 0:
logger.info("Marked %d stale generation(s) as failed", result.rowcount)
from .database import Generation as DBGeneration, VoiceProfile as DBVoiceProfile
from .database import VoiceProfile as DBVoiceProfile, Generation as DBGeneration
profile_count = db.query(DBVoiceProfile).count()
generation_count = db.query(DBGeneration).count()
@@ -354,15 +360,15 @@ async def _run_shutdown() -> None:
"""Unload models on lifespan exit."""
logger.info("Voicebox server shutting down...")
try:
await tts.unload_tts_model()
tts.unload_tts_model()
except Exception:
logger.exception("Failed to unload TTS model")
try:
await transcribe.unload_whisper_model()
transcribe.unload_whisper_model()
except Exception:
logger.exception("Failed to unload Whisper model")
try:
await llm.unload_llm_model()
llm.unload_llm_model()
except Exception:
logger.exception("Failed to unload LLM model")
+30 -52
View File
@@ -10,14 +10,13 @@ and a model config registry that eliminates per-engine dispatch maps.
# import time, which wraps transformers' tokenizer load against the
# unconditional HuggingFace metadata call that otherwise raises on
# HF_HUB_OFFLINE=1 and on network failures.
from ..utils import hf_offline_patch # noqa: F401
import threading
from dataclasses import dataclass, field
from typing import Protocol
import numpy as np
from typing import Protocol, Optional, Tuple, List
from typing_extensions import runtime_checkable
from ..utils import hf_offline_patch
import numpy as np
DEFAULT_LLM_MAX_TOKENS = 512
DEFAULT_LLM_TEMPERATURE = 0.7
@@ -78,7 +77,7 @@ class TTSBackend(Protocol):
audio_path: str,
reference_text: str,
use_cache: bool = True,
) -> tuple[dict, bool]:
) -> Tuple[dict, bool]:
"""
Create voice prompt from reference audio.
@@ -89,9 +88,9 @@ class TTSBackend(Protocol):
async def combine_voice_prompts(
self,
audio_paths: list[str],
reference_texts: list[str],
) -> tuple[np.ndarray, str]:
audio_paths: List[str],
reference_texts: List[str],
) -> Tuple[np.ndarray, str]:
"""
Combine multiple voice prompts.
@@ -105,9 +104,9 @@ class TTSBackend(Protocol):
text: str,
voice_prompt: dict,
language: str = "en",
seed: int | None = None,
instruct: str | None = None,
) -> tuple[np.ndarray, int]:
seed: Optional[int] = None,
instruct: Optional[str] = None,
) -> Tuple[np.ndarray, int]:
"""
Generate audio from text.
@@ -145,8 +144,8 @@ class STTBackend(Protocol):
async def transcribe(
self,
audio_path: str,
language: str | None = None,
model_size: str | None = None,
language: Optional[str] = None,
model_size: Optional[str] = None,
) -> str:
"""
Transcribe audio to text.
@@ -176,11 +175,11 @@ class LLMBackend(Protocol):
async def generate(
self,
prompt: str,
system: str | None = None,
system: Optional[str] = None,
max_tokens: int = DEFAULT_LLM_MAX_TOKENS,
temperature: float = DEFAULT_LLM_TEMPERATURE,
model_size: str | None = None,
examples: list[tuple[str, str]] | None = None,
model_size: Optional[str] = None,
examples: Optional[list[tuple[str, str]]] = None,
) -> str:
"""Run a single-turn chat completion and return the assistant reply.
@@ -200,11 +199,10 @@ class LLMBackend(Protocol):
# Global backend instances
_tts_backend: TTSBackend | None = None
_tts_backend: Optional[TTSBackend] = None
_tts_backends: dict[str, TTSBackend] = {}
_tts_backends_lock = threading.Lock()
_stt_backend: STTBackend | None = None
_stt_backend_lock = threading.Lock()
_stt_backend: Optional[STTBackend] = None
_llm_backends: dict[str, LLMBackend] = {}
_llm_backends_lock = threading.Lock()
@@ -497,7 +495,7 @@ def get_stt_model_configs() -> list[ModelConfig]:
# Lookup helpers — these replace the if/elif chains in main.py
def get_model_config(model_name: str) -> ModelConfig | None:
def get_model_config(model_name: str) -> Optional[ModelConfig]:
"""Look up a model config by model_name."""
for cfg in get_all_model_configs():
if cfg.model_name == model_name:
@@ -564,29 +562,15 @@ async def ensure_model_cached_or_raise(engine: str, model_size: str = "default")
)
async def unload_backend(backend) -> None:
"""Free a backend's model, serialized onto the MLX worker when it has one.
MLX backends expose an async ``unload`` that runs the free on the dedicated
MLX thread so it can't collide with an in-flight load/generate. Other
backends only carry the synchronous ``unload_model``.
"""
unload = getattr(backend, "unload", None)
if unload is not None:
await unload()
else:
backend.unload_model()
async def unload_model_by_config(config: ModelConfig) -> bool:
def unload_model_by_config(config: ModelConfig) -> bool:
"""Unload a model given its config. Returns True if it was loaded, False otherwise."""
from ..services import llm as llm_service, transcribe, tts
from . import get_tts_backend_for_engine
from ..services import tts, transcribe, llm as llm_service
if config.engine == "whisper":
whisper_model = transcribe.get_whisper_model()
if whisper_model.is_loaded() and whisper_model.model_size == config.model_size:
await unload_backend(whisper_model)
transcribe.unload_whisper_model()
return True
return False
@@ -594,7 +578,7 @@ async def unload_model_by_config(config: ModelConfig) -> bool:
backend = llm_service.get_llm_model()
loaded_size = getattr(backend, "_current_model_size", None) or getattr(backend, "model_size", None)
if backend.is_loaded() and loaded_size == config.model_size:
await unload_backend(backend)
backend.unload_model()
return True
return False
@@ -602,7 +586,7 @@ async def unload_model_by_config(config: ModelConfig) -> bool:
tts_model = tts.get_tts_model()
loaded_size = getattr(tts_model, "_current_model_size", None) or getattr(tts_model, "model_size", None)
if tts_model.is_loaded() and loaded_size == config.model_size:
await unload_backend(tts_model)
tts.unload_tts_model()
return True
return False
@@ -610,22 +594,22 @@ async def unload_model_by_config(config: ModelConfig) -> bool:
backend = get_tts_backend_for_engine(config.engine)
loaded_size = getattr(backend, "_current_model_size", None) or getattr(backend, "model_size", None)
if backend.is_loaded() and loaded_size == config.model_size:
await unload_backend(backend)
backend.unload_model()
return True
return False
# All other TTS engines
backend = get_tts_backend_for_engine(config.engine)
if backend.is_loaded():
await unload_backend(backend)
backend.unload_model()
return True
return False
def check_model_loaded(config: ModelConfig) -> bool:
"""Check if a model is currently loaded."""
from ..services import llm as llm_service, transcribe, tts
from . import get_tts_backend_for_engine
from ..services import tts, transcribe, llm as llm_service
try:
if config.engine == "whisper":
@@ -655,8 +639,8 @@ def check_model_loaded(config: ModelConfig) -> bool:
def get_model_load_func(config: ModelConfig):
"""Return a callable that loads/downloads the model."""
from ..services import llm as llm_service, transcribe, tts
from . import get_tts_backend_for_engine
from ..services import tts, transcribe, llm as llm_service
if config.engine == "whisper":
return lambda: transcribe.get_whisper_model().load_model(config.model_size)
@@ -755,13 +739,7 @@ def get_stt_backend() -> STTBackend:
"""
global _stt_backend
if _stt_backend is not None:
return _stt_backend
with _stt_backend_lock:
if _stt_backend is not None:
return _stt_backend
if _stt_backend is None:
backend_type = get_backend_type()
if backend_type == "mlx":
@@ -773,7 +751,7 @@ def get_stt_backend() -> STTBackend:
_stt_backend = PyTorchSTTBackend()
return _stt_backend
return _stt_backend
def get_llm_backend() -> LLMBackend:
+9 -8
View File
@@ -9,12 +9,13 @@ import logging
import platform
from contextlib import contextmanager
from pathlib import Path
from typing import Callable, List, Optional, Tuple
import numpy as np
from ..utils.audio import load_audio, normalize_audio
from ..utils.hf_progress import HFProgressTracker, create_hf_progress_callback
from ..utils.audio import normalize_audio, load_audio
from ..utils.progress import get_progress_manager
from ..utils.hf_progress import HFProgressTracker, create_hf_progress_callback
from ..utils.tasks import get_task_manager
logger = logging.getLogger(__name__)
@@ -24,7 +25,7 @@ def is_model_cached(
hf_repo: str,
*,
weight_extensions: tuple[str, ...] = (".safetensors", ".bin"),
required_files: list[str] | None = None,
required_files: Optional[list[str]] = None,
) -> bool:
"""
Check if a HuggingFace model is fully cached locally.
@@ -200,11 +201,11 @@ def manual_seed(seed: int, device: str) -> None:
async def combine_voice_prompts(
audio_paths: list[str],
reference_texts: list[str],
audio_paths: List[str],
reference_texts: List[str],
*,
sample_rate: int | None = None,
) -> tuple[np.ndarray, str]:
sample_rate: Optional[int] = None,
) -> Tuple[np.ndarray, str]:
"""
Combine multiple reference audio samples into one.
@@ -234,7 +235,7 @@ async def combine_voice_prompts(
def model_load_progress(
model_name: str,
is_cached: bool,
filter_non_downloads: bool | None = None,
filter_non_downloads: Optional[bool] = None,
):
"""
Context manager for model loading with HF download progress tracking.
+12 -11
View File
@@ -10,16 +10,17 @@ import asyncio
import logging
import threading
from pathlib import Path
from typing import ClassVar
from typing import ClassVar, List, Optional, Tuple
import numpy as np
from . import TTSBackend
from .base import (
combine_voice_prompts as _combine_voice_prompts,
empty_device_cache,
get_torch_device,
is_model_cached,
get_torch_device,
empty_device_cache,
manual_seed,
combine_voice_prompts as _combine_voice_prompts,
model_load_progress,
patch_chatterbox_f32,
)
@@ -126,7 +127,7 @@ class ChatterboxTTSBackend:
audio_path: str,
reference_text: str,
use_cache: bool = True,
) -> tuple[dict, bool]:
) -> Tuple[dict, bool]:
"""
Create voice prompt from reference audio.
@@ -142,9 +143,9 @@ class ChatterboxTTSBackend:
async def combine_voice_prompts(
self,
audio_paths: list[str],
reference_texts: list[str],
) -> tuple[np.ndarray, str]:
audio_paths: List[str],
reference_texts: List[str],
) -> Tuple[np.ndarray, str]:
return await _combine_voice_prompts(audio_paths, reference_texts)
# Per-language generation defaults. Lower temp + higher cfg = clearer speech.
@@ -168,9 +169,9 @@ class ChatterboxTTSBackend:
text: str,
voice_prompt: dict,
language: str = "en",
seed: int | None = None,
instruct: str | None = None,
) -> tuple[np.ndarray, int]:
seed: Optional[int] = None,
instruct: Optional[str] = None,
) -> Tuple[np.ndarray, int]:
"""
Generate audio using Chatterbox Multilingual TTS.
+13 -12
View File
@@ -10,16 +10,17 @@ import asyncio
import logging
import threading
from pathlib import Path
from typing import ClassVar
from typing import ClassVar, List, Optional, Tuple
import numpy as np
from . import TTSBackend
from .base import (
combine_voice_prompts as _combine_voice_prompts,
empty_device_cache,
get_torch_device,
is_model_cached,
get_torch_device,
empty_device_cache,
manual_seed,
combine_voice_prompts as _combine_voice_prompts,
model_load_progress,
patch_chatterbox_f32,
)
@@ -80,8 +81,8 @@ class ChatterboxTurboTTSBackend:
logger.info(f"Loading Chatterbox Turbo TTS on {device}...")
import torch
from chatterbox.tts_turbo import ChatterboxTurboTTS
from huggingface_hub import snapshot_download
from chatterbox.tts_turbo import ChatterboxTurboTTS
local_path = snapshot_download(
repo_id=CHATTERBOX_TURBO_HF_REPO,
@@ -125,7 +126,7 @@ class ChatterboxTurboTTSBackend:
audio_path: str,
reference_text: str,
use_cache: bool = True,
) -> tuple[dict, bool]:
) -> Tuple[dict, bool]:
"""
Create voice prompt from reference audio.
@@ -140,9 +141,9 @@ class ChatterboxTurboTTSBackend:
async def combine_voice_prompts(
self,
audio_paths: list[str],
reference_texts: list[str],
) -> tuple[np.ndarray, str]:
audio_paths: List[str],
reference_texts: List[str],
) -> Tuple[np.ndarray, str]:
return await _combine_voice_prompts(audio_paths, reference_texts)
async def generate(
@@ -150,9 +151,9 @@ class ChatterboxTurboTTSBackend:
text: str,
voice_prompt: dict,
language: str = "en",
seed: int | None = None,
instruct: str | None = None,
) -> tuple[np.ndarray, int]:
seed: Optional[int] = None,
instruct: Optional[str] = None,
) -> Tuple[np.ndarray, int]:
"""
Generate audio using Chatterbox Turbo TTS.
+18 -15
View File
@@ -16,19 +16,20 @@ causal LM generates speech via flow-matching diffusion.
import asyncio
import logging
import threading
from typing import ClassVar
from typing import ClassVar, List, Optional, Tuple
import numpy as np
from ..utils.cache import cache_voice_prompt, get_cache_key, get_cached_voice_prompt
from . import TTSBackend
from .base import (
combine_voice_prompts as _combine_voice_prompts,
empty_device_cache,
get_torch_device,
is_model_cached,
get_torch_device,
empty_device_cache,
manual_seed,
combine_voice_prompts as _combine_voice_prompts,
model_load_progress,
)
from ..utils.cache import get_cache_key, get_cached_voice_prompt, cache_voice_prompt
logger = logging.getLogger(__name__)
@@ -181,7 +182,7 @@ class HumeTadaBackend:
# getattr(config, "tokenizer_name", "meta-llama/Llama-3.2-1B")
# which hits the gated repo. Pre-load the config from HF,
# inject the local tokenizer path, then pass it in.
from tada.modules.tada import TadaConfig, TadaForCausalLM
from tada.modules.tada import TadaForCausalLM, TadaConfig
logger.info(f"Loading TADA {model_size} model...")
config = TadaConfig.from_pretrained(repo)
@@ -213,7 +214,7 @@ class HumeTadaBackend:
audio_path: str,
reference_text: str,
use_cache: bool = True,
) -> tuple[dict, bool]:
) -> Tuple[dict, bool]:
"""
Create voice prompt from reference audio using TADA's encoder.
@@ -233,8 +234,8 @@ class HumeTadaBackend:
return cached, True
def _encode_sync():
import soundfile as sf
import torch
import soundfile as sf
device = self._device
@@ -261,7 +262,9 @@ class HumeTadaBackend:
val = getattr(prompt, field_name)
if isinstance(val, torch.Tensor):
prompt_dict[field_name] = val.detach().cpu()
elif isinstance(val, (list, int, float)):
elif isinstance(val, list):
prompt_dict[field_name] = val
elif isinstance(val, (int, float)):
prompt_dict[field_name] = val
else:
prompt_dict[field_name] = val
@@ -276,9 +279,9 @@ class HumeTadaBackend:
async def combine_voice_prompts(
self,
audio_paths: list[str],
reference_texts: list[str],
) -> tuple[np.ndarray, str]:
audio_paths: List[str],
reference_texts: List[str],
) -> Tuple[np.ndarray, str]:
return await _combine_voice_prompts(audio_paths, reference_texts, sample_rate=24000)
async def generate(
@@ -286,9 +289,9 @@ class HumeTadaBackend:
text: str,
voice_prompt: dict,
language: str = "en",
seed: int | None = None,
instruct: str | None = None,
) -> tuple[np.ndarray, int]:
seed: Optional[int] = None,
instruct: Optional[str] = None,
) -> Tuple[np.ndarray, int]:
"""
Generate audio from text using HumeAI TADA.
+14 -10
View File
@@ -17,12 +17,15 @@ Languages supported (via misaki G2P):
import asyncio
import logging
import os
from typing import Optional
import numpy as np
from . import TTSBackend
from .base import (
combine_voice_prompts as _combine_voice_prompts,
get_torch_device,
combine_voice_prompts as _combine_voice_prompts,
model_load_progress,
)
@@ -93,11 +96,16 @@ KOKORO_VOICES = [
("pf_dora", "Dora", "female", "pt"),
("pm_alex", "Alex", "male", "pt"),
("pm_santa", "Santa", "male", "pt"),
# Chinese
# Chinese female
("zf_xiaobei", "Xiaobei", "female", "zh"),
("zf_xiaoni", "Xiaoni", "female", "zh"),
("zf_xiaoxiao", "Xiaoxiao", "female", "zh"),
("zf_xiaoyi", "Xiaoyi", "female", "zh"),
# Chinese male
("zm_yunjian", "Yunjian", "male", "zh"),
("zm_yunxi", "Yunxi", "male", "zh"),
("zm_yunxia", "Yunxia", "male", "zh"),
("zm_yunyang", "Yunyang", "male", "zh"),
]
# Map our ISO language codes to Kokoro lang_code characters
@@ -119,9 +127,8 @@ class KokoroTTSBackend:
def __init__(self):
self._model = None
self._pipelines: dict = {} # lang_code -> KPipeline
self._device: str | None = None
self._device: Optional[str] = None
self.model_size = "default"
self._model_load_lock = asyncio.Lock()
def _get_device(self) -> str:
"""Select device. Kokoro supports CUDA and CPU. MPS needs fallback env var."""
@@ -155,10 +162,7 @@ class KokoroTTSBackend:
"""Load the Kokoro model."""
if self._model is not None:
return
async with self._model_load_lock:
if self._model is not None:
return
await asyncio.to_thread(self._load_model_sync)
await asyncio.to_thread(self._load_model_sync)
def _load_model_sync(self):
"""Synchronous model loading."""
@@ -240,8 +244,8 @@ class KokoroTTSBackend:
text: str,
voice_prompt: dict,
language: str = "en",
seed: int | None = None,
instruct: str | None = None,
seed: Optional[int] = None,
instruct: Optional[str] = None,
) -> tuple[np.ndarray, int]:
"""
Generate audio from text using Kokoro.
+12 -13
View File
@@ -7,18 +7,20 @@ Wraps the LuxTTS (ZipVoice) model for zero-shot voice cloning.
import asyncio
import logging
from typing import Optional, Tuple
import numpy as np
from ..utils.cache import cache_voice_prompt, get_cache_key, get_cached_voice_prompt
from . import TTSBackend
from .base import (
combine_voice_prompts as _combine_voice_prompts,
empty_device_cache,
get_torch_device,
is_model_cached,
get_torch_device,
empty_device_cache,
manual_seed,
combine_voice_prompts as _combine_voice_prompts,
model_load_progress,
)
from ..utils.cache import get_cache_key, get_cached_voice_prompt, cache_voice_prompt
logger = logging.getLogger(__name__)
@@ -33,7 +35,6 @@ class LuxTTSBackend:
self.model = None
self.model_size = "default" # LuxTTS has only one model size
self._device = None
self._model_load_lock = asyncio.Lock()
def _get_device(self) -> str:
return get_torch_device(allow_mps=True, allow_xpu=True)
@@ -60,10 +61,8 @@ class LuxTTSBackend:
"""Load the LuxTTS model."""
if self.model is not None:
return
async with self._model_load_lock:
if self.model is not None:
return
await asyncio.to_thread(self._load_model_sync)
await asyncio.to_thread(self._load_model_sync)
def _load_model_sync(self):
model_name = "luxtts"
@@ -106,7 +105,7 @@ class LuxTTSBackend:
audio_path: str,
reference_text: str,
use_cache: bool = True,
) -> tuple[dict, bool]:
) -> Tuple[dict, bool]:
"""
Create voice prompt from reference audio.
@@ -146,9 +145,9 @@ class LuxTTSBackend:
text: str,
voice_prompt: dict,
language: str = "en",
seed: int | None = None,
instruct: str | None = None,
) -> tuple[np.ndarray, int]:
seed: Optional[int] = None,
instruct: Optional[str] = None,
) -> Tuple[np.ndarray, int]:
"""
Generate audio from text using LuxTTS.
+78 -91
View File
@@ -2,24 +2,24 @@
MLX backend implementation for TTS and STT using mlx-audio.
"""
from typing import Optional, List, Tuple
import asyncio
import logging
from pathlib import Path
import numpy as np
from pathlib import Path
logger = logging.getLogger(__name__)
# PATCH: Import and apply offline patch BEFORE any huggingface_hub usage
# This prevents mlx_audio from making network requests when models are cached
from ..utils.hf_offline_patch import ensure_original_qwen_config_cached, patch_huggingface_hub_offline
from ..utils.hf_offline_patch import patch_huggingface_hub_offline, ensure_original_qwen_config_cached
patch_huggingface_hub_offline()
ensure_original_qwen_config_cached()
from ..services.mlx_thread import clear_mlx_cache, run_on_mlx_thread
from ..utils.cache import cache_voice_prompt, get_cache_key, get_cached_voice_prompt
from . import LANGUAGE_CODE_TO_NAME, WHISPER_HF_REPOS
from .base import combine_voice_prompts as _combine_voice_prompts, is_model_cached, model_load_progress
from . import TTSBackend, STTBackend, LANGUAGE_CODE_TO_NAME, WHISPER_HF_REPOS
from .base import is_model_cached, combine_voice_prompts as _combine_voice_prompts, model_load_progress
from ..utils.cache import get_cache_key, get_cached_voice_prompt, cache_voice_prompt
class MLXTTSBackend:
@@ -63,38 +63,30 @@ class MLXTTSBackend:
weight_extensions=(".safetensors", ".bin", ".npz"),
)
def _ensure_loaded_sync(self, model_size: str | None):
"""Load the model if the requested size isn't already resident.
Runs on the MLX worker thread so it stays serialized with generation.
"""
if model_size is None:
model_size = self.model_size
if self.model is not None and self._current_model_size == model_size:
return
if self.model is not None and self._current_model_size != model_size:
self.unload_model()
self._load_model_sync(model_size)
async def load_model_async(self, model_size: str | None = None):
async def load_model_async(self, model_size: Optional[str] = None):
"""
Lazy load the MLX TTS model.
Args:
model_size: Model size to load (1.7B or 0.6B)
"""
await run_on_mlx_thread(self._ensure_loaded_sync, model_size)
if model_size is None:
model_size = self.model_size
# If already loaded with correct size, return
if self.model is not None and self._current_model_size == model_size:
return
# Unload existing model if different size requested
if self.model is not None and self._current_model_size != model_size:
self.unload_model()
# Run blocking load in thread pool
await asyncio.to_thread(self._load_model_sync, model_size)
# Alias for compatibility
load_model = load_model_async
async def unload(self):
"""Free the model, serialized onto the MLX worker thread."""
await run_on_mlx_thread(self.unload_model)
def _load_model_sync(self, model_size: str):
"""Synchronous model loading."""
model_path = self._get_model_path(model_size)
@@ -118,7 +110,6 @@ class MLXTTSBackend:
del self.model
self.model = None
self._current_model_size = None
clear_mlx_cache()
logger.info("MLX TTS model unloaded")
async def create_voice_prompt(
@@ -126,7 +117,7 @@ class MLXTTSBackend:
audio_path: str,
reference_text: str,
use_cache: bool = True,
) -> tuple[dict, bool]:
) -> Tuple[dict, bool]:
"""
Create voice prompt from reference audio.
@@ -154,8 +145,9 @@ class MLXTTSBackend:
cached_audio_path = cached_prompt.get("ref_audio") or cached_prompt.get("ref_audio_path")
if cached_audio_path and Path(cached_audio_path).exists():
return cached_prompt, True
# Cached file no longer exists, invalidate cache
logger.warning("Cached audio file not found: %s, regenerating prompt", cached_audio_path)
else:
# Cached file no longer exists, invalidate cache
logger.warning("Cached audio file not found: %s, regenerating prompt", cached_audio_path)
# MLX voice prompt format - store audio path and text
# The model will process this during generation
@@ -179,9 +171,9 @@ class MLXTTSBackend:
text: str,
voice_prompt: dict,
language: str = "en",
seed: int | None = None,
instruct: str | None = None,
) -> tuple[np.ndarray, int]:
seed: Optional[int] = None,
instruct: Optional[str] = None,
) -> Tuple[np.ndarray, int]:
"""
Generate audio from text using voice prompt.
@@ -195,6 +187,8 @@ class MLXTTSBackend:
Returns:
Tuple of (audio_array, sample_rate)
"""
await self.load_model_async(None)
logger.info("Generating audio for text: %s", text)
def _generate_sync():
@@ -227,39 +221,45 @@ class MLXTTSBackend:
# mlx_audio lookups hanging when the network drops mid-inference,
# issue #462) regressed online users because libraries make
# legitimate metadata calls during generation.
# A cloning failure surfaces as a failed generation; substituting
# the model's default voice would silently break the clone the
# user asked for.
if ref_audio:
import inspect
try:
if ref_audio:
# Check if generate accepts ref_audio parameter
import inspect
sig = inspect.signature(self.model.generate)
if "ref_audio" not in sig.parameters:
raise RuntimeError(
"Loaded MLX model does not support voice cloning "
"(generate() has no ref_audio parameter)"
)
for result in self.model.generate(text, ref_audio=ref_audio, ref_text=ref_text, lang_code=lang):
audio_chunks.append(np.array(result.audio))
sample_rate = result.sample_rate
else:
sig = inspect.signature(self.model.generate)
if "ref_audio" in sig.parameters:
# Generate with voice cloning
for result in self.model.generate(text, ref_audio=ref_audio, ref_text=ref_text, lang_code=lang):
audio_chunks.append(np.array(result.audio))
sample_rate = result.sample_rate
else:
# Fallback: generate without voice cloning
for result in self.model.generate(text, lang_code=lang):
audio_chunks.append(np.array(result.audio))
sample_rate = result.sample_rate
else:
# No voice prompt, generate normally
for result in self.model.generate(text, lang_code=lang):
audio_chunks.append(np.array(result.audio))
sample_rate = result.sample_rate
except Exception as e:
# If voice cloning fails, try without it
logger.warning("Voice cloning failed, generating without voice prompt: %s", e)
for result in self.model.generate(text, lang_code=lang):
audio_chunks.append(np.array(result.audio))
sample_rate = result.sample_rate
if not audio_chunks:
raise RuntimeError("Model produced no audio")
# Concatenate all chunks
if audio_chunks:
audio = np.concatenate([np.asarray(chunk, dtype=np.float32) for chunk in audio_chunks])
else:
# Fallback: empty audio
audio = np.array([], dtype=np.float32)
audio = np.concatenate([np.asarray(chunk, dtype=np.float32) for chunk in audio_chunks])
return audio, sample_rate
# Load-if-needed and inference run as one job on the MLX worker so a
# concurrent unload or different-size load can't land between them.
def _load_and_generate():
self._ensure_loaded_sync(None)
return _generate_sync()
audio, sample_rate = await run_on_mlx_thread(_load_and_generate)
# Run blocking inference in thread pool
audio, sample_rate = await asyncio.to_thread(_generate_sync)
return audio, sample_rate
@@ -279,10 +279,12 @@ class MLXSTTBackend:
hf_repo = WHISPER_HF_REPOS.get(model_size, f"openai/whisper-{model_size}")
return is_model_cached(hf_repo, weight_extensions=(".safetensors", ".bin", ".npz"))
def _ensure_loaded_sync(self, model_size: str | None):
"""Load the model if the requested size isn't already resident.
async def load_model_async(self, model_size: Optional[str] = None):
"""
Lazy load the MLX Whisper model.
Runs on the MLX worker thread so it stays serialized with transcription.
Args:
model_size: Model size (tiny, base, small, medium, large)
"""
if model_size is None:
model_size = self.model_size
@@ -290,24 +292,12 @@ class MLXSTTBackend:
if self.model is not None and self.model_size == model_size:
return
self._load_model_sync(model_size)
async def load_model_async(self, model_size: str | None = None):
"""
Lazy load the MLX Whisper model.
Args:
model_size: Model size (tiny, base, small, medium, large)
"""
await run_on_mlx_thread(self._ensure_loaded_sync, model_size)
# Run blocking load in thread pool
await asyncio.to_thread(self._load_model_sync, model_size)
# Alias for compatibility
load_model = load_model_async
async def unload(self):
"""Free the model, serialized onto the MLX worker thread."""
await run_on_mlx_thread(self.unload_model)
def _load_model_sync(self, model_size: str):
"""Synchronous model loading."""
progress_model_name = f"whisper-{model_size}"
@@ -329,14 +319,13 @@ class MLXSTTBackend:
if self.model is not None:
del self.model
self.model = None
clear_mlx_cache()
logger.info("MLX Whisper model unloaded")
async def transcribe(
self,
audio_path: str,
language: str | None = None,
model_size: str | None = None,
language: Optional[str] = None,
model_size: Optional[str] = None,
) -> str:
"""
Transcribe audio to text.
@@ -349,6 +338,8 @@ class MLXSTTBackend:
Returns:
Transcribed text
"""
await self.load_model_async(model_size)
def _transcribe_sync():
"""Run synchronous transcription in thread pool."""
# MLX Whisper transcription using generate method
@@ -365,16 +356,12 @@ class MLXSTTBackend:
# Extract text from result
if isinstance(result, str):
return result.strip()
if isinstance(result, dict):
elif isinstance(result, dict):
return result.get("text", "").strip()
if hasattr(result, "text"):
elif hasattr(result, "text"):
return result.text.strip()
return str(result).strip()
else:
return str(result).strip()
# Load-if-needed and transcription run as one job on the MLX worker so
# a concurrent unload or load can't land between them.
def _load_and_transcribe():
self._ensure_loaded_sync(model_size)
return _transcribe_sync()
return await run_on_mlx_thread(_load_and_transcribe)
# Run blocking transcription in thread pool
return await asyncio.to_thread(_transcribe_sync)
+21 -21
View File
@@ -2,25 +2,25 @@
PyTorch backend implementation for TTS and STT.
"""
from typing import Optional, List, Tuple
import asyncio
import logging
import numpy as np
import torch
import numpy as np
logger = logging.getLogger(__name__)
from ..utils.audio import load_audio
from ..utils.cache import cache_voice_prompt, get_cache_key, get_cached_voice_prompt
from . import LANGUAGE_CODE_TO_NAME, WHISPER_HF_REPOS
from . import TTSBackend, STTBackend, LANGUAGE_CODE_TO_NAME, WHISPER_HF_REPOS
from .base import (
combine_voice_prompts as _combine_voice_prompts,
empty_device_cache,
get_torch_device,
is_model_cached,
get_torch_device,
empty_device_cache,
manual_seed,
combine_voice_prompts as _combine_voice_prompts,
model_load_progress,
)
from ..utils.cache import get_cache_key, get_cached_voice_prompt, cache_voice_prompt
from ..utils.audio import load_audio
class PyTorchTTSBackend:
@@ -63,7 +63,7 @@ class PyTorchTTSBackend:
def _is_model_cached(self, model_size: str) -> bool:
return is_model_cached(self._get_model_path(model_size))
async def load_model_async(self, model_size: str | None = None):
async def load_model_async(self, model_size: Optional[str] = None):
"""
Lazy load the TTS model with automatic downloading from HuggingFace Hub.
@@ -140,7 +140,7 @@ class PyTorchTTSBackend:
audio_path: str,
reference_text: str,
use_cache: bool = True,
) -> tuple[dict, bool]:
) -> Tuple[dict, bool]:
"""
Create voice prompt from reference audio.
@@ -165,7 +165,7 @@ class PyTorchTTSBackend:
# For PyTorch backend, the dict should contain tensors, not file paths
# So we can safely return it
return cached_prompt, True
if isinstance(cached_prompt, torch.Tensor):
elif isinstance(cached_prompt, torch.Tensor):
# Legacy cache format - convert to dict
# This shouldn't happen in practice, but handle it
return {"prompt": cached_prompt}, True
@@ -194,9 +194,9 @@ class PyTorchTTSBackend:
async def combine_voice_prompts(
self,
audio_paths: list[str],
reference_texts: list[str],
) -> tuple[np.ndarray, str]:
audio_paths: List[str],
reference_texts: List[str],
) -> Tuple[np.ndarray, str]:
return await _combine_voice_prompts(audio_paths, reference_texts)
async def generate(
@@ -204,9 +204,9 @@ class PyTorchTTSBackend:
text: str,
voice_prompt: dict,
language: str = "en",
seed: int | None = None,
instruct: str | None = None,
) -> tuple[np.ndarray, int]:
seed: Optional[int] = None,
instruct: Optional[str] = None,
) -> Tuple[np.ndarray, int]:
"""
Generate audio from text using voice prompt.
@@ -266,7 +266,7 @@ class PyTorchSTTBackend:
hf_repo = WHISPER_HF_REPOS.get(model_size, f"openai/whisper-{model_size}")
return is_model_cached(hf_repo)
async def load_model_async(self, model_size: str | None = None):
async def load_model_async(self, model_size: Optional[str] = None):
"""
Lazy load the Whisper model.
@@ -290,7 +290,7 @@ class PyTorchSTTBackend:
is_cached = self._is_model_cached(model_size)
with model_load_progress(progress_model_name, is_cached):
from transformers import WhisperForConditionalGeneration, WhisperProcessor
from transformers import WhisperProcessor, WhisperForConditionalGeneration
model_name = WHISPER_HF_REPOS.get(model_size, f"openai/whisper-{model_size}")
logger.info("Loading Whisper model %s on %s...", model_size, self.device)
@@ -317,8 +317,8 @@ class PyTorchSTTBackend:
async def transcribe(
self,
audio_path: str,
language: str | None = None,
model_size: str | None = None,
language: Optional[str] = None,
model_size: Optional[str] = None,
) -> str:
"""
Transcribe audio to text.
@@ -16,15 +16,16 @@ Languages supported: zh, en, ja, ko, de, fr, ru, pt, es, it
import asyncio
import logging
from typing import Optional
import numpy as np
import torch
from . import LANGUAGE_CODE_TO_NAME
from . import TTSBackend, LANGUAGE_CODE_TO_NAME
from .base import (
combine_voice_prompts as _combine_voice_prompts,
get_torch_device,
is_model_cached,
get_torch_device,
combine_voice_prompts as _combine_voice_prompts,
model_load_progress,
)
@@ -61,7 +62,7 @@ class QwenCustomVoiceBackend:
self.model = None
self.model_size = model_size
self.device = self._get_device()
self._current_model_size: str | None = None
self._current_model_size: Optional[str] = None
def _get_device(self) -> str:
return get_torch_device(allow_xpu=True, allow_directml=True)
@@ -74,11 +75,11 @@ class QwenCustomVoiceBackend:
raise ValueError(f"Unknown model size: {model_size}")
return QWEN_CV_HF_REPOS[model_size]
def _is_model_cached(self, model_size: str | None = None) -> bool:
def _is_model_cached(self, model_size: Optional[str] = None) -> bool:
size = model_size or self.model_size
return is_model_cached(self._get_model_path(size))
async def load_model_async(self, model_size: str | None = None) -> None:
async def load_model_async(self, model_size: Optional[str] = None) -> None:
if model_size is None:
model_size = self.model_size
@@ -163,8 +164,8 @@ class QwenCustomVoiceBackend:
text: str,
voice_prompt: dict,
language: str = "en",
seed: int | None = None,
instruct: str | None = None,
seed: Optional[int] = None,
instruct: Optional[str] = None,
) -> tuple[np.ndarray, int]:
"""
Generate audio using Qwen CustomVoice.
+41 -52
View File
@@ -9,14 +9,14 @@ and STT engines.
import asyncio
import logging
from typing import Optional
from ..services.mlx_thread import clear_mlx_cache, run_on_mlx_thread
from ..utils.hf_offline_patch import force_offline_if_cached
from . import DEFAULT_LLM_MAX_TOKENS, DEFAULT_LLM_TEMPERATURE
from . import LLMBackend, DEFAULT_LLM_MAX_TOKENS, DEFAULT_LLM_TEMPERATURE
from .base import (
empty_device_cache,
get_torch_device,
is_model_cached,
get_torch_device,
empty_device_cache,
manual_seed,
model_load_progress,
)
@@ -42,8 +42,8 @@ def _progress_name(model_size: str) -> str:
def _build_messages(
prompt: str,
system: str | None,
examples: list[tuple[str, str]] | None = None,
system: Optional[str],
examples: Optional[list[tuple[str, str]]] = None,
) -> list[dict]:
messages: list[dict] = []
if system:
@@ -63,7 +63,7 @@ class PyTorchQwenLLMBackend:
self.model = None
self.tokenizer = None
self.model_size = model_size
self._current_model_size: str | None = None
self._current_model_size: Optional[str] = None
self.device = self._get_device()
def _get_device(self) -> str:
@@ -80,7 +80,7 @@ class PyTorchQwenLLMBackend:
def _is_model_cached(self, model_size: str) -> bool:
return is_model_cached(self._get_model_path(model_size))
async def load_model(self, model_size: str | None = None) -> None:
async def load_model(self, model_size: Optional[str] = None) -> None:
if model_size is None:
model_size = self.model_size
@@ -102,15 +102,19 @@ class PyTorchQwenLLMBackend:
with model_load_progress(progress_model_name, is_cached):
logger.info("Loading Qwen3 %s on %s...", model_size, self.device)
with force_offline_if_cached(is_cached, progress_model_name):
self.tokenizer = AutoTokenizer.from_pretrained(repo)
dtype = torch.float16 if self.device in ("cuda", "mps") else torch.float32
self.model = AutoModelForCausalLM.from_pretrained(
repo,
dtype=dtype,
)
self.model.to(self.device)
self.model.eval()
# Loads run with the process's default HF_HUB_OFFLINE state.
# Forcing offline for cached models flips process-global state
# and silently switches every concurrent download/load on other
# threads to offline mode (issue #841) — the same regression
# removed app-wide in #524/#530.
self.tokenizer = AutoTokenizer.from_pretrained(repo)
dtype = torch.float16 if self.device in ("cuda", "mps") else torch.float32
self.model = AutoModelForCausalLM.from_pretrained(
repo,
dtype=dtype,
)
self.model.to(self.device)
self.model.eval()
self._current_model_size = model_size
self.model_size = model_size
@@ -130,11 +134,11 @@ class PyTorchQwenLLMBackend:
async def generate(
self,
prompt: str,
system: str | None = None,
system: Optional[str] = None,
max_tokens: int = DEFAULT_LLM_MAX_TOKENS,
temperature: float = DEFAULT_LLM_TEMPERATURE,
model_size: str | None = None,
examples: list[tuple[str, str]] | None = None,
model_size: Optional[str] = None,
examples: Optional[list[tuple[str, str]]] = None,
) -> str:
await self.load_model(model_size)
return await asyncio.to_thread(
@@ -144,10 +148,10 @@ class PyTorchQwenLLMBackend:
def _generate_sync(
self,
prompt: str,
system: str | None,
system: Optional[str],
max_tokens: int,
temperature: float,
examples: list[tuple[str, str]] | None = None,
examples: Optional[list[tuple[str, str]]] = None,
) -> str:
import torch
@@ -185,7 +189,7 @@ class MLXQwenLLMBackend:
self.model = None
self.tokenizer = None
self.model_size = model_size
self._current_model_size: str | None = None
self._current_model_size: Optional[str] = None
def is_loaded(self) -> bool:
return self.model is not None
@@ -201,11 +205,7 @@ class MLXQwenLLMBackend:
weight_extensions=(".safetensors", ".bin", ".npz"),
)
def _ensure_loaded_sync(self, model_size: str | None) -> None:
"""Load the model if the requested size isn't already resident.
Runs on the MLX worker thread so it stays serialized with generation.
"""
async def load_model(self, model_size: Optional[str] = None) -> None:
if model_size is None:
model_size = self.model_size
@@ -215,14 +215,7 @@ class MLXQwenLLMBackend:
if self.model is not None and self._current_model_size != model_size:
self.unload_model()
self._load_model_sync(model_size)
async def load_model(self, model_size: str | None = None) -> None:
await run_on_mlx_thread(self._ensure_loaded_sync, model_size)
async def unload(self) -> None:
"""Free the model, serialized onto the MLX worker thread."""
await run_on_mlx_thread(self.unload_model)
await asyncio.to_thread(self._load_model_sync, model_size)
def _load_model_sync(self, model_size: str) -> None:
from mlx_lm import load as mlx_load
@@ -233,8 +226,8 @@ class MLXQwenLLMBackend:
with model_load_progress(progress_model_name, is_cached):
logger.info("Loading Qwen3 %s via MLX...", model_size)
with force_offline_if_cached(is_cached, progress_model_name):
loaded = mlx_load(repo)
# See the PyTorch loader comment — no offline forcing (issue #841).
loaded = mlx_load(repo)
# mlx_lm.load returns (model, tokenizer) by default and
# (model, tokenizer, config) when return_config=True.
@@ -253,33 +246,29 @@ class MLXQwenLLMBackend:
self.model = None
self.tokenizer = None
self._current_model_size = None
clear_mlx_cache()
logger.info("Qwen3 (MLX) unloaded")
async def generate(
self,
prompt: str,
system: str | None = None,
system: Optional[str] = None,
max_tokens: int = DEFAULT_LLM_MAX_TOKENS,
temperature: float = DEFAULT_LLM_TEMPERATURE,
model_size: str | None = None,
examples: list[tuple[str, str]] | None = None,
model_size: Optional[str] = None,
examples: Optional[list[tuple[str, str]]] = None,
) -> str:
# Load-if-needed and inference run as one job on the MLX worker so a
# concurrent unload or different-size load can't land between them.
def _load_and_generate() -> str:
self._ensure_loaded_sync(model_size)
return self._generate_sync(prompt, system, max_tokens, temperature, examples)
return await run_on_mlx_thread(_load_and_generate)
await self.load_model(model_size)
return await asyncio.to_thread(
self._generate_sync, prompt, system, max_tokens, temperature, examples
)
def _generate_sync(
self,
prompt: str,
system: str | None,
system: Optional[str],
max_tokens: int,
temperature: float,
examples: list[tuple[str, str]] | None = None,
examples: Optional[list[tuple[str, str]]] = None,
) -> str:
from mlx_lm import generate as mlx_generate
from mlx_lm.sample_utils import make_sampler
+3
View File
@@ -330,6 +330,9 @@ def build_server(cuda=False, rocm=False):
]
)
if sys.version_info >= (3, 13):
args.extend(["--hidden-import", "audioop"])
# Add CUDA/ROCm-specific hidden imports
if cuda or rocm:
variant = "ROCm" if rocm else "CUDA"
+5
View File
@@ -80,6 +80,11 @@ def resolve_storage_path(path: str | Path | None) -> Path | None:
return None
stored_path = Path(path)
# Empty paths (e.g. failed generations) must not resolve to the data
# dir itself, which exists and would defeat the callers' 404 guards.
# Path("") is truthy, so check parts rather than the raw value.
if not stored_path.parts:
return None
if stored_path.is_absolute():
rebased_path = _path_relative_to_any_data_dir(stored_path)
if rebased_path is not None:
+6 -6
View File
@@ -6,8 +6,8 @@ without changing any importers.
"""
from .models import (
AudioChannel,
Base,
AudioChannel,
Capture,
CaptureSettings,
ChannelDeviceMapping,
@@ -24,12 +24,12 @@ from .models import (
StoryItem,
VoiceProfile,
)
from .session import SessionLocal, _db_path, engine, get_db, init_db
from .session import engine, SessionLocal, _db_path, init_db, get_db
__all__ = [
"AudioChannel",
# Models
"Base",
"AudioChannel",
"Capture",
"CaptureSettings",
"ChannelDeviceMapping",
@@ -42,13 +42,13 @@ __all__ = [
"ProfileChannelMapping",
"ProfileSample",
"Project",
"SessionLocal",
"Story",
"StoryItem",
"VoiceProfile",
"_db_path",
# Session
"engine",
"get_db",
"SessionLocal",
"_db_path",
"init_db",
"get_db",
]
+1 -8
View File
@@ -243,13 +243,6 @@ def _migrate_capture_settings(engine, inspector, tables: set[str]) -> None:
"hotkey_enabled BOOLEAN NOT NULL DEFAULT 0",
"hotkey_enabled",
)
if "keep_mic_warm" not in columns:
_add_column(
engine,
"capture_settings",
"keep_mic_warm BOOLEAN NOT NULL DEFAULT 0",
"keep_mic_warm",
)
def _migrate_mcp_bindings(engine, inspector, tables: set[str]) -> None:
@@ -303,7 +296,7 @@ def _normalize_storage_paths(engine, tables: set[str]) -> None:
"""Normalize stored file paths to be relative to the configured data dir."""
from pathlib import Path
from ..config import get_data_dir, resolve_storage_path, to_storage_path
from ..config import get_data_dir, to_storage_path, resolve_storage_path
data_dir = get_data_dir()
+2 -6
View File
@@ -1,9 +1,9 @@
"""ORM model definitions for the voicebox SQLite database."""
import uuid
from datetime import datetime
import uuid
from sqlalchemy import JSON, Boolean, Column, DateTime, Float, ForeignKey, Integer, String, Text
from sqlalchemy import Column, String, Integer, Float, DateTime, Text, ForeignKey, Boolean, JSON
from sqlalchemy.ext.declarative import declarative_base
from ..utils.capture_chords import (
@@ -210,10 +210,6 @@ class CaptureSettings(Base):
# "Voicebox would like to receive keystrokes from any application" dialog
# before they've even opened the Captures tab.
hotkey_enabled = Column(Boolean, nullable=False, default=False)
# Hold the microphone open while dictation is enabled so push-to-talk
# doesn't clip the first words. Off by default — when on, the OS mic-in-use
# indicator stays lit the whole time dictation is enabled.
keep_mic_warm = Column(Boolean, nullable=False, default=False)
# Lists of keytap key names (e.g. "MetaRight", "ControlRight"). Right-hand
# modifiers by default so they don't collide with left-hand shortcuts.
chord_push_to_talk_keys = Column(
+5 -18
View File
@@ -3,20 +3,20 @@
import logging
import uuid
from sqlalchemy import create_engine, event
from sqlalchemy import create_engine
from sqlalchemy.orm import sessionmaker
from .. import config
from .migrations import run_migrations
from .models import (
AudioChannel,
Base,
AudioChannel,
EffectPreset,
Generation,
GenerationVersion,
ProfileChannelMapping,
VoiceProfile,
)
from .migrations import run_migrations
from .seed import backfill_generation_versions, seed_builtin_presets
logger = logging.getLogger(__name__)
@@ -36,22 +36,9 @@ def init_db() -> None:
engine = create_engine(
f"sqlite:///{_db_path}",
# timeout is sqlite3's busy handler: wait up to 30s on a locked
# database instead of raising "database is locked" immediately.
connect_args={"check_same_thread": False, "timeout": 30},
connect_args={"check_same_thread": False},
)
@event.listens_for(engine, "connect")
def _set_sqlite_pragmas(dbapi_connection, _connection_record):
# WAL lets readers proceed while a writer holds the lock, which is
# the main source of lock racing between the generation worker and
# request handlers. synchronous=NORMAL is the recommended pairing
# (durable across app crashes, fsyncs only on checkpoint).
cursor = dbapi_connection.cursor()
cursor.execute("PRAGMA journal_mode=WAL")
cursor.execute("PRAGMA synchronous=NORMAL")
cursor.close()
SessionLocal = sessionmaker(autocommit=False, autoflush=False, bind=engine)
run_migrations(engine)
@@ -60,7 +47,7 @@ def init_db() -> None:
# Create default audio channel if it doesn't exist
db = SessionLocal()
try:
default_channel = db.query(AudioChannel).filter(AudioChannel.is_default).first()
default_channel = db.query(AudioChannel).filter(AudioChannel.is_default == True).first()
if not default_channel:
default_channel = AudioChannel(
id=str(uuid.uuid4()),
+1 -2
View File
@@ -5,11 +5,10 @@ entry point for development.
"""
import argparse
import uvicorn
from . import config, database
from .app import app # noqa: F401 -- re-export for uvicorn "backend.main:app"
from . import config, database
if __name__ == "__main__":
parser = argparse.ArgumentParser(description="voicebox backend server")
+3 -2
View File
@@ -11,13 +11,14 @@ import asyncio
import ipaddress
import logging
from contextvars import ContextVar
from datetime import UTC, datetime
from datetime import datetime, timezone
from starlette.middleware.base import BaseHTTPMiddleware
from starlette.requests import Request
from starlette.responses import Response
from starlette.types import ASGIApp
logger = logging.getLogger(__name__)
# Strong refs to in-flight stamp tasks so asyncio.create_task results
@@ -140,7 +141,7 @@ def _stamp_last_seen(client_id: str) -> None:
if row is None:
row = MCPClientBinding(client_id=client_id)
db.add(row)
row.last_seen_at = datetime.now(UTC)
row.last_seen_at = datetime.now(timezone.utc)
db.commit()
except Exception:
logger.debug(
+1
View File
@@ -8,6 +8,7 @@ floating pill surfaces whenever an agent is speaking.
import asyncio
from typing import Any
# Each subscriber gets its own queue. Bounded to drop oldest if a client lags.
_subscribers: set[asyncio.Queue[dict[str, Any]]] = set()
+1 -1
View File
@@ -30,7 +30,7 @@ def resolve_profile(
if client_id:
# Per-client binding. Imported lazily so this module stays importable
# even before the migration adds the table on first boot.
from ..database.models import MCPClientBinding
from ..database.models import MCPClientBinding # noqa: WPS433
binding = (
db.query(MCPClientBinding)
+2 -1
View File
@@ -9,8 +9,8 @@ binary bundled with the desktop app.
from __future__ import annotations
import logging
from collections.abc import Callable
from contextlib import AsyncExitStack, asynccontextmanager
from collections.abc import Callable
from fastapi import FastAPI
from fastmcp import FastMCP
@@ -18,6 +18,7 @@ from fastmcp import FastMCP
from .context import ClientIdMiddleware
from .tools import register_tools
logger = logging.getLogger(__name__)
+3 -1
View File
@@ -18,11 +18,13 @@ from fastmcp import FastMCP
from .. import models
from ..database import get_db
from ..services import captures as captures_service, profiles as profiles_service
from ..services import captures as captures_service
from ..services import profiles as profiles_service
from . import events as mcp_events
from .context import current_client_id, request_is_loopback
from .resolve import resolve_profile
logger = logging.getLogger(__name__)
# Absolute-path transcribes are bounded to keep a bad client from
+1
View File
@@ -23,6 +23,7 @@ from typing import Any
import httpx
CLIENT_ID_HEADER = "X-Voicebox-Client-Id"
SESSION_HEADER = "mcp-session-id"
HEALTH_TIMEOUT_S = 30.0
+152 -154
View File
@@ -2,9 +2,9 @@
Pydantic models for request/response validation.
"""
from datetime import datetime
from pydantic import BaseModel, Field
from typing import Optional, List
from datetime import datetime
from .utils.capture_chords import (
default_push_to_talk_chord,
@@ -16,16 +16,16 @@ class VoiceProfileCreate(BaseModel):
"""Request model for creating a voice profile."""
name: str = Field(..., min_length=1, max_length=100)
description: str | None = Field(None, max_length=500)
description: Optional[str] = Field(None, max_length=500)
language: str = Field(
default="en", pattern="^(zh|en|ja|ko|de|fr|ru|pt|es|it|he|ar|da|el|fi|hi|ms|nl|no|pl|sv|sw|tr)$"
)
voice_type: str | None = Field(default="cloned", pattern="^(cloned|preset|designed)$")
preset_engine: str | None = Field(None, max_length=50)
preset_voice_id: str | None = Field(None, max_length=100)
design_prompt: str | None = Field(None, max_length=2000)
default_engine: str | None = Field(None, max_length=50)
personality: str | None = Field(None, max_length=2000)
voice_type: Optional[str] = Field(default="cloned", pattern="^(cloned|preset|designed)$")
preset_engine: Optional[str] = Field(None, max_length=50)
preset_voice_id: Optional[str] = Field(None, max_length=100)
design_prompt: Optional[str] = Field(None, max_length=2000)
default_engine: Optional[str] = Field(None, max_length=50)
personality: Optional[str] = Field(None, max_length=2000)
class VoiceProfileResponse(BaseModel):
@@ -33,16 +33,16 @@ class VoiceProfileResponse(BaseModel):
id: str
name: str
description: str | None
description: Optional[str]
language: str
avatar_path: str | None = None
effects_chain: list["EffectConfig"] | None = None
avatar_path: Optional[str] = None
effects_chain: Optional[List["EffectConfig"]] = None
voice_type: str = "cloned"
preset_engine: str | None = None
preset_voice_id: str | None = None
design_prompt: str | None = None
default_engine: str | None = None
personality: str | None = None
preset_engine: Optional[str] = None
preset_voice_id: Optional[str] = None
design_prompt: Optional[str] = None
default_engine: Optional[str] = None
personality: Optional[str] = None
generation_count: int = 0
sample_count: int = 0
created_at: datetime
@@ -82,10 +82,10 @@ class GenerationRequest(BaseModel):
profile_id: str
text: str = Field(..., min_length=1, max_length=50000)
language: str = Field(default="en", pattern="^(zh|en|ja|ko|de|fr|ru|pt|es|it|he|ar|da|el|fi|hi|ms|nl|no|pl|sv|sw|tr)$")
seed: int | None = Field(None, ge=0)
model_size: str | None = Field(default="1.7B", pattern="^(1\\.7B|0\\.6B|1B|3B)$")
instruct: str | None = Field(None, max_length=500)
engine: str | None = Field(default="qwen", pattern="^(qwen|qwen_custom_voice|luxtts|chatterbox|chatterbox_turbo|tada|kokoro)$")
seed: Optional[int] = Field(None, ge=0)
model_size: Optional[str] = Field(default="1.7B", pattern="^(1\\.7B|0\\.6B|1B|3B)$")
instruct: Optional[str] = Field(None, max_length=500)
engine: Optional[str] = Field(default="qwen", pattern="^(qwen|qwen_custom_voice|luxtts|chatterbox|chatterbox_turbo|tada|kokoro)$")
personality: bool = Field(
default=False,
description="When true and the profile has a personality prompt, the input text is rewritten in-character before TTS.",
@@ -97,7 +97,7 @@ class GenerationRequest(BaseModel):
default=50, ge=0, le=500, description="Crossfade duration in ms between chunks (0 for hard cut)"
)
normalize: bool = Field(default=True, description="Normalize output audio volume")
effects_chain: list["EffectConfig"] | None = Field(
effects_chain: Optional[List["EffectConfig"]] = Field(
None, description="Effects chain to apply after generation (overrides profile default)"
)
@@ -109,19 +109,19 @@ class GenerationResponse(BaseModel):
profile_id: str
text: str
language: str
audio_path: str | None = None
duration: float | None = None
seed: int | None = None
instruct: str | None = None
engine: str | None = "qwen"
model_size: str | None = None
audio_path: Optional[str] = None
duration: Optional[float] = None
seed: Optional[int] = None
instruct: Optional[str] = None
engine: Optional[str] = "qwen"
model_size: Optional[str] = None
status: str = "completed"
error: str | None = None
error: Optional[str] = None
is_favorited: bool = False
source: str = "manual"
created_at: datetime
versions: list["GenerationVersionResponse"] | None = None
active_version_id: str | None = None
versions: Optional[List["GenerationVersionResponse"]] = None
active_version_id: Optional[str] = None
class Config:
from_attributes = True
@@ -130,8 +130,8 @@ class GenerationResponse(BaseModel):
class HistoryQuery(BaseModel):
"""Query model for generation history."""
profile_id: str | None = None
search: str | None = None
profile_id: Optional[str] = None
search: Optional[str] = None
limit: int = Field(default=50, ge=1, le=100)
offset: int = Field(default=0, ge=0)
@@ -144,18 +144,18 @@ class HistoryResponse(BaseModel):
profile_name: str
text: str
language: str
audio_path: str | None = None
duration: float | None = None
seed: int | None = None
instruct: str | None = None
engine: str | None = "qwen"
model_size: str | None = None
audio_path: Optional[str] = None
duration: Optional[float] = None
seed: Optional[int] = None
instruct: Optional[str] = None
engine: Optional[str] = "qwen"
model_size: Optional[str] = None
status: str = "completed"
error: str | None = None
error: Optional[str] = None
is_favorited: bool = False
created_at: datetime
versions: list["GenerationVersionResponse"] | None = None
active_version_id: str | None = None
versions: Optional[List["GenerationVersionResponse"]] = None
active_version_id: Optional[str] = None
class Config:
from_attributes = True
@@ -164,15 +164,15 @@ class HistoryResponse(BaseModel):
class HistoryListResponse(BaseModel):
"""Response model for history list."""
items: list[HistoryResponse]
items: List[HistoryResponse]
total: int
class TranscriptionRequest(BaseModel):
"""Request model for audio transcription."""
language: str | None = Field(None, pattern="^(en|zh|ja|ko|de|fr|ru|pt|es|it)$")
model: str | None = Field(None, pattern="^(base|small|medium|large|turbo)$")
language: Optional[str] = Field(None, pattern="^(en|zh|ja|ko|de|fr|ru|pt|es|it)$")
model: Optional[str] = Field(None, pattern="^(base|small|medium|large|turbo)$")
class TranscriptionResponse(BaseModel):
@@ -196,13 +196,13 @@ class CaptureResponse(BaseModel):
id: str
audio_path: str
source: str
language: str | None = None
duration_ms: int | None = None
language: Optional[str] = None
duration_ms: Optional[int] = None
transcript_raw: str
transcript_refined: str | None = None
stt_model: str | None = None
llm_model: str | None = None
refinement_flags: RefinementFlagsModel | None = None
transcript_refined: Optional[str] = None
stt_model: Optional[str] = None
llm_model: Optional[str] = None
refinement_flags: Optional[RefinementFlagsModel] = None
created_at: datetime
class Config:
@@ -212,7 +212,7 @@ class CaptureResponse(BaseModel):
class CaptureListResponse(BaseModel):
"""Response model for paginated capture list."""
items: list[CaptureResponse]
items: List[CaptureResponse]
total: int
@@ -234,15 +234,15 @@ class CaptureCreateResponse(CaptureResponse):
class CaptureRefineRequest(BaseModel):
"""Request to refine a capture's transcript via the LLM."""
flags: RefinementFlagsModel | None = None
model_size: str | None = Field(default=None, pattern="^(0\\.6B|1\\.7B|4B)$")
flags: Optional[RefinementFlagsModel] = None
model_size: Optional[str] = Field(default=None, pattern="^(0\\.6B|1\\.7B|4B)$")
class CaptureRetranscribeRequest(BaseModel):
"""Request to re-run STT on a capture's audio with a different model."""
model: str | None = Field(None, pattern="^(base|small|medium|large|turbo)$")
language: str | None = Field(None, pattern="^(en|zh|ja|ko|de|fr|ru|pt|es|it)$")
model: Optional[str] = Field(None, pattern="^(base|small|medium|large|turbo)$")
language: Optional[str] = Field(None, pattern="^(en|zh|ja|ko|de|fr|ru|pt|es|it)$")
class CaptureSettingsResponse(BaseModel):
@@ -256,13 +256,12 @@ class CaptureSettingsResponse(BaseModel):
self_correction: bool = True
preserve_technical: bool = True
allow_auto_paste: bool = True
default_playback_voice_id: str | None = None
default_playback_voice_id: Optional[str] = None
hotkey_enabled: bool = False
keep_mic_warm: bool = False
chord_push_to_talk_keys: list[str] = Field(
chord_push_to_talk_keys: List[str] = Field(
default_factory=default_push_to_talk_chord
)
chord_toggle_to_talk_keys: list[str] = Field(
chord_toggle_to_talk_keys: List[str] = Field(
default_factory=default_toggle_to_talk_chord
)
@@ -273,19 +272,18 @@ class CaptureSettingsResponse(BaseModel):
class CaptureSettingsUpdate(BaseModel):
"""Partial update for capture settings — every field is optional."""
stt_model: str | None = Field(default=None, pattern="^(base|small|medium|large|turbo)$")
language: str | None = None
auto_refine: bool | None = None
llm_model: str | None = Field(default=None, pattern="^(0\\.6B|1\\.7B|4B)$")
smart_cleanup: bool | None = None
self_correction: bool | None = None
preserve_technical: bool | None = None
allow_auto_paste: bool | None = None
default_playback_voice_id: str | None = None
hotkey_enabled: bool | None = None
keep_mic_warm: bool | None = None
chord_push_to_talk_keys: list[str] | None = Field(default=None, min_length=1, max_length=6)
chord_toggle_to_talk_keys: list[str] | None = Field(default=None, min_length=1, max_length=6)
stt_model: Optional[str] = Field(default=None, pattern="^(base|small|medium|large|turbo)$")
language: Optional[str] = None
auto_refine: Optional[bool] = None
llm_model: Optional[str] = Field(default=None, pattern="^(0\\.6B|1\\.7B|4B)$")
smart_cleanup: Optional[bool] = None
self_correction: Optional[bool] = None
preserve_technical: Optional[bool] = None
allow_auto_paste: Optional[bool] = None
default_playback_voice_id: Optional[str] = None
hotkey_enabled: Optional[bool] = None
chord_push_to_talk_keys: Optional[List[str]] = Field(default=None, min_length=1, max_length=6)
chord_toggle_to_talk_keys: Optional[List[str]] = Field(default=None, min_length=1, max_length=6)
class GenerationSettingsResponse(BaseModel):
@@ -303,10 +301,10 @@ class GenerationSettingsResponse(BaseModel):
class GenerationSettingsUpdate(BaseModel):
"""Partial update for generation settings — every field is optional."""
max_chunk_chars: int | None = Field(default=None, ge=100, le=5000)
crossfade_ms: int | None = Field(default=None, ge=0, le=500)
normalize_audio: bool | None = None
autoplay_on_generate: bool | None = None
max_chunk_chars: Optional[int] = Field(default=None, ge=100, le=5000)
crossfade_ms: Optional[int] = Field(default=None, ge=0, le=500)
normalize_audio: Optional[bool] = None
autoplay_on_generate: Optional[bool] = None
class MCPClientBindingResponse(BaseModel):
@@ -315,14 +313,14 @@ class MCPClientBindingResponse(BaseModel):
opt-in personality-rewrite default."""
client_id: str
label: str | None = None
profile_id: str | None = None
default_engine: str | None = Field(
label: Optional[str] = None
profile_id: Optional[str] = None
default_engine: Optional[str] = Field(
None,
pattern="^(qwen|qwen_custom_voice|luxtts|chatterbox|chatterbox_turbo|tada|kokoro)$",
)
default_personality: bool = False
last_seen_at: datetime | None = None
last_seen_at: Optional[datetime] = None
created_at: datetime
updated_at: datetime
@@ -334,9 +332,9 @@ class MCPClientBindingUpsert(BaseModel):
"""Create or update a binding. Matched by ``client_id``."""
client_id: str = Field(..., min_length=1, max_length=64)
label: str | None = Field(None, max_length=128)
profile_id: str | None = None
default_engine: str | None = Field(
label: Optional[str] = Field(None, max_length=128)
profile_id: Optional[str] = None
default_engine: Optional[str] = Field(
None,
pattern="^(qwen|qwen_custom_voice|luxtts|chatterbox|chatterbox_turbo|tada|kokoro)$",
)
@@ -344,26 +342,26 @@ class MCPClientBindingUpsert(BaseModel):
class MCPClientBindingListResponse(BaseModel):
items: list[MCPClientBindingResponse]
items: List[MCPClientBindingResponse]
class SpeakRequest(BaseModel):
"""Body for POST /speak — non-MCP REST surface that mirrors voicebox.speak."""
text: str = Field(..., min_length=1, max_length=10000)
profile: str | None = Field(
profile: Optional[str] = Field(
None,
description="Voice profile name or id. Falls back to per-client binding, then default.",
)
engine: str | None = Field(
engine: Optional[str] = Field(
None,
pattern="^(qwen|qwen_custom_voice|luxtts|chatterbox|chatterbox_turbo|tada|kokoro)$",
)
personality: bool | None = Field(
personality: Optional[bool] = Field(
None,
description="When true and the profile has a personality prompt, the input text is rewritten in-character before TTS. When null, the per-client binding's default_personality flag decides.",
)
language: str | None = Field(
language: Optional[str] = Field(
None,
pattern="^(zh|en|ja|ko|de|fr|ru|pt|es|it|he|ar|da|el|fi|hi|ms|nl|no|pl|sv|sw|tr)$",
)
@@ -373,15 +371,15 @@ class LLMGenerateRequest(BaseModel):
"""Request model for LLM text generation."""
prompt: str = Field(..., min_length=1, max_length=50000)
system: str | None = Field(None, max_length=4000)
model_size: str | None = Field(default="0.6B", pattern="^(0\\.6B|1\\.7B|4B)$")
system: Optional[str] = Field(None, max_length=4000)
model_size: Optional[str] = Field(default="0.6B", pattern="^(0\\.6B|1\\.7B|4B)$")
max_tokens: int = Field(default=512, ge=1, le=4096)
temperature: float = Field(default=0.7, ge=0.0, le=2.0)
# Few-shot (user, assistant) pairs prepended as real chat turns.
# Used by the refinement service to pin tricky rules (imperatives
# staying imperatives, technical-term punctuation) that small models
# lose when the examples live inline in the system prompt.
examples: list[list[str]] | None = Field(default=None, max_length=8)
examples: Optional[List[List[str]]] = Field(default=None, max_length=8)
class LLMGenerateResponse(BaseModel):
@@ -418,7 +416,7 @@ class ModelReadiness(BaseModel):
model_name: str
display_name: str
size: str
size_mb: int | None = None
size_mb: Optional[int] = None
class CaptureReadinessResponse(BaseModel):
@@ -438,15 +436,15 @@ class HealthResponse(BaseModel):
status: str
model_loaded: bool
model_downloaded: bool | None = None # Whether model is cached/downloaded
model_size: str | None = None # Current model size if loaded
model_downloaded: Optional[bool] = None # Whether model is cached/downloaded
model_size: Optional[str] = None # Current model size if loaded
gpu_available: bool
gpu_type: str | None = None # GPU type (CUDA, MPS, or None)
vram_used_mb: float | None = None
backend_type: str | None = None # Backend type (mlx or pytorch)
backend_variant: str | None = None # Binary variant (cpu, cuda, or rocm)
gpu_type: Optional[str] = None # GPU type (CUDA, MPS, or None)
vram_used_mb: Optional[float] = None
backend_type: Optional[str] = None # Backend type (mlx or pytorch)
backend_variant: Optional[str] = None # Binary variant (cpu, cuda, or rocm)
supports_rocm: bool = False # AMD GPU on Windows — the ROCm backend is applicable
gpu_compatibility_warning: str | None = None # Warning if GPU arch unsupported
gpu_compatibility_warning: Optional[str] = None # Warning if GPU arch unsupported
class DirectoryCheck(BaseModel):
@@ -455,16 +453,16 @@ class DirectoryCheck(BaseModel):
path: str
exists: bool
writable: bool
error: str | None = None
error: Optional[str] = None
class FilesystemHealthResponse(BaseModel):
"""Response model for filesystem health check."""
healthy: bool
disk_free_mb: float | None = None
disk_total_mb: float | None = None
directories: list[DirectoryCheck]
disk_free_mb: Optional[float] = None
disk_total_mb: Optional[float] = None
directories: List[DirectoryCheck]
class ModelStatus(BaseModel):
@@ -472,17 +470,17 @@ class ModelStatus(BaseModel):
model_name: str
display_name: str
hf_repo_id: str | None = None # HuggingFace repository ID
hf_repo_id: Optional[str] = None # HuggingFace repository ID
downloaded: bool
downloading: bool = False # True if download is in progress
size_mb: float | None = None
size_mb: Optional[float] = None
loaded: bool = False
class ModelStatusListResponse(BaseModel):
"""Response model for model status list."""
models: list[ModelStatus]
models: List[ModelStatus]
class ModelDownloadRequest(BaseModel):
@@ -503,11 +501,11 @@ class ActiveDownloadTask(BaseModel):
model_name: str
status: str
started_at: datetime
error: str | None = None
progress: float | None = None # 0-100 percentage
current: int | None = None # bytes downloaded
total: int | None = None # total bytes
filename: str | None = None # current file being downloaded
error: Optional[str] = None
progress: Optional[float] = None # 0-100 percentage
current: Optional[int] = None # bytes downloaded
total: Optional[int] = None # total bytes
filename: Optional[str] = None # current file being downloaded
class ActiveGenerationTask(BaseModel):
@@ -522,22 +520,22 @@ class ActiveGenerationTask(BaseModel):
class ActiveTasksResponse(BaseModel):
"""Response model for active tasks."""
downloads: list[ActiveDownloadTask]
generations: list[ActiveGenerationTask]
downloads: List[ActiveDownloadTask]
generations: List[ActiveGenerationTask]
class AudioChannelCreate(BaseModel):
"""Request model for creating an audio channel."""
name: str = Field(..., min_length=1, max_length=100)
device_ids: list[str] = Field(default_factory=list)
device_ids: List[str] = Field(default_factory=list)
class AudioChannelUpdate(BaseModel):
"""Request model for updating an audio channel."""
name: str | None = Field(None, min_length=1, max_length=100)
device_ids: list[str] | None = None
name: Optional[str] = Field(None, min_length=1, max_length=100)
device_ids: Optional[List[str]] = None
class AudioChannelResponse(BaseModel):
@@ -546,7 +544,7 @@ class AudioChannelResponse(BaseModel):
id: str
name: str
is_default: bool
device_ids: list[str]
device_ids: List[str]
created_at: datetime
class Config:
@@ -556,20 +554,20 @@ class AudioChannelResponse(BaseModel):
class ChannelVoiceAssignment(BaseModel):
"""Request model for assigning voices to a channel."""
profile_ids: list[str]
profile_ids: List[str]
class ProfileChannelAssignment(BaseModel):
"""Request model for assigning channels to a profile."""
channel_ids: list[str]
channel_ids: List[str]
class StoryCreate(BaseModel):
"""Request model for creating a story."""
name: str = Field(..., min_length=1, max_length=100)
description: str | None = Field(None, max_length=500)
description: Optional[str] = Field(None, max_length=500)
class StoryResponse(BaseModel):
@@ -577,7 +575,7 @@ class StoryResponse(BaseModel):
id: str
name: str
description: str | None
description: Optional[str]
created_at: datetime
updated_at: datetime
item_count: int = 0
@@ -592,7 +590,7 @@ class StoryItemDetail(BaseModel):
id: str
story_id: str
generation_id: str
version_id: str | None = None
version_id: Optional[str] = None
start_time_ms: int
track: int = 0
trim_start_ms: int = 0
@@ -605,14 +603,14 @@ class StoryItemDetail(BaseModel):
language: str
audio_path: str
duration: float
seed: int | None
instruct: str | None
engine: str | None = None
seed: Optional[int]
instruct: Optional[str]
engine: Optional[str] = None
volume: float = 1.0
generation_created_at: datetime
# Versions available for this generation
versions: list["GenerationVersionResponse"] | None = None
active_version_id: str | None = None
versions: Optional[List["GenerationVersionResponse"]] = None
active_version_id: Optional[str] = None
class Config:
from_attributes = True
@@ -623,10 +621,10 @@ class StoryDetailResponse(BaseModel):
id: str
name: str
description: str | None
description: Optional[str]
created_at: datetime
updated_at: datetime
items: list[StoryItemDetail] = []
items: List[StoryItemDetail] = []
class Config:
from_attributes = True
@@ -636,8 +634,8 @@ class StoryItemCreate(BaseModel):
"""Request model for adding a generation to a story."""
generation_id: str
start_time_ms: int | None = None # If not provided, will be calculated automatically
track: int | None = 0 # Track number (0 = main track)
start_time_ms: Optional[int] = None # If not provided, will be calculated automatically
track: Optional[int] = 0 # Track number (0 = main track)
class StoryItemUpdateTime(BaseModel):
@@ -650,13 +648,13 @@ class StoryItemUpdateTime(BaseModel):
class StoryItemBatchUpdate(BaseModel):
"""Request model for batch updating story item timecodes."""
updates: list[StoryItemUpdateTime]
updates: List[StoryItemUpdateTime]
class StoryItemReorder(BaseModel):
"""Request model for reordering story items."""
generation_ids: list[str] = Field(..., min_length=1)
generation_ids: List[str] = Field(..., min_length=1)
class StoryItemMove(BaseModel):
@@ -682,7 +680,7 @@ class StoryItemSplit(BaseModel):
class StoryItemVersionUpdate(BaseModel):
"""Request model for setting a story item's pinned version."""
version_id: str | None = None # null = use generation default
version_id: Optional[str] = None # null = use generation default
class StoryItemVolumeUpdate(BaseModel):
@@ -707,23 +705,23 @@ class EffectConfig(BaseModel):
class EffectsChain(BaseModel):
"""An ordered list of effects to apply."""
effects: list[EffectConfig] = Field(default_factory=list)
effects: List[EffectConfig] = Field(default_factory=list)
class EffectPresetCreate(BaseModel):
"""Request model for creating an effect preset."""
name: str = Field(..., min_length=1, max_length=100)
description: str | None = Field(None, max_length=500)
effects_chain: list[EffectConfig]
description: Optional[str] = Field(None, max_length=500)
effects_chain: List[EffectConfig]
class EffectPresetUpdate(BaseModel):
"""Request model for updating an effect preset."""
name: str | None = Field(None, min_length=1, max_length=100)
description: str | None = None
effects_chain: list[EffectConfig] | None = None
name: Optional[str] = Field(None, min_length=1, max_length=100)
description: Optional[str] = None
effects_chain: Optional[List[EffectConfig]] = None
class EffectPresetResponse(BaseModel):
@@ -731,8 +729,8 @@ class EffectPresetResponse(BaseModel):
id: str
name: str
description: str | None = None
effects_chain: list[EffectConfig]
description: Optional[str] = None
effects_chain: List[EffectConfig]
is_builtin: bool = False
created_at: datetime
@@ -747,8 +745,8 @@ class GenerationVersionResponse(BaseModel):
generation_id: str
label: str
audio_path: str
effects_chain: list[EffectConfig] | None = None
source_version_id: str | None = None
effects_chain: Optional[List[EffectConfig]] = None
source_version_id: Optional[str] = None
is_default: bool
created_at: datetime
@@ -759,18 +757,18 @@ class GenerationVersionResponse(BaseModel):
class ApplyEffectsRequest(BaseModel):
"""Request to apply effects to an existing generation."""
effects_chain: list[EffectConfig]
source_version_id: str | None = Field(
effects_chain: List[EffectConfig]
source_version_id: Optional[str] = Field(
None, description="Version to use as source audio (defaults to clean/original)"
)
label: str | None = Field(None, max_length=100, description="Label for this version (auto-generated if omitted)")
label: Optional[str] = Field(None, max_length=100, description="Label for this version (auto-generated if omitted)")
set_as_default: bool = Field(default=True, description="Set this version as the default")
class ProfileEffectsUpdate(BaseModel):
"""Request to update the default effects chain on a profile."""
effects_chain: list[EffectConfig] | None = Field(None, description="Effects chain (null to remove)")
effects_chain: Optional[List[EffectConfig]] = Field(None, description="Effects chain (null to remove)")
class AvailableEffectParam(BaseModel):
@@ -795,7 +793,7 @@ class AvailableEffect(BaseModel):
class AvailableEffectsResponse(BaseModel):
"""Response listing all available effect types."""
effects: list[AvailableEffect]
effects: List[AvailableEffect]
# ─── Cloud (backup & sync) ──────────────────────────────────────────────
@@ -812,8 +810,8 @@ class CloudStatusResponse(BaseModel):
"""Current link between this device and a Voicebox Cloud account."""
connected: bool
device_name: str | None = None
account_user_id: str | None = None
key_prefix: str | None = None
connected_at: datetime | None = None
device_name: Optional[str] = None
account_user_id: Optional[str] = None
key_prefix: Optional[str] = None
connected_at: Optional[datetime] = None
dashboard_url: str
+11 -12
View File
@@ -44,7 +44,6 @@ def _patch_torch_from_numpy():
return
try:
import ctypes
import numpy as np
_orig = torch.from_numpy
@@ -54,17 +53,17 @@ def _patch_torch_from_numpy():
# silently corrupt data (e.g. fp16 tensors from some TTS engines),
# so we raise instead.
dtype_map = {
"float16": torch.float16,
"float32": torch.float32,
"float64": torch.float64,
"int8": torch.int8,
"int16": torch.int16,
"int32": torch.int32,
"int64": torch.int64,
"uint8": torch.uint8,
"bool": torch.bool,
"complex64": torch.complex64,
"complex128": torch.complex128,
"float16": _t.float16,
"float32": _t.float32,
"float64": _t.float64,
"int8": _t.int8,
"int16": _t.int16,
"int32": _t.int32,
"int64": _t.int64,
"uint8": _t.uint8,
"bool": _t.bool,
"complex64": _t.complex64,
"complex128": _t.complex128,
}
def _safe_from_numpy(
@@ -56,6 +56,7 @@ import sys
import tempfile
import types
# Diagnostics — log hook activity to a file alongside the bundle so we can
# see what's happening when the server is run as a sidecar (no stdout for
# runtime hook prints). Safe no-op if the file can't be written.

Some files were not shown because too many files have changed in this diff Show More