The PyTorch Whisper transcribe() called the HF processor without
truncation=False and model.generate() without return_timestamps=True.
With those defaults, WhisperFeatureExtractor silently truncates inputs
to 30 s (Whisper's native receptive field), so any dictation longer
than ~30 s lost its tail.
Setting truncation=False + padding="longest" + return_attention_mask=True
on the processor, then forwarding the attention mask plus
return_timestamps=True to generate(), flips HF Whisper into long-form
mode: autoregressive decoding over rolling 30 s windows.
Verified by round-tripping a 56.6 s Kokoro TTS sample through
/transcribe — full text returned including content past the 30 s mark;
previously the transcript was cut off roughly halfway through.
MLX backend (mlx_backend.py) is intentionally unchanged: mlx_audio.stt's
generate() already implements rolling-window long-form transcription
with condition_on_previous_text in the upstream library, so it does not
have the same bug. The HF-only kwargs added here would also break the
MLX call signature.
Co-Authored-By: Claude Opus 4.7 <[email protected]>
Registering Kokoro with needs_trim=True routed its output through the
generic trim_tts_output, whose 1s internal-silence cut was tuned for
Chatterbox hallucinations. KPipeline synthesizes newline- and
token-limit-separated segments independently, and the ~0.3s lead plus
~0.7s tail pads at each boundary add up to 1.2s of silence, so with
af_sarah a four-paragraph script came back as its first paragraph only
(10.9s -> 1.8s) and a 51s text lost half its segments.
Drop the needs_trim flag, keep the in-backend trim, and give
trim_tts_output a max_internal_silence_ms=None mode that only trims the
leading and trailing pads. Tests updated for the new behaviour, with a
two-segment fake pipeline and an explicit internal-gap case.
Wrap empty_device_cache in release_generation_memory(), which logs and
swallows cleanup failures so a poisoned CUDA context cannot replace a
finished generation's result, and call it from the streaming endpoint
too, which drives generate_chunked directly.
Initialise tts_model before the try so the finally can read its device
without a locals() probe, import empty_device_cache alongside the other
backend imports instead of inside a bare try/except, and flatten the MPS
branch in empty_device_cache.
The file still used the pre-refactor flat imports and a sys.path hack:
sys.path.insert(0, str(Path(__file__).parent.parent))
from database import Base, VoiceProfile as DBVoiceProfile
from profiles import create_profile, update_profile
`profiles` now lives at backend/services/profiles.py, so collection raised
ImportError. Because pytest aborts the whole run on a collection error, this
one file meant `just test` ran zero tests — duplicate-name validation has had
no coverage since the services refactor.
Switch to package imports like every other test module, and drop
DBVoiceProfile, which was imported but never used.
That exposed a second, latent bug: all 6 tests passed but every one errored in
teardown with PermissionError WinError 32. The fixture closed the session and
then rmtree'd the temp dir, but closing a session does not release
SQLAlchemy's pooled connection, so SQLite still held test.db open on Windows.
Dispose the engine before removing the directory.
6 passed, and full-suite collection goes from aborting to 170 tests.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Collecting backend/tests/ fails outright on a clean checkout:
backend/tests/test_profile_duplicate_names.py:19: in <module>
from database import Base, VoiceProfile as DBVoiceProfile
E ImportError: attempted relative import beyond top-level package
pytest stops at the collection error, so the whole backend suite runs zero
tests rather than the ~142 it otherwise would.
There are two causes stacked on top of each other.
First, a real cycle in production code. routes/profiles.py, routes/history.py
and routes/stories.py each import safe_content_disposition from ..app, while
app.py builds the FastAPI instance at module scope (app = create_app() on
import), which registers those same routers. Importing any of those three
route modules first therefore re-enters a partially initialised app and dies
with "cannot import name 'router' from partially initialized module". It only
works today because app.py always happens to be imported first.
safe_content_disposition is a pure helper over urllib.parse.quote with no
application state, so it moves to backend/utils/http.py. app.py re-exports it
so any external caller importing it from the old location keeps working.
Second, the test reached for modules through a sys.path hack
(sys.path.insert(parent) + "from database import ...") rather than the
"from backend.X import ..." style the rest of the suite uses. That flat import
makes database/models.py's "from ..utils.capture_chords import ..." point
outside the package. It also aimed at the wrong module: it wants the service
layer, which raises ValueError, not the route handler, which converts that
into an HTTPException.
Result: the full suite goes from 0 collected to 148 passed. The one remaining
failure, test_progress.py::test_hf_progress_tracker, is pre-existing and
unrelated (tqdm patching) - it reproduces identically on an unpatched tree.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Update the README, MCP docs, in-app MCP page, i18n strings, landing copy,
docstrings and comments to match the renamed tools. CHANGELOG entries and
docs/plans are historical and left as they were.
Claude Desktop validates tool names against ^[a-zA-Z0-9_-]{1,64}$ and
rejects the whole tool list when any name contains a dot, so the Voicebox
MCP server was unusable there. Rename voicebox.speak, voicebox.transcribe,
voicebox.list_captures and voicebox.list_profiles to voicebox_speak,
voicebox_transcribe, voicebox_list_captures and voicebox_list_profiles.
Fixes#790
Queries throughout the codebase filter on generations.profile_id,
generations.status, generations.created_at, story_items.story_id,
story_items.generation_id, generation_versions.generation_id, and
profile_samples.profile_id with every request. Without indexes SQLite
falls back to a full table scan; as history grows (hundreds or thousands
of generations) these scans become the dominant latency.
Changes:
- Add index=True on the most-queried FK and sort columns in models.py so
new installs get them from Base.metadata.create_all
- Add _migrate_add_indexes() called from run_migrations() so existing
installs get the same indexes on next startup (uses CREATE INDEX IF
NOT EXISTS — idempotent, <10 ms on any realistic dataset)
Switch from the default DELETE/ROLLBACK journal to WAL so concurrent
readers (SSE status polls, history queries) are not blocked while the
generation worker holds a write transaction. Set a 5-second busy
timeout to eliminate "database is locked" errors under brief write
contention.
Both PRAGMAs are applied via a custom creator function so every
connection in the pool gets the settings at open time, not just the
first one.
is_model_cached() marked a model as not-cached whenever any .incomplete
blob existed in its cache dir, even when a completed blob with the same
hash already sat next to it. A retried/concurrent download can leave
this orphan behind after the real transfer already finished, which made
the model appear perpetually "downloading" and re-trigger a full
re-download on every load.
Only .incomplete files with no matching completed blob now count as a
genuinely in-progress download.
Both speak surfaces built their GenerationRequest with a hardcoded "en"
fallback and never consulted the resolved profile, so a profile created
with language="fr" was still synthesised as English unless the caller
passed language= explicitly.
This hurts the MCP path most: an agent calling voicebox.speak has no way
to know the bound profile's language, so it cannot pass the argument
either. Every agent-triggered generation on a non-English profile came
out with an English accent.
The fallback chain is now explicit argument -> resolved profile's
language -> "en", which matches how engine and personality already
consult the resolved binding. The "en" backstop is kept so profiles with
no language set behave exactly as before.
Adds backend/tests/test_speak_language.py covering both surfaces: the
fallback, explicit-argument precedence, and the unchanged "en" default.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Without a trace the Settings GPU label and /health keep reporting CUDA
while generation runs on CPU, so the user has no way to confirm the
override engaged.
The override is documented in gpu-acceleration.mdx and listed as step 1 of
the get_torch_device() precedence in tts-generation.mdx, but grepping the
tree for VOICEBOX_FORCE_CPU matched only those two doc files - nothing read
it. Users whose GPU has no compiled kernels in the bundled PyTorch had no
way to fall back to CPU short of renaming the installed CUDA backend
directory.
Resolve it before torch is imported, so it still works when the installed
build is itself the reason CPU is wanted.
list_generations() fetched versions with one SELECT per generation on the
page (50 rows -> 51 queries). Add _get_versions_for_generations() which
loads all versions for the page in a single WHERE generation_id IN (...)
query and groups them in memory; the single-generation helper now
delegates to it so story item details behave identically.
Generated with Codebuff 🤖
Co-Authored-By: Codebuff <[email protected]>
When the server outlives the Tauri app that spawned it (keep-running
mode, or a sidecar the next launch reuses), its stdout/stderr pipe has
no reader. Every later print()/tqdm write raises BrokenPipeError, so
POST /captures and /transcribe return "[Errno 32] Broken pipe".
Wrap stdout/stderr so they fall back to devnull on the first failed
write instead of raising.
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Export filenames were derived from only the first 30 characters of the
generation text. Generations with similar wording (a common workflow when
iterating on the same line) produced identical filenames, so exports
collided on disk — the browser appended " (1)"/" (2)" and users ended up
opening audio that didn't match the expected filename.
Append the first 8 chars of the generation id to the .wav and .voicebox.zip
export filenames, in both the backend Content-Disposition headers and the
frontend save-file hooks.
Co-authored-by: Claude Opus 4.8 <[email protected]>
`UploadFile.filename` can be None, and `Path(None)` raises TypeError. On the
avatar endpoint this happens before the try/except, so a filename-less upload
surfaces as an unhandled 500 instead of a clean response. Every other upload
handler already guards this with `file.filename or ""` (add_profile_sample,
transcription, generations); apply the same guard here.
Co-authored-by: Claude Opus 4.8 (1M context) <[email protected]>
The MCP voicebox.speak tool built its GenerationRequest without a
model_size, so every agent-triggered generation fell back to the schema
default ("1.7B"). There was no way to reach the 0.6B Qwen variant (or
TADA's 1B/3B) through MCP, and callers paid a model reload whenever the
requested size differed from what was already loaded.
Thread an optional model_size through voicebox.speak and the _speak
helper into GenerationRequest, mirroring the REST /generate surface.
Omitting it passes None, which generate_speech normalizes to the engine
default, so existing callers are unaffected.
Add backend/tests/test_mcp_speak.py covering the forwarded value, the
omitted-default path, and rejection of an invalid size.
Fixes#884
The /transcribe endpoint passed the raw uploaded file straight to the STT
backend (mlx_audio.stt -> miniaudio), which only decodes WAV/FLAC/MP3/Vorbis.
Browser recordings arrive as WebM/Opus (Chrome/Firefox MediaRecorder), so
web-mode dictation failed with 500 "unsupported file format". The Tauri app
was unaffected because WebKit produces MP4.
librosa already fully decodes the upload to compute duration (falling back to
audioread/ffmpeg for exotic containers), so re-encode that PCM to a temp WAV
and hand it to Whisper. WAV inputs pass through unchanged; the temp file is
cleaned up in the finally block.
Co-authored-by: Claude Opus 4.8 <[email protected]>
Encoder.eval() alone still builds an autograd graph because parameters
require grad by default. On 8GB GPUs that ballooned TADA encode VRAM far
past the model footprint (issue 890). Wrap the encode forward in
inference_mode and add a unit test that asserts the flag is set.
Co-authored-by: fooSynaptic <[email protected]>
* Fix voice sample validation on Python 3.13
Python 3.13 removed audioop from the standard library, which broke reference
audio validation when adding voice samples. Add the audioop-lts backport for
3.13+ installs and bundle audioop in PyInstaller builds on the same versions.
* style(tests): satisfy Ruff import ordering
---------
Co-authored-by: Jamie Pine <[email protected]>
* fix(backend): return 404 instead of 500 for audio of failed generations
A failed generation stores an empty audio_path. resolve_storage_path("")
resolved to the data directory itself, which exists, so the route's 404
guard passed and FileResponse raised RuntimeError ("File at path .../data
is not a file"), surfacing as a 500.
- resolve_storage_path now returns None for empty paths
- audio routes check is_file() instead of exists() so directories never
reach FileResponse
- GET /audio/{generation_id} reports "Generation failed; no audio
available" when the generation status is failed
Co-Authored-By: Claude Fable 5 <[email protected]>
* fix(backend): reject empty Path objects in resolve_storage_path
Path("") is truthy, so the previous `if not path` guard only caught
None and empty strings. Callers such as database/migrations.py pass
Path objects, so an empty Path could still resolve to the data dir.
Check None separately and reject paths with no parts.
Also add regression tests asserting the version and sample audio
endpoints 404 when a stored path resolves to an existing directory
(guards the is_file() checks against regressing to exists()).
Addresses CodeRabbit review on PR #893.
Co-Authored-By: Claude Fable 5 <[email protected]>
* style(tests): drop parentheses on pytest.fixture decorator (ruff PT001)
Co-Authored-By: Claude Fable 5 <[email protected]>
* style(tests): satisfy Ruff naming rule
---------
Co-authored-by: Claude Fable 5 <[email protected]>
Co-authored-by: Jamie Pine <[email protected]>
* fix(setup): install mlx-lm and mlx-audio in setup-python on Apple Silicon
The dev setup installed requirements-mlx.txt but not mlx-audio/mlx-lm
themselves, so POST /transcribe failed on a fresh Apple Silicon setup
with "No module named 'mlx_audio'" (then "No module named 'mlx_lm'").
The release workflow already installs both with --no-deps (they declare
transformers>=5.x, conflicting with our <=4.57.x cap); mirror that in
the setup-python recipe with the same pins.
Co-Authored-By: Claude Fable 5 <[email protected]>
* test: add MLX smoke test for the --no-deps mlx-audio/mlx-lm install
mlx-audio and mlx-lm are installed --no-deps, so a missing transitive
dependency only surfaces at import time. Add a pytest-discoverable
smoke test (skipped off Apple Silicon) covering the exact entry points
the backend uses: mlx_audio.tts.load, mlx_audio.stt.load (which also
exercises the miniaudio dep from issue #505), mlx_lm.load/generate,
and a basic mlx.core op.
Co-Authored-By: Claude Fable 5 <[email protected]>
---------
Co-authored-by: Claude Fable 5 <[email protected]>
Docker compose sets HSA_OVERRIDE_GFX_VERSION=${HSA_OVERRIDE_GFX_VERSION:-}
which results in an empty string when not provided. An empty string is
not the same as unset - ROCm treats it as 'force-empty' and no GPU is
detected, even natively supported ones (e.g. gfx1201 / RX 9070 on ROCm 7.2).
Pop the env var when it is empty, before torch loads, so ROCm auto-detects
the GPU correctly.
Tested on RX 9070 (gfx1201) with ROCm 7.2 and PyTorch 2.12.1+rocm7.2.
list_stories() previously executed one COUNT(story_items) query per
story in a Python loop. With N stories that is N+1 round-trips to
SQLite regardless of list length. Replace with a single aggregated
GROUP BY query that fetches all counts at once, then populate each
StoryResponse from a dict lookup.
force_offline_if_cached flips HF_HUB_OFFLINE (env + huggingface_hub
constant + transformers._is_offline_mode) process-wide for the duration
of a cached LLM load, silently switching every concurrent model
download/load on other threads to offline mode. With default capture
settings (whisper-turbo STT + Qwen3 refinement + auto_refine) a first
run downloads several models concurrently, and a poisoned fetch
surfaces as "Can't load feature extractor..." (whisper) or
"Unrecognized model ... model_type" (Qwen3) rather than anything
mentioning offline mode.
These are the last two call sites of the guard — the same pattern was
deliberately removed app-wide in #524/#530 after identical failures,
and the 0.5.0 LLM backend reintroduced it. LLM loads now run with the
process's default HF_HUB_OFFLINE state, matching every other backend
(issue #462 precedent).
Fixes#841
Claude-Session: https://claude.ai/code/session_011iwL9AyeAWgz2jpgcHxJpC
Co-authored-by: Claude Fable 5 <[email protected]>
TaskManager.error_download() intentionally keeps a failed task in the
active list (status="error") so /tasks/active can surface the error
and retry UI — but /models/status derived its "downloading" flag from
the same unfiltered list. One failed download therefore showed the
model as downloading:true / downloaded:false for the life of the
process, masking the model's real cache state (even a fully valid
on-disk cache) until an app restart. Likely behind endless-spinner
reports like #181 and the restart-fixes-it pattern in #883.
Add TaskManager.get_pending_downloads() (downloading/extracting only)
and use it in /models/status; /tasks/active behavior is unchanged.
Fixes#925
Claude-Session: https://claude.ai/code/session_011iwL9AyeAWgz2jpgcHxJpC
Co-authored-by: Claude Fable 5 <[email protected]>
/transcribe wrote every upload to a temp file named .wav regardless of its
real format. librosa picks its decoder from the extension, so any non-wav
upload failed with "could not open/decode file" even though the format is
one the app handles elsewhere.
profiles.py already solves this for voice samples by keeping the uploaded
extension when it is one of the audio types it accepts, and falling back to
.wav otherwise. Same approach here, same set. The fallback means an unknown
or missing extension behaves exactly as it does today.
* Add "Log in with browser" cloud device login
Connects the desktop app to Voicebox Cloud without the user ever handling an
API key. One button in Settings → General opens the system browser to
voicebox.sh, the user authorizes while signed in, and the credential lands
back in the app automatically.
Backend (FastAPI):
- /cloud/login/start opens the browser to the cloud authorize page with a
state we mint; the existing loopback server catches the redirect at
/cloud/callback and exchanges the one-time code (server-to-server, over TLS)
for a voicebox_ API key, verifies it against the API, and stores it.
- /cloud/status and /cloud/disconnect back the settings UI.
- state round-trip guards against login-CSRF; the key never crosses a browser
URL and is never exposed to the frontend (status returns a prefix only).
- CloudSettings singleton row; config gains VOICEBOX_CLOUD_URL /
VOICEBOX_CLOUD_API_URL (default the prod hosts, overridable for dev).
Frontend (React):
- CloudSection in Settings → General: "Log in with browser", polls status,
shows the connected device + a dashboard link. API keys are the advanced
path only, surfaced in the web dashboard.
The key is stored in the local app DB for now; OS keychain is a marked
follow-up.
* Address review feedback on cloud login
- time out status polling after 2 min so an abandoned browser flow
doesn't leave the button stuck on "Waiting for browser…"
- handle non-JSON / non-object payloads from the exchange and account
endpoints instead of 500ing after the state is consumed
- make singleton row creation race-safe (IntegrityError -> re-query)
- clear device_name on disconnect along with the rest of the metadata
- serve the dashboard URL from /cloud/status so the Manage link follows
VOICEBOX_CLOUD_URL instead of hardcoding production
- keep a "Disconnecting…" label on the disconnect button while pending
* Remove orphaned react-qr-code entries from lockfile
bun.lock was out of date with package.json (react-qr-code was removed
without reinstalling), failing the frozen-lockfile install in CI.
* feat(windows): add native ROCm support for AMD GPUs
Implements native ROCm architecture for Windows.
- Adds backend build pipeline for voicebox-server-rocm.exe
- Detects AMD GPUs dynamically and routes PyTorch allocations
- Adds automatic download and update logic for ROCm dependencies
- Refactors UI in GpuPage.tsx and GpuAcceleration.tsx to add AMD flows
- Fixes 'Switch to CPU' lock on Windows via Tauri backend_override state
- Resolves PyInstaller/rocm_sdk UnboundLocalError silent crashes
- Resolves Numba/NumPy 2.x incompatibilities during Qwen3-TTS load
- Resolves HF_HUB_OFFLINE Catch-22 for CustomVoice processor caching
* fix(rocm): host libs archive under the app release tag, drop offline-load regression
Align the ROCm libs download with the CUDA pattern: both the server core and
the libs archive are published under the app-version release tag, with the libs
content version encoded in the filename only. The previous code fetched libs
from a separate rocm7.2-v1 tag, which disagreed with the download test.
Also revert the unrelated Qwen CustomVoice changes that wrapped model loading in
force_offline_if_cached (not imported — a NameError on load for every platform)
and re-added a Base-model cache gate. The inference-path offline guard was
deliberately removed previously.
* feat(rocm): gate download on AMD detection and persist the backend variant
The ROCm download section now only shows when the backend reports an AMD GPU on
Windows (new supports_rocm health field, backed by the memoized
is_amd_gpu_windows detection that was previously unused), or when ROCm is already
downloaded/active.
Make the backend override honor a pinned variant: set_backend_override persists
the choice to disk so it survives an app restart, start_server reads it back,
and a cuda/rocm pin now actually selects that variant instead of always
preferring ROCm. A stale pin to a deleted backend self-heals to the default
order rather than forcing CPU. Add the web no-op stub for the new method.
* chore(rocm): drop incomplete vitest harness for the unused GpuAcceleration component
GpuAcceleration.tsx is not routed anywhere (GpuPage is the live settings view),
and the added vitest setup referenced testing-library/vitest deps that were not
in the lockfile, breaking the web typecheck. Remove the dead component's test
and its scaffolding to keep this PR scoped to the ROCm feature.
* ci(rocm): add ROCm release-artifact pipeline
Mirror the CUDA packaging path for ROCm so the runtime download has artifacts to
fetch. scripts/package_rocm.py splits the PyInstaller --rocm onedir into
voicebox-server-rocm.tar.gz (core) + rocm-libs-rocm7.2-v1.tar.gz (AMD runtime:
HIP DLLs, rocBLAS Tensile data, MIOpen kernel DBs) + rocm-libs.json, matching
the names services/rocm.py expects, both under the app-version release tag.
The new build-rocm-windows job in release.yml builds on windows-latest/cp312 and
lets build_binary.py --rocm pull the official AMD Radeon wheels.
The file classifier can't be validated against a real AMD build on CI, so it has
unit coverage (test_package_rocm.py) against a synthetic onedir layout. The
prefixes/dir markers may need a tweak after the first real build on AMD
hardware — the packager hard-fails loudly if it classifies zero ROCm files.
---------
Co-authored-by: Jamie Pine <[email protected]>
Previously, HSA_OVERRIDE_GFX_VERSION=10.3.0 was unconditionally set for
all AMD GPUs, which caused suboptimal performance on RDNA 3/4 GPUs
(gfx11xx/gfx12xx) that have native ROCm support.
Now uses rocminfo to detect all GPUs and only sets the override for
systems where the oldest GPU needs it (RDNA 2 and older, gfx10xx and
below). Newer GPUs are left untouched.
Addresses CodeRabbit review:
- Case-insensitive regex matching on lowercased line
- Log level changed to INFO for rocminfo failures
- Multi-GPU support: iterates all GPUs, uses oldest for decision
Fixes#469
Signed-off-by: Amitesh Gupta
Signed-off-by: Amitesh Gupta
Signed-off-by: singlaamitesh <[email protected]>
