Compare commits

..
Author SHA1 Message Date
James PineandClaude Opus 4.7 791c0509a9 fix(backend): pin miniaudio in requirements-mlx.txt (#505)
mlx-audio's STT path imports miniaudio, but we install mlx-audio
--no-deps to dodge its transformers>=5.x pin. Nothing else pulls
miniaudio transitively, so fresh Apple Silicon installs fail to
transcribe with ModuleNotFoundError: miniaudio. Listed explicitly
and updated the stale comments in requirements-mlx.txt and
release.yml that claimed it came from other engines.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
2026-04-20 01:18:51 -07:00
5aa1677a25 fix(offline): guard inference paths with HF_HUB_OFFLINE (#503)
* fix(offline): guard inference paths with HF_HUB_OFFLINE (#462)

PR #443 wrapped the model *load* path with `force_offline_if_cached` so
cached models don't phone home at startup. The context manager restores
`HF_HUB_OFFLINE` on exit, which left inference paths (generate,
transcribe, voice-prompt creation) unguarded — and `qwen_tts`,
`mlx_audio`, and `transformers` perform lazy tokenizer/processor/config
lookups during inference. With internet on, those lookups are
near-instant and invisible; with internet off, `requests` hangs on DNS
or connect until the network returns. This is exactly what users in
#462 describe: model shows "Loaded", internet drops, generation
"thinks" forever, internet comes back, generation completes.

Chatterbox and LuxTTS don't exhibit this because their engine libs
resolve everything through already-cached paths at load time.

Fix: wrap each inference-sync body with `force_offline_if_cached(True,
...)`. Since inference only runs after a successful load, weights are
known to be on disk, so `is_cached=True` is unconditional.

Also adds the load-time guard that was missing from
`qwen_custom_voice_backend.py` — CustomVoice previously had no offline
protection at all.

Paths patched:
  - PyTorchTTSBackend.create_voice_prompt (create_voice_clone_prompt)
  - PyTorchTTSBackend.generate (generate_voice_clone)
  - PyTorchSTTBackend.transcribe (Whisper generate + decoder-prompt-ids)
  - MLXTTSBackend.generate (mlx_audio generate, all branches)
  - MLXSTTBackend.transcribe (mlx_audio whisper generate)
  - QwenCustomVoiceBackend._load_model_sync + generate

Does not address the secondary `check_model_inputs() missing 'func'`
error reported in the same issue — that's a `transformers` 5.x
version-skew bug on the install path, separate concern.

Fixes #462.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

* fix(offline): mutate cached HF constants + threadsafe refcount

Review feedback on the initial fix surfaced two real issues:

1. ``os.environ`` toggles alone don't flip offline mode.
   ``huggingface_hub.constants.HF_HUB_OFFLINE`` is read once at import
   time into a module-level bool; ``transformers.utils.hub._is_offline_mode``
   mirrors that bool at its own import time. The hot paths
   (``_http._default_backend_factory`` in huggingface_hub,
   ``is_offline_mode`` in transformers) read the cached bools — not the
   env — so mutating only ``os.environ`` was a no-op.

2. Race condition on concurrent inference. Two threads running inside
   ``force_offline_if_cached`` via ``asyncio.to_thread`` could have
   thread A's ``finally`` strip thread B's offline protection mid-run.

Rewrite the helper to:
  - mutate ``huggingface_hub.constants.HF_HUB_OFFLINE`` and
    ``transformers.utils.hub._is_offline_mode`` directly
  - refcount concurrent users under a single ``threading.RLock`` so a
    shared offline window is restored only when the last caller exits
  - still write ``os.environ`` for anything that reads it dynamically

Also addresses the unused-variable ruff flag on the Whisper transcribe
path (``audio, sr`` → ``audio, _sr``).

New unit tests cover the cached-constant mutation, env propagation,
no-op on ``is_cached=False``, nested contexts, and a threaded race
where a slow thread must retain offline mode after a peer exits.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

* fix(offline): atomic entry rollback + tidy test assertions

Review follow-up:

- Wrap the `_offline_refcount == 0` setup in a try/except so any failure
  during the cached-constant mutation (including unexpected non-ImportError
  like RuntimeError or AttributeError from a half-initialized module)
  rolls back *all* partial state before re-raising. Without this, a
  mid-setup crash could leave `huggingface_hub.constants.HF_HUB_OFFLINE`
  mutated but the refcount at 0 — a persistent offline flag outliving
  the process.
- Swap ruff-flagged Yoda comparisons in the new test file (SIM300) and
  add a module-level note warning that these tests mutate global state
  and are not safe under cross-process parallelism.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

* test(offline): make concurrency test deterministic and bounded

Replace the `sleep(0.15)` ordering hack with an explicit `threading.Event`
the fast thread sets in `finally`. The slow thread waits on that event
(bounded), then observes the flag — so we deterministically verify the
slow thread still sees offline mode after the fast thread has exited.

Also add timeouts to `barrier.wait()` and assert `not thread.is_alive()`
after the joins so the test can't hang on an unexpected failure path.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <[email protected]>
2026-04-19 19:27:42 -07:00
5964af5dea feat(ci): re-enable Linux release builds (CPU, deb+rpm) (#488)
* feat(ci): re-enable Linux release builds (CPU, deb+rpm)

Linux shipped briefly in March 2026 (b580189..103e98b) then was removed
with the message "github runners suck." The post-mortem: standard
ubuntu-22.04 hit disk-pressure during pip+PyInstaller, and a namespace
custom-runner attempt proved flaky. Nothing in the app itself was the
problem — the NVIDIA-package exclusions in build_binary.py (~3 GB
shaved from CPU builds), the CPU-only torch install order, the
PulseAudio/PipeWire audio capture code, and the Tauri deb/rpm bundle
targets all still work.

This restores ubuntu-22.04 to the release matrix as a CPU-only Linux
build with a disk-space cleanup step (jlumbroso/free-disk-space) to
address the root cause of the March failures. Ships .deb + .rpm only
— AppImage was explicitly dropped in e18757b due to glibc portability
issues, keeping that decision.

CUDA-for-Linux is intentionally deferred to a follow-up PR: the
GpuAcceleration.tsx frontend hard-codes CUDA download as Tauri-only
without a Linux branch, and AMD users already get ROCm acceleration
for free on a stock CPU-torch install (backend/app.py:148-160). The
NVIDIA-on-Linux case is the only remaining gap and is non-blocking
for a v1 Linux ship.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

* chore(release): bump version 0.4.1 → 0.4.2

tauri-action uses tauri.conf.json's version to name the release, so
pushing v0.4.2 as a tag alone was insufficient — the workflow was still
trying to publish to v0.4.1 (immutable) and failing. Bumps all workspace
package.json files, Cargo.toml, Cargo.lock, and tauri.conf.json.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

* ci(release): add Linux bundle-step watchdog + verbose cargo logging

The first v0.4.2 attempt hung inside tauri-action at the bundling stage
on ubuntu-22.04 for ~28 min with no output. Same failure mode that drove
the March 2026 removal (commit 103e98b). Without visibility we're
guessing — probable causes include linuxdeploy/AppImage download stalls
(despite --bundles deb,rpm), cargo link on a cold cache, or disk
pressure during the final link step.

- timeout-minutes: 30 on tauri-action for Ubuntu (45 min elsewhere) —
  fail fast with logs instead of waiting out the 6hr job timeout.
- --verbose added to the Linux build args so cargo streams progress.
- CARGO_TERM_VERBOSE + RUST_BACKTRACE=1 exported on tauri-action.
- New 'Disk / environment snapshot' step dumps df/free/tool versions
  right before the tauri step so we can correlate with any later
  OOM/ENOSPC failure.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

* ci(release): disable Tauri updater artifacts on Linux (stops linuxdeploy hang)

Round-2 diagnosis from the v0.4.2 attempt: cargo finished in 3m 55s,
.deb and .rpm were bundled within 22s, then the step hung silently for
25 min until our 30-min watchdog killed it — no further log lines.

tauri.conf.json has `createUpdaterArtifacts: "v1Compatible"`. On Linux
the v1-compatible updater path wants a .AppImage.tar.gz, which means
downloading linuxdeploy-x86_64.AppImage from GitHub at build time.
That download is the silent blocker — same signature as the March 2026
"github runners suck" removal (commit 103e98b).

Fix: pass `--config {"bundle":{"createUpdaterArtifacts":false}}` on the
Linux build only. Mac/Windows continue to produce signed updater
artifacts as before. Linux users update via apt/dnf; Tauri in-app
auto-update for Linux can come later (and would require shipping
AppImage alongside deb/rpm).

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

* ci(release): pin free-disk-space action + stop removing LLVM

Review feedback on the Linux release workflow:

1. `jlumbroso/free-disk-space@main` was an unpinned ref in a job that
   runs with `contents: write` and handles signing secrets. Pin to the
   v1.3.1 commit SHA so a force-push or repo compromise can't inject
   arbitrary code into our release flow.

2. `large-packages: true` runs `apt-get remove '^llvm-.*'`, wiping LLVM
   just before the next step installs `llvm-dev`. That wastes CI time
   and risks cascade-removal of reverse deps that won't be pulled back
   in by `llvm-dev` alone. The remaining toggles (android, dotnet,
   haskell, swap-storage) already clear ~20 GB, which is enough
   headroom for the Python + torch + PyInstaller build.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <[email protected]>
2026-04-19 19:22:21 -07:00
115de231d0 fix(audio): preprocess reference samples instead of rejecting them (#502)
* fix(audio): preprocess reference samples instead of rejecting them

Uploaded/recorded voice samples were rejected outright whenever the peak
exceeded 0.99 ("Audio is clipping (reduce input gain)"). That wasn't
actionable: a recording in the app has no pre-gain control, and an
already-captured file can't be re-taken by the user. The Settings
"Normalize audio" toggle only affects generated TTS output, so users who
enabled it expecting it to help with sample uploads were still blocked.

Replace the hard reject with a small, always-on preprocess step that
runs right after load:
  - DC-offset removal
  - Conservative edge-silence trim (top_db=30) with 100 ms padding kept
  - Peak cap at 0.95 if the input peak exceeds that

Duration and RMS checks now run on the preprocessed waveform, so
samples that were previously rejected for being "hot" are accepted and
stored with safe headroom. True in-waveform clipping artifacts still
can't be repaired — peak scaling only prevents downstream re-clipping
during multi-sample combination and TTS inference.

Adds a unit-test file (previously none existed for audio.py).

Fixes #456.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

* fix(audio): raise trim threshold, cap pad at net-neutral

Review feedback on the preprocessor:

1. ``trim_top_db=30`` was labelled "conservative" in the docstring but is
   actually *more* aggressive than librosa's default of 60. Normal
   speech dynamic range sits around 30 dB, so 30 dB would eat quiet
   trailing syllables and soft consonants. Raise the default to 40 dB —
   below normal speech dynamic range but still catching obvious edge
   silence — and fix the docstring.

2. Unconditional 100 ms edge padding ran even when ``librosa.effects.trim``
   removed nothing. For a well-recorded 29.9 s upload that path would
   push the waveform past the 30 s ceiling and trigger a spurious "too
   long" rejection. Only pad when trimming actually shortened the
   audio, and cap the pad so the output never exceeds the input length.

Adds a regression test for the net-neutral length behaviour.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <[email protected]>
2026-04-19 19:22:18 -07:00
8929947c7a fix(mlx): point Qwen 0.6B at the published mlx-community repo (#501)
The 0.6B slot was aliased to the 1.7B repo as a temporary fallback
because `mlx-community/Qwen3-TTS-12Hz-0.6B-Base-bf16` wasn't published
when MLX support shipped. That conversion is live now, so use it —
Apple Silicon users picking 0.6B get the actual 0.6B model (1.2 GB
instead of 3.5 GB).

Also drops the now-obsolete troubleshooting entry and updates the
triage notes in PROJECT_STATUS.md.

Fixes #485.

Co-authored-by: Claude Opus 4.7 (1M context) <[email protected]>
2026-04-19 19:22:15 -07:00
19 changed files with 423 additions and 140 deletions
+61 -2
View File
@@ -26,12 +26,45 @@ jobs:
args: ""
python-version: "3.12"
backend: "pytorch"
- platform: "ubuntu-22.04"
# --config override disables updater-artifact generation on Linux.
# tauri.conf.json has createUpdaterArtifacts: "v1Compatible" which
# on Linux wants to synthesize a .AppImage.tar.gz by downloading
# linuxdeploy at build time — this is what silently hangs CI
# (see v0.4.2 round 2, 25 min of no output after rpm bundling).
# We ship deb+rpm only; Linux users update via apt/dnf, not the
# Tauri in-app updater.
args: '--target x86_64-unknown-linux-gnu --bundles deb,rpm --verbose --config {"bundle":{"createUpdaterArtifacts":false}}'
python-version: "3.12"
backend: "pytorch"
runs-on: ${{ matrix.platform }}
steps:
- uses: actions/checkout@v4
# Ubuntu runners ship with ~14 GB free; pip + PyInstaller + torch can
# peak well above that during the build. Reclaim ~25 GB by pruning
# preinstalled toolchains we don't use. This is what likely tripped
# the March 2026 Linux release attempts (see commit 103e98b
# "github runners suck") — not a code issue, a disk-pressure one.
- name: Free up disk space (ubuntu)
if: contains(matrix.platform, 'ubuntu') || contains(matrix.platform, 'namespace')
# Pinned to v1.3.1 (SHA) — this job runs with contents: write and
# handles signing secrets later, so we don't want a floating ref.
uses: jlumbroso/free-disk-space@54081f138730dfa15788a46383842cd2f914a1be
with:
tool-cache: false
android: true
dotnet: true
haskell: true
# large-packages: true would `apt-get remove '^llvm-.*'`, which
# cascade-removes reverse deps that won't be pulled back in by the
# `llvm-dev` install below. The other flags already free ~20 GB,
# enough for the Python + torch + PyInstaller build.
large-packages: false
swap-storage: true
- name: Install dependencies (ubuntu only)
if: contains(matrix.platform, 'ubuntu') || contains(matrix.platform, 'namespace')
run: |
@@ -73,8 +106,10 @@ jobs:
# fine on transformers 4.57.x in practice (verified in dev), so install
# them --no-deps. mlx-audio's other runtime deps (huggingface_hub,
# librosa, numpy, numba, pyloudnorm) are already in requirements.txt;
# the rest (sounddevice, miniaudio, protobuf, sentencepiece, pyyaml,
# jinja2) are pulled in by other engines.
# miniaudio is in requirements-mlx.txt (needed by mlx_audio.stt,
# not transitively pulled by anything else — see issue #505); the
# rest (sounddevice, protobuf, sentencepiece, pyyaml, jinja2) are
# pulled in by other engines.
pip install --no-deps mlx-lm==0.31.1
pip install --no-deps mlx-audio==0.4.1
@@ -133,6 +168,21 @@ jobs:
p12-file-base64: ${{ secrets.APPLE_CERTIFICATE }}
p12-password: ${{ secrets.APPLE_CERTIFICATE_PASSWORD }}
- name: Disk / environment snapshot (pre-bundle debug)
if: contains(matrix.platform, 'ubuntu') || contains(matrix.platform, 'namespace')
run: |
echo "=== df -h ==="
df -h
echo "=== free -h ==="
free -h
echo "=== Rust / Cargo ==="
rustc --version
cargo --version
echo "=== Bun ==="
bun --version
echo "=== Tauri CLI ==="
cd tauri && bun run tauri --version
- name: Extract release notes from CHANGELOG.md
id: changelog
shell: bash
@@ -156,7 +206,13 @@ jobs:
echo "CHANGELOG_EOF"
} >> "$GITHUB_OUTPUT"
# Linux hang watchdog: previous releases silently wedged inside tauri
# bundling (possibly linuxdeploy/AppImage download, possibly cargo link).
# Cap the step at 30 min so we get logs instead of waiting out the 6hr
# job timeout. Other platforms historically complete in ~25 min, so 45
# is comfortable.
- uses: tauri-apps/[email protected]
timeout-minutes: ${{ (contains(matrix.platform, 'ubuntu') || contains(matrix.platform, 'namespace')) && 30 || 45 }}
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
TAURI_SIGNING_PRIVATE_KEY: ${{ secrets.TAURI_SIGNING_PRIVATE_KEY }}
@@ -168,6 +224,9 @@ jobs:
APPLE_PROVIDER_SHORT_NAME: ${{ secrets.APPLE_PROVIDER_SHORT_NAME }}
APPLE_API_ISSUER: ${{ secrets.APPLE_API_ISSUER }}
APPLE_API_KEY: ${{ secrets.APPLE_API_KEY }}
# Stream subprocess stdout/stderr so the hang is visible in logs.
CARGO_TERM_VERBOSE: "true"
RUST_BACKTRACE: "1"
with:
projectPath: tauri
tagName: v__VERSION__
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@voicebox/app",
"version": "0.4.1",
"version": "0.4.2",
"private": true,
"type": "module",
"scripts": {
+1 -1
View File
@@ -177,7 +177,7 @@ def _get_qwen_model_configs() -> list[ModelConfig]:
backend_type = get_backend_type()
if backend_type == "mlx":
repo_1_7b = "mlx-community/Qwen3-TTS-12Hz-1.7B-Base-bf16"
repo_0_6b = "mlx-community/Qwen3-TTS-12Hz-1.7B-Base-bf16" # 0.6B not available in MLX, falls back
repo_0_6b = "mlx-community/Qwen3-TTS-12Hz-0.6B-Base-bf16"
else:
repo_1_7b = "Qwen/Qwen3-TTS-12Hz-1.7B-Base"
repo_0_6b = "Qwen/Qwen3-TTS-12Hz-0.6B-Base"
+36 -26
View File
@@ -45,11 +45,9 @@ class MLXTTSBackend:
Returns:
HuggingFace Hub model ID for MLX
"""
# MLX model mapping
mlx_model_map = {
"1.7B": "mlx-community/Qwen3-TTS-12Hz-1.7B-Base-bf16",
# 0.6B not yet converted to MLX format
"0.6B": "mlx-community/Qwen3-TTS-12Hz-1.7B-Base-bf16", # Fallback to 1.7B
"0.6B": "mlx-community/Qwen3-TTS-12Hz-0.6B-Base-bf16",
}
if model_size not in mlx_model_map:
@@ -195,6 +193,8 @@ class MLXTTSBackend:
logger.info("Generating audio for text: %s", text)
model_name = f"qwen-tts-{self._current_model_size}"
def _generate_sync():
"""Run synchronous generation in thread pool."""
# MLX generate() returns a generator yielding GenerationResult objects
@@ -220,36 +220,40 @@ class MLXTTSBackend:
logger.warning("Regenerating without voice prompt.")
ref_audio = None
# Check if model supports voice cloning via generate method
# MLX API may support ref_audio parameter directly
try:
# Try with voice cloning parameters if supported
if ref_audio:
# Check if generate accepts ref_audio parameter
import inspect
# Model is loaded → weights are on disk. Force offline so
# lazy tokenizer/config lookups inside mlx_audio don't hang
# when the user is disconnected (issue #462).
with force_offline_if_cached(True, model_name):
# Check if model supports voice cloning via generate method
# MLX API may support ref_audio parameter directly
try:
# Try with voice cloning parameters if supported
if ref_audio:
# Check if generate accepts ref_audio parameter
import inspect
sig = inspect.signature(self.model.generate)
if "ref_audio" in sig.parameters:
# Generate with voice cloning
for result in self.model.generate(text, ref_audio=ref_audio, ref_text=ref_text, lang_code=lang):
audio_chunks.append(np.array(result.audio))
sample_rate = result.sample_rate
sig = inspect.signature(self.model.generate)
if "ref_audio" in sig.parameters:
# Generate with voice cloning
for result in self.model.generate(text, ref_audio=ref_audio, ref_text=ref_text, lang_code=lang):
audio_chunks.append(np.array(result.audio))
sample_rate = result.sample_rate
else:
# Fallback: generate without voice cloning
for result in self.model.generate(text, lang_code=lang):
audio_chunks.append(np.array(result.audio))
sample_rate = result.sample_rate
else:
# Fallback: generate without voice cloning
# No voice prompt, generate normally
for result in self.model.generate(text, lang_code=lang):
audio_chunks.append(np.array(result.audio))
sample_rate = result.sample_rate
else:
# No voice prompt, generate normally
except Exception as e:
# If voice cloning fails, try without it
logger.warning("Voice cloning failed, generating without voice prompt: %s", e)
for result in self.model.generate(text, lang_code=lang):
audio_chunks.append(np.array(result.audio))
sample_rate = result.sample_rate
except Exception as e:
# If voice cloning fails, try without it
logger.warning("Voice cloning failed, generating without voice prompt: %s", e)
for result in self.model.generate(text, lang_code=lang):
audio_chunks.append(np.array(result.audio))
sample_rate = result.sample_rate
# Concatenate all chunks
if audio_chunks:
@@ -343,6 +347,8 @@ class MLXSTTBackend:
"""
await self.load_model_async(model_size)
progress_model_name = f"whisper-{self.model_size}"
def _transcribe_sync():
"""Run synchronous transcription in thread pool."""
# MLX Whisper transcription using generate method
@@ -351,7 +357,11 @@ class MLXSTTBackend:
if language:
decode_options["language"] = language
result = self.model.generate(str(audio_path), **decode_options)
# Model is loaded → weights are on disk. Force offline so
# lazy tokenizer/config lookups don't hang when the user is
# disconnected (issue #462).
with force_offline_if_cached(True, progress_model_name):
result = self.model.generate(str(audio_path), **decode_options)
# Extract text from result
if isinstance(result, str):
+56 -39
View File
@@ -172,13 +172,19 @@ class PyTorchTTSBackend:
# This shouldn't happen in practice, but handle it
return {"prompt": cached_prompt}, True
model_name = f"qwen-tts-{self._current_model_size}"
def _create_prompt_sync():
"""Run synchronous voice prompt creation in thread pool."""
return self.model.create_voice_clone_prompt(
ref_audio=str(audio_path),
ref_text=reference_text,
x_vector_only_mode=False,
)
# Model is loaded → weights are on disk. Force offline so
# lazy tokenizer/config lookups inside qwen_tts don't hang
# when the user is disconnected (issue #462).
with force_offline_if_cached(True, model_name):
return self.model.create_voice_clone_prompt(
ref_audio=str(audio_path),
ref_text=reference_text,
x_vector_only_mode=False,
)
# Run blocking operation in thread pool
voice_prompt_items = await asyncio.to_thread(_create_prompt_sync)
@@ -221,19 +227,24 @@ class PyTorchTTSBackend:
# Load model
await self.load_model_async(None)
model_name = f"qwen-tts-{self._current_model_size}"
def _generate_sync():
"""Run synchronous generation in thread pool."""
# Set seed if provided
if seed is not None:
manual_seed(seed, self.device)
# Generate audio - this is the blocking operation
wavs, sample_rate = self.model.generate_voice_clone(
text=text,
voice_clone_prompt=voice_prompt,
language=LANGUAGE_CODE_TO_NAME.get(language, "auto"),
instruct=instruct,
)
# Model is loaded → weights are on disk. Force offline so
# lazy tokenizer/config lookups inside qwen_tts don't hang
# when the user is disconnected (issue #462).
with force_offline_if_cached(True, model_name):
wavs, sample_rate = self.model.generate_voice_clone(
text=text,
voice_clone_prompt=voice_prompt,
language=LANGUAGE_CODE_TO_NAME.get(language, "auto"),
instruct=instruct,
)
return wavs[0], sample_rate
# Run blocking inference in thread pool to avoid blocking event loop
@@ -331,40 +342,46 @@ class PyTorchSTTBackend:
"""
await self.load_model_async(model_size)
progress_model_name = f"whisper-{self.model_size}"
def _transcribe_sync():
"""Run synchronous transcription in thread pool."""
# Load audio
audio, sr = load_audio(audio_path, sample_rate=16000)
audio, _sr = load_audio(audio_path, sample_rate=16000)
# Process audio
inputs = self.processor(
audio,
sampling_rate=16000,
return_tensors="pt",
)
inputs = inputs.to(self.device)
# Generate transcription
# If language is provided, force it; otherwise let Whisper auto-detect
generate_kwargs = {}
if language:
forced_decoder_ids = self.processor.get_decoder_prompt_ids(
language=language,
task="transcribe",
# Model is loaded → weights are on disk. Force offline so
# `get_decoder_prompt_ids` and any lazy tokenizer lookups
# don't hang when the user is disconnected (issue #462).
with force_offline_if_cached(True, progress_model_name):
# Process audio
inputs = self.processor(
audio,
sampling_rate=16000,
return_tensors="pt",
)
generate_kwargs["forced_decoder_ids"] = forced_decoder_ids
inputs = inputs.to(self.device)
with torch.no_grad():
predicted_ids = self.model.generate(
inputs["input_features"],
**generate_kwargs,
)
# Generate transcription
# If language is provided, force it; otherwise let Whisper auto-detect
generate_kwargs = {}
if language:
forced_decoder_ids = self.processor.get_decoder_prompt_ids(
language=language,
task="transcribe",
)
generate_kwargs["forced_decoder_ids"] = forced_decoder_ids
# Decode
transcription = self.processor.batch_decode(
predicted_ids,
skip_special_tokens=True,
)[0]
with torch.no_grad():
predicted_ids = self.model.generate(
inputs["input_features"],
**generate_kwargs,
)
# Decode
transcription = self.processor.batch_decode(
predicted_ids,
skip_special_tokens=True,
)[0]
return transcription.strip()
+20 -13
View File
@@ -28,6 +28,7 @@ from .base import (
combine_voice_prompts as _combine_voice_prompts,
model_load_progress,
)
from ..utils.hf_offline_patch import force_offline_if_cached
logger = logging.getLogger(__name__)
@@ -104,18 +105,19 @@ class QwenCustomVoiceBackend:
model_path = self._get_model_path(model_size)
logger.info("Loading Qwen CustomVoice %s on %s...", model_size, self.device)
if self.device == "cpu":
self.model = Qwen3TTSModel.from_pretrained(
model_path,
torch_dtype=torch.float32,
low_cpu_mem_usage=False,
)
else:
self.model = Qwen3TTSModel.from_pretrained(
model_path,
device_map=self.device,
torch_dtype=torch.bfloat16,
)
with force_offline_if_cached(is_cached, model_name):
if self.device == "cpu":
self.model = Qwen3TTSModel.from_pretrained(
model_path,
torch_dtype=torch.float32,
low_cpu_mem_usage=False,
)
else:
self.model = Qwen3TTSModel.from_pretrained(
model_path,
device_map=self.device,
torch_dtype=torch.bfloat16,
)
self._current_model_size = model_size
self.model_size = model_size
@@ -184,6 +186,7 @@ class QwenCustomVoiceBackend:
await self.load_model_async(None)
speaker = voice_prompt.get("preset_voice_id") or QWEN_CV_DEFAULT_SPEAKER
model_name = f"qwen-custom-voice-{self._current_model_size}"
def _generate_sync():
if seed is not None:
@@ -203,7 +206,11 @@ class QwenCustomVoiceBackend:
if instruct:
kwargs["instruct"] = instruct
wavs, sample_rate = self.model.generate_custom_voice(**kwargs)
# Model is loaded → weights are on disk. Force offline so
# lazy tokenizer/config lookups inside qwen_tts don't hang
# when the user is disconnected (issue #462).
with force_offline_if_cached(True, model_name):
wavs, sample_rate = self.model.generate_custom_voice(**kwargs)
return wavs[0], sample_rate
audio, sample_rate = await asyncio.to_thread(_generate_sync)
+10 -3
View File
@@ -3,6 +3,12 @@
mlx>=0.30.0
# miniaudio is a runtime dep of mlx-audio's STT path (mlx_audio.stt).
# mlx-audio itself is installed --no-deps (see comment below), so we
# must list miniaudio explicitly here or transcription fails on fresh
# M1 installs with `ModuleNotFoundError: miniaudio` (issue #505).
miniaudio>=1.59
# NOTE: mlx-audio is intentionally not listed here. From 0.3.1 onward it
# declares `transformers==5.0.0rc3` / `>=5.0.0`, which conflicts with the
# `transformers<=4.57.6` cap in requirements.txt and breaks CI's clean
@@ -10,6 +16,7 @@ mlx>=0.30.0
# mlx_audio.stt.load) works fine on transformers 4.57.x in practice.
#
# Install it via `pip install --no-deps mlx-audio==0.4.1` after this file
# (see .github/workflows/release.yml). All other mlx-audio runtime deps
# (huggingface_hub, librosa, miniaudio, mlx-lm, numba, numpy, protobuf,
# pyloudnorm, sounddevice, tqdm) are already in requirements.txt.
# (see .github/workflows/release.yml). Most other mlx-audio runtime deps
# (huggingface_hub, librosa, mlx-lm, numba, numpy, protobuf, pyloudnorm,
# sounddevice, tqdm) are already in requirements.txt or pulled in by
# other engines.
+118
View File
@@ -0,0 +1,118 @@
"""
Unit tests for the ``force_offline_if_cached`` helper.
Verifies that the helper mutates the cached module constants in
``huggingface_hub.constants`` and ``transformers.utils.hub`` — not just
``os.environ`` — and that concurrent users are refcount-coordinated so
one thread's exit can't strip another thread's offline protection.
NOTE: These tests mutate process-global state in ``huggingface_hub.constants``
and ``transformers.utils.hub``. They are not safe under cross-process
parallelism (e.g. ``pytest-xdist`` with ``--dist=loadfile``/``loadscope``);
run this file serially.
"""
import os
import sys
import threading
from pathlib import Path
import pytest
sys.path.insert(0, str(Path(__file__).parent.parent))
from utils.hf_offline_patch import force_offline_if_cached # noqa: E402
def _hf_const():
import huggingface_hub.constants as hf_const
return hf_const
def _tf_hub():
import transformers.utils.hub as tf_hub
return tf_hub
def test_mutates_cached_huggingface_hub_constant():
original = _hf_const().HF_HUB_OFFLINE
with force_offline_if_cached(True, "t"):
assert _hf_const().HF_HUB_OFFLINE is True
assert original == _hf_const().HF_HUB_OFFLINE
def test_mutates_cached_transformers_constant():
original = _tf_hub()._is_offline_mode
with force_offline_if_cached(True, "t"):
assert _tf_hub()._is_offline_mode is True
assert original == _tf_hub()._is_offline_mode
def test_sets_env_variable():
original = os.environ.get("HF_HUB_OFFLINE")
with force_offline_if_cached(True, "t"):
assert "1" == os.environ.get("HF_HUB_OFFLINE")
assert original == os.environ.get("HF_HUB_OFFLINE")
def test_noop_when_not_cached():
before = _hf_const().HF_HUB_OFFLINE
with force_offline_if_cached(False, "t"):
assert before == _hf_const().HF_HUB_OFFLINE
def test_nested_contexts_respect_refcount():
original = _hf_const().HF_HUB_OFFLINE
with force_offline_if_cached(True, "outer"):
assert _hf_const().HF_HUB_OFFLINE is True
with force_offline_if_cached(True, "inner"):
assert _hf_const().HF_HUB_OFFLINE is True
# inner exit must not restore while outer is still active
assert _hf_const().HF_HUB_OFFLINE is True
assert original == _hf_const().HF_HUB_OFFLINE
def test_concurrent_threads_share_offline_window():
"""A slow thread must keep seeing offline mode even if a peer exits first."""
original = _hf_const().HF_HUB_OFFLINE
observations: list[bool] = []
errors: list[Exception] = []
barrier = threading.Barrier(2)
fast_exited = threading.Event()
def slow():
try:
with force_offline_if_cached(True, "slow"):
barrier.wait(timeout=5)
assert fast_exited.wait(timeout=5), "fast thread did not exit"
observations.append(_hf_const().HF_HUB_OFFLINE)
except Exception as exc: # noqa: BLE001
errors.append(exc)
def fast():
try:
with force_offline_if_cached(True, "fast"):
barrier.wait(timeout=5)
except Exception as exc: # noqa: BLE001
errors.append(exc)
finally:
fast_exited.set()
t_slow = threading.Thread(target=slow)
t_fast = threading.Thread(target=fast)
t_slow.start()
t_fast.start()
t_slow.join(timeout=5)
t_fast.join(timeout=5)
assert not t_slow.is_alive(), "slow thread did not finish"
assert not t_fast.is_alive(), "fast thread did not finish"
assert not errors, errors
assert observations == [True], "slow thread lost offline protection"
assert original == _hf_const().HF_HUB_OFFLINE
if __name__ == "__main__":
pytest.main([__file__, "-v"])
+1 -1
View File
@@ -175,7 +175,7 @@ async def main():
print(" ✅ Server is running")
# Test model
model_name = "qwen-tts-0.6B" # Note: 0.6B currently maps to 1.7B on MLX
model_name = "qwen-tts-0.6B"
# Check current status
print(f"\n📊 Checking status of {model_name}...")
+110 -27
View File
@@ -6,6 +6,7 @@ are already downloaded. Must be imported BEFORE mlx_audio.
import logging
import os
import threading
from contextlib import contextmanager
from pathlib import Path
from typing import Optional, Union
@@ -13,13 +14,33 @@ from typing import Optional, Union
logger = logging.getLogger(__name__)
# huggingface_hub reads ``HF_HUB_OFFLINE`` once at import time into
# ``huggingface_hub.constants.HF_HUB_OFFLINE``; transformers mirrors that into
# ``transformers.utils.hub._is_offline_mode`` at *its* import time. Toggling
# ``os.environ`` after either module is imported does not flip those cached
# bools, and the hot paths (``_http._default_backend_factory``,
# ``transformers.utils.hub.is_offline_mode``) read the bools — not the env.
# We mutate the cached constants directly, guarded by a refcount so
# concurrent inference threads share a single offline window safely.
_offline_lock = threading.RLock()
_offline_refcount = 0
_saved_env: Optional[str] = None
_saved_hf_const: Optional[bool] = None
_saved_transformers_const: Optional[bool] = None
@contextmanager
def force_offline_if_cached(is_cached: bool, model_label: str = ""):
"""Context manager that sets ``HF_HUB_OFFLINE=1`` while loading a cached model.
"""Force offline mode for the duration of a cached-model operation.
Flips ``HF_HUB_OFFLINE`` in the process env **and** in the cached bools
inside ``huggingface_hub.constants`` / ``transformers.utils.hub`` so HTTP
adapters and offline-mode checks actually see the change. Uses a refcount
so multiple concurrent inference threads share a single offline window
and the last one to exit restores state.
If *is_cached* is ``False`` the block runs normally (network allowed).
If the offline load raises an error containing "offline" we automatically
retry with network access so a partially-cached model still works.
Args:
is_cached: Whether the model weights are already on disk.
@@ -29,34 +50,96 @@ def force_offline_if_cached(is_cached: bool, model_label: str = ""):
yield
return
original_value = os.environ.get("HF_HUB_OFFLINE")
os.environ["HF_HUB_OFFLINE"] = "1"
logger.info(
"[offline-guard] %s is cached — forcing HF_HUB_OFFLINE=1",
model_label or "model",
)
global _offline_refcount, _saved_env, _saved_hf_const, _saved_transformers_const
with _offline_lock:
if _offline_refcount == 0:
# Snapshot prior state, apply new state, roll back on *any*
# failure. Catching only ImportError here would let a partially
# broken install (RuntimeError, AttributeError from a half-init
# module, etc.) leave the cached HF constants mutated without
# bumping the refcount — a persistent offline leak that outlives
# the process and is miserable to debug.
prev_env = os.environ.get("HF_HUB_OFFLINE")
prev_hf: Optional[bool] = None
prev_tf: Optional[bool] = None
try:
try:
import huggingface_hub.constants as hf_const
prev_hf = hf_const.HF_HUB_OFFLINE
hf_const.HF_HUB_OFFLINE = True
except ImportError:
prev_hf = None
try:
import transformers.utils.hub as tf_hub
prev_tf = tf_hub._is_offline_mode
tf_hub._is_offline_mode = True
except ImportError:
prev_tf = None
os.environ["HF_HUB_OFFLINE"] = "1"
except BaseException:
# Roll back whatever we already changed, then re-raise so
# the caller sees the real failure.
if prev_hf is not None:
try:
import huggingface_hub.constants as hf_const
hf_const.HF_HUB_OFFLINE = prev_hf
except ImportError:
pass
if prev_tf is not None:
try:
import transformers.utils.hub as tf_hub
tf_hub._is_offline_mode = prev_tf
except ImportError:
pass
if prev_env is not None:
os.environ["HF_HUB_OFFLINE"] = prev_env
else:
os.environ.pop("HF_HUB_OFFLINE", None)
raise
_saved_env = prev_env
_saved_hf_const = prev_hf
_saved_transformers_const = prev_tf
logger.info(
"[offline-guard] %s is cached — forcing offline mode",
model_label or "model",
)
_offline_refcount += 1
try:
yield
except Exception as exc:
if "offline" in str(exc).lower():
logger.warning(
"[offline-guard] Offline load failed for %s, retrying with network: %s",
model_label or "model",
exc,
)
# Restore original env and retry — caller must wrap the load
# inside force_offline_if_cached so retrying here isn't possible.
# Instead, propagate a flag via the exception so the caller can
# decide. For simplicity we just let it fall through to the
# finally block and re-raise.
raise
raise
finally:
if original_value is not None:
os.environ["HF_HUB_OFFLINE"] = original_value
else:
os.environ.pop("HF_HUB_OFFLINE", None)
with _offline_lock:
_offline_refcount -= 1
if _offline_refcount == 0:
if _saved_env is not None:
os.environ["HF_HUB_OFFLINE"] = _saved_env
else:
os.environ.pop("HF_HUB_OFFLINE", None)
if _saved_hf_const is not None:
try:
import huggingface_hub.constants as hf_const
hf_const.HF_HUB_OFFLINE = _saved_hf_const
except ImportError:
pass
if _saved_transformers_const is not None:
try:
import transformers.utils.hub as tf_hub
tf_hub._is_offline_mode = _saved_transformers_const
except ImportError:
pass
_saved_env = None
_saved_hf_const = None
_saved_transformers_const = None
def patch_huggingface_hub_offline():
+2 -6
View File
@@ -309,11 +309,7 @@ Still reported. Users get stuck downloads, can't resume, offline mode edge cases
**Fix path:** PR #443 addresses infinite offline retry. CustomVoice-specific download failures (#475, #445) need triage — likely related to frozen-binary import fixes in PR #438. TADA cluster (#336, #348) and macOS ARM import regressions (#287, #275, #304) need a dedicated triage pass.
**Qwen 0.6B-downloads-1.7B reports:** **#485** (2026-04-19), **#423** (macOS M1), **#329**. Platform-dependent:
- **On MLX (Apple Silicon) — not a bug.** `mlx-community` only publishes 1.7B-Base-bf16 weights, so the 0.6B Base option intentionally resolves to the same repo (`backend/backends/__init__.py:180` — `# 0.6B not available in MLX, falls back`). UX gap: the selector offers a size that doesn't exist on the active backend. Fix: (a) hide the 0.6B option on MLX, or (b) label it "0.6B (uses 1.7B on Apple Silicon)".
- **On PyTorch (Windows/Linux/CUDA/ROCm/XPU/CPU) — real bug if reported.** Both 0.6B and 1.7B have distinct repos (`Qwen/Qwen3-TTS-12Hz-0.6B-Base` vs `-1.7B-Base`). Triage each report by platform before merging into the MLX cluster.
- **Qwen CustomVoice (either platform)** — no fallback, both sizes always have dedicated repos.
**Qwen 0.6B-downloads-1.7B reports:** **#485** (2026-04-19), **#423** (macOS M1), **#329**. Originally a stale-fallback bug: `mlx-community/Qwen3-TTS-12Hz-0.6B-Base-bf16` wasn't published when MLX support shipped, so the 0.6B slot was aliased to the 1.7B repo. The 0.6B bf16 conversion is live now and both `backend/backends/mlx_backend.py` and `backend/backends/__init__.py` point at their correct repos. Qwen CustomVoice is unaffected — it runs via PyTorch on all platforms, both sizes always have dedicated repos.
### Language Requests (ongoing)
@@ -393,7 +389,7 @@ Notable:
| **#306** ("voice model"), **#389** ("New model"), **#473** ("New functionality") | Title-only issues, no content. Request details or close. |
| **#309** | Uninstall/cleanup question. Answer and close. |
| **#241** | "How to use in Colab" — support question, not a bug. |
| **#423** / **#485** / **#329** | Platform-dependent. On MLX: not a bug (0.6B weights don't exist upstream, fallback is intentional — fix UX). On PyTorch: real bug if reproducible. Classify each by reporter's platform before deduping. |
| **#423** / **#485** / **#329** | Stale MLX fallback to 1.7B repo — fixed; 0.6B bf16 conversion now live on `mlx-community`, registry points at correct repo on both backends. |
| **#336** / **#348** | TADA download/registration cluster — triage together. |
| **#287** / **#275** / **#304** | macOS ARM import regressions on new version — likely one root cause. |
| **#292**, **#349** | Possibly already fixed by merged PRs (#321/#412 and #345). Verify + close. |
@@ -446,20 +446,6 @@ Restart the app to create a fresh database.
huggingface-cli download Qwen/Qwen3-TTS-12Hz-1.7B-Base
```
### Qwen 0.6B Downloads the Same Files as 1.7B on Apple Silicon
**Symptoms:**
- You select Qwen 0.6B on an Apple Silicon Mac and the download is the same size as 1.7B
- Generation speed and VRAM usage match 1.7B, not the expected smaller model
**Explanation:**
This is intentional, not a bug. The MLX community only publishes `mlx-community/Qwen3-TTS-12Hz-1.7B-Base-bf16` — there is no 0.6B MLX build. Voicebox's model registry falls back to the 1.7B weights when 0.6B is selected on MLX (see `backend/backends/__init__.py`).
**Solution:**
- On Apple Silicon, both size options use the 1.7B model — pick either.
- If you specifically need a smaller model, switch to **Kokoro 82M** (~350 MB) or **LuxTTS** (~300 MB) — both CPU-realtime.
- On Windows/Linux with PyTorch, 0.6B and 1.7B are distinct repos and behave differently.
### Wrong Model Version
**Symptoms:**
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@voicebox/landing",
"version": "0.4.1",
"version": "0.4.2",
"description": "Landing page for voicebox.sh",
"scripts": {
"dev": "next dev --turbo",
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "voicebox",
"version": "0.4.1",
"version": "0.4.2",
"private": true,
"workspaces": [
"app",
+1 -1
View File
@@ -1,7 +1,7 @@
{
"name": "@voicebox/tauri",
"private": true,
"version": "0.4.1",
"version": "0.4.2",
"type": "module",
"scripts": {
"dev": "vite",
+1 -1
View File
@@ -5041,7 +5041,7 @@ checksum = "0b928f33d975fc6ad9f86c8f283853ad26bdd5b10b7f1542aa2fa15e2289105a"
[[package]]
name = "voicebox"
version = "0.4.1"
version = "0.4.2"
dependencies = [
"base64 0.22.1",
"core-foundation-sys",
+1 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "voicebox"
version = "0.4.1"
version = "0.4.2"
description = "A production-quality desktop app for Qwen3-TTS voice cloning and generation"
authors = ["you"]
license = ""
+1 -1
View File
@@ -1,7 +1,7 @@
{
"$schema": "https://schema.tauri.app/config/2",
"productName": "Voicebox",
"version": "0.4.1",
"version": "0.4.2",
"identifier": "sh.voicebox.app",
"build": {
"beforeDevCommand": "bun run dev",
+1 -1
View File
@@ -1,7 +1,7 @@
{
"name": "@voicebox/web",
"private": true,
"version": "0.4.1",
"version": "0.4.2",
"type": "module",
"scripts": {
"dev": "vite",