* fix(offline): guard inference paths with HF_HUB_OFFLINE (#462)
PR #443 wrapped the model *load* path with `force_offline_if_cached` so
cached models don't phone home at startup. The context manager restores
`HF_HUB_OFFLINE` on exit, which left inference paths (generate,
transcribe, voice-prompt creation) unguarded — and `qwen_tts`,
`mlx_audio`, and `transformers` perform lazy tokenizer/processor/config
lookups during inference. With internet on, those lookups are
near-instant and invisible; with internet off, `requests` hangs on DNS
or connect until the network returns. This is exactly what users in
#462 describe: model shows "Loaded", internet drops, generation
"thinks" forever, internet comes back, generation completes.
Chatterbox and LuxTTS don't exhibit this because their engine libs
resolve everything through already-cached paths at load time.
Fix: wrap each inference-sync body with `force_offline_if_cached(True,
...)`. Since inference only runs after a successful load, weights are
known to be on disk, so `is_cached=True` is unconditional.
Also adds the load-time guard that was missing from
`qwen_custom_voice_backend.py` — CustomVoice previously had no offline
protection at all.
Paths patched:
- PyTorchTTSBackend.create_voice_prompt (create_voice_clone_prompt)
- PyTorchTTSBackend.generate (generate_voice_clone)
- PyTorchSTTBackend.transcribe (Whisper generate + decoder-prompt-ids)
- MLXTTSBackend.generate (mlx_audio generate, all branches)
- MLXSTTBackend.transcribe (mlx_audio whisper generate)
- QwenCustomVoiceBackend._load_model_sync + generate
Does not address the secondary `check_model_inputs() missing 'func'`
error reported in the same issue — that's a `transformers` 5.x
version-skew bug on the install path, separate concern.
Fixes#462.
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
* fix(offline): mutate cached HF constants + threadsafe refcount
Review feedback on the initial fix surfaced two real issues:
1. ``os.environ`` toggles alone don't flip offline mode.
``huggingface_hub.constants.HF_HUB_OFFLINE`` is read once at import
time into a module-level bool; ``transformers.utils.hub._is_offline_mode``
mirrors that bool at its own import time. The hot paths
(``_http._default_backend_factory`` in huggingface_hub,
``is_offline_mode`` in transformers) read the cached bools — not the
env — so mutating only ``os.environ`` was a no-op.
2. Race condition on concurrent inference. Two threads running inside
``force_offline_if_cached`` via ``asyncio.to_thread`` could have
thread A's ``finally`` strip thread B's offline protection mid-run.
Rewrite the helper to:
- mutate ``huggingface_hub.constants.HF_HUB_OFFLINE`` and
``transformers.utils.hub._is_offline_mode`` directly
- refcount concurrent users under a single ``threading.RLock`` so a
shared offline window is restored only when the last caller exits
- still write ``os.environ`` for anything that reads it dynamically
Also addresses the unused-variable ruff flag on the Whisper transcribe
path (``audio, sr`` → ``audio, _sr``).
New unit tests cover the cached-constant mutation, env propagation,
no-op on ``is_cached=False``, nested contexts, and a threaded race
where a slow thread must retain offline mode after a peer exits.
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
* fix(offline): atomic entry rollback + tidy test assertions
Review follow-up:
- Wrap the `_offline_refcount == 0` setup in a try/except so any failure
during the cached-constant mutation (including unexpected non-ImportError
like RuntimeError or AttributeError from a half-initialized module)
rolls back *all* partial state before re-raising. Without this, a
mid-setup crash could leave `huggingface_hub.constants.HF_HUB_OFFLINE`
mutated but the refcount at 0 — a persistent offline flag outliving
the process.
- Swap ruff-flagged Yoda comparisons in the new test file (SIM300) and
add a module-level note warning that these tests mutate global state
and are not safe under cross-process parallelism.
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
* test(offline): make concurrency test deterministic and bounded
Replace the `sleep(0.15)` ordering hack with an explicit `threading.Event`
the fast thread sets in `finally`. The slow thread waits on that event
(bounded), then observes the flag — so we deterministically verify the
slow thread still sees offline mode after the fast thread has exited.
Also add timeouts to `barrier.wait()` and assert `not thread.is_alive()`
after the joins so the test can't hang on an unexpected failure path.
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <[email protected]>
Applies the cache_dir portion of #218. On Windows local setups, model
assets can split between .hf-cache/hub and .hf-cache/transformers when
Qwen3TTSModel.from_pretrained doesn't explicitly pin the cache root —
speech_tokenizer and preprocessor_config.json then fail to resolve
during load, causing 500s at generation time.
Routes both HF Hub and Transformers through hf_constants.HF_HUB_CACHE.
Skipped the torch_dtype= → dtype= rename from #218: transformers 4.36
(our minimum) doesn't accept the dtype alias, only 4.46+. Once we bump
the minimum we can make that change.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Auto-detect Intel Arc GPUs during Windows setup and install PyTorch
with XPU support + intel-extension-for-pytorch. Enable allow_xpu=True
on all TTS backends (Chatterbox, Chatterbox Turbo, Hume TADA, LuxTTS)
that previously only supported CUDA. Add shared empty_device_cache()
and manual_seed() helpers in base.py to handle XPU memory management
and reproducible seeding alongside CUDA.
Qwen TTS and Whisper Base make network calls to HuggingFace even when
model weights are fully cached locally, because from_pretrained()
defaults to local_files_only=False. This causes failures for offline
users.
Add a reusable force_offline_if_cached() context manager that sets
HF_HUB_OFFLINE=1 during model loading when is_model_cached() is True.
Applied to all four affected load paths:
- PyTorchTTSBackend (Qwen TTS)
- PyTorchSTTBackend (Whisper)
- MLXTTSBackend (refactored from inline implementation)
- MLXSTTBackend (previously unprotected)
Closes#82
- Use DB COUNT query instead of list length for take-N label to avoid
TOCTOU race between list_versions and create_version
- Add focus:bg-muted to SelectTrigger for keyboard focus visibility
Both PyTorch and MLX backends silently dropped the language parameter —
it was accepted by generate() but never forwarded to the underlying
Qwen3-TTS model, causing it to default to auto-detection which
frequently confuses similar languages (e.g. Portuguese for Spanish).
- Add LANGUAGE_CODE_TO_NAME mapping (ISO 639-1 to full name) to both backends
- PyTorch: pass language= to generate_voice_clone()
- MLX: pass lang_code= to all 4 model.generate() call sites
- Frontend: auto-sync generation form language with selected voice profile
Closes#97
Cherry-picked and adapted from PR #89 and #214:
- Linux audio capture via PulseAudio/PipeWire monitor sources (cpal)
- AMD ROCm GPU support: HSA_OVERRIDE_GFX_VERSION env var, ROCm detection
- Whisper Turbo model (openai/whisper-large-v3-turbo) in all endpoints
- Cleaner Whisper language handling via generate_kwargs
- tauri::async_runtime::spawn fix to prevent panic on app shutdown
- Enable Linux (ubuntu-22.04) in release CI matrix
- Add cancel (X) button on downloading and errored model items
- Add collapsible Problems panel (VS Code-style) showing error details
- Add "Clear All" button to reset all stale download/error state
- Add POST /models/download/cancel endpoint to dismiss individual downloads
- Add POST /tasks/clear endpoint to reset all task and progress state
- Include error messages in /tasks/active response for visibility
- Capture SSE error messages client-side for immediate display
- Fix whisper-large using wrong HF repo (openai/whisper-large → openai/whisper-large-v3)
- Fix Whisper HF repo mapping in both PyTorch and MLX backends
- Shorten error toast to point users to Problems panel instead of wall of text
- Updated caching methods in MLX, PyTorch, and backend to ensure models are fully downloaded before being marked as cached.
- Improved progress tracking to filter out non-download progress and provide accurate feedback during model downloads.
- Enhanced HFProgressTracker to skip non-byte progress bars and ensure meaningful progress reporting.
- Refactored progress initialization to provide immediate feedback while fetching metadata from HuggingFace.
- Added error handling and logging for better debugging during cache checks and download processes.
- Introduced methods to check if models are cached locally in MLX and PyTorch backends.
- Enhanced progress tracking during model loading to filter out non-download progress when models are cached.
- Updated HFProgressTracker to conditionally report progress based on download status.
- Added test scripts for monitoring SSE events during model downloads and verifying progress tracking functionality.
- Improved overall error handling and logging for better debugging during model download processes.
- Updated HistoryTable to implement infinite scrolling for loading history items dynamically.
- Introduced state management for accumulated history and total item count.
- Added Intersection Observer for triggering additional data fetches when scrolling.
- Implemented cache clearing functionality in the backend to manage voice prompt caches effectively.
- Improved loading indicators and user feedback for data fetching states.
- Refactored code for better readability and maintainability.
- Updated hidden imports in build_binary.py to replace 'mlx_audio.asr' with 'mlx_audio.stt'.
- Enhanced model loading logic in MLX and PyTorch backends to ensure proper progress tracking during model downloads.
- Improved error handling and context management for progress tracking in both backends.
- Bumped version to 0.1.10 in Cargo.lock to reflect recent changes.
- Added support for MLX backend on Apple Silicon, enabling optimized performance for TTS and STT tasks.
- Updated release workflow to include MLX-specific dependencies and configurations for macOS platforms.
- Refactored backend code to dynamically select between MLX and PyTorch based on the runtime environment.
- Enhanced model loading and inference logic to accommodate backend-specific requirements, including updated model IDs and hidden imports.
- Improved health check and model status reporting to reflect the active backend type.
- Streamlined caching mechanisms to support both backend types, ensuring compatibility and performance.
