fix(offline): guard inference paths with HF_HUB_OFFLINE (#462)

PR #443 wrapped the model *load* path with `force_offline_if_cached` so
cached models don't phone home at startup. The context manager restores
`HF_HUB_OFFLINE` on exit, which left inference paths (generate,
transcribe, voice-prompt creation) unguarded — and `qwen_tts`,
`mlx_audio`, and `transformers` perform lazy tokenizer/processor/config
lookups during inference. With internet on, those lookups are
near-instant and invisible; with internet off, `requests` hangs on DNS
or connect until the network returns. This is exactly what users in
#462 describe: model shows "Loaded", internet drops, generation
"thinks" forever, internet comes back, generation completes.

Chatterbox and LuxTTS don't exhibit this because their engine libs
resolve everything through already-cached paths at load time.

Fix: wrap each inference-sync body with `force_offline_if_cached(True,
...)`. Since inference only runs after a successful load, weights are
known to be on disk, so `is_cached=True` is unconditional.

Also adds the load-time guard that was missing from
`qwen_custom_voice_backend.py` — CustomVoice previously had no offline
protection at all.

Paths patched:
  - PyTorchTTSBackend.create_voice_prompt (create_voice_clone_prompt)
  - PyTorchTTSBackend.generate (generate_voice_clone)
  - PyTorchSTTBackend.transcribe (Whisper generate + decoder-prompt-ids)
  - MLXTTSBackend.generate (mlx_audio generate, all branches)
  - MLXSTTBackend.transcribe (mlx_audio whisper generate)
  - QwenCustomVoiceBackend._load_model_sync + generate

Does not address the secondary `check_model_inputs() missing 'func'`
error reported in the same issue — that's a `transformers` 5.x
version-skew bug on the install path, separate concern.

Fixes #462.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
This commit is contained in:
James Pine
2026-04-19 16:56:28 -07:00
co-authored by Claude Opus 4.7
parent e3f7cd9d00
commit f3ed312cf2
3 changed files with 110 additions and 74 deletions
+20 -13
View File
@@ -28,6 +28,7 @@ from .base import (
combine_voice_prompts as _combine_voice_prompts,
model_load_progress,
)
from ..utils.hf_offline_patch import force_offline_if_cached
logger = logging.getLogger(__name__)
@@ -104,18 +105,19 @@ class QwenCustomVoiceBackend:
model_path = self._get_model_path(model_size)
logger.info("Loading Qwen CustomVoice %s on %s...", model_size, self.device)
if self.device == "cpu":
self.model = Qwen3TTSModel.from_pretrained(
model_path,
torch_dtype=torch.float32,
low_cpu_mem_usage=False,
)
else:
self.model = Qwen3TTSModel.from_pretrained(
model_path,
device_map=self.device,
torch_dtype=torch.bfloat16,
)
with force_offline_if_cached(is_cached, model_name):
if self.device == "cpu":
self.model = Qwen3TTSModel.from_pretrained(
model_path,
torch_dtype=torch.float32,
low_cpu_mem_usage=False,
)
else:
self.model = Qwen3TTSModel.from_pretrained(
model_path,
device_map=self.device,
torch_dtype=torch.bfloat16,
)
self._current_model_size = model_size
self.model_size = model_size
@@ -184,6 +186,7 @@ class QwenCustomVoiceBackend:
await self.load_model_async(None)
speaker = voice_prompt.get("preset_voice_id") or QWEN_CV_DEFAULT_SPEAKER
model_name = f"qwen-custom-voice-{self._current_model_size}"
def _generate_sync():
if seed is not None:
@@ -203,7 +206,11 @@ class QwenCustomVoiceBackend:
if instruct:
kwargs["instruct"] = instruct
wavs, sample_rate = self.model.generate_custom_voice(**kwargs)
# Model is loaded → weights are on disk. Force offline so
# lazy tokenizer/config lookups inside qwen_tts don't hang
# when the user is disconnected (issue #462).
with force_offline_if_cached(True, model_name):
wavs, sample_rate = self.model.generate_custom_voice(**kwargs)
return wavs[0], sample_rate
audio, sample_rate = await asyncio.to_thread(_generate_sync)