fix(offline): patch transformers mistral-regex check to survive HF failures (#530)

* fix(offline): patch transformers mistral-regex check to survive HF failures

transformers 4.57.x's `PreTrainedTokenizerBase._patch_mistral_regex` calls
`huggingface_hub.model_info(repo_id)` unconditionally during any non-local
tokenizer load to probe for Mistral-family models. The call raises on
`HF_HUB_OFFLINE=1`, on network outages, and on slow/blocked HF endpoints,
and transformers doesn't catch any of it — the exception bubbles out of
`from_pretrained` and kills the load for unrelated engines (Qwen TTS,
Qwen CustomVoice, TADA, etc.).

0.4.2's load-time `force_offline_if_cached` guard walked straight into
this trap: on cached online users it flipped `HF_HUB_OFFLINE=1` and
converted a healthy load into a hard crash. 0.4.3's inference-path guard
masked it; #524 removed the inference guard in 0.4.4, and users updating
to 0.4.4 started hitting the same error on the load path instead
(#526).

Fix:
- Wrap `_patch_mistral_regex` so any exception from the inner HF
  metadata check is swallowed and the tokenizer is returned unchanged.
  Voicebox never loads Mistral models, so the regex rewrite this check
  gates is a no-op for us; matches the success-path behavior for
  non-Mistral repos (tokenization_utils_base.py:2503).
- Drop the `force_offline_if_cached` wraps from every load path
  (pytorch_backend Qwen + Whisper, qwen_custom_voice_backend,
  mlx_backend Qwen + Whisper). With the mistral patch in place they
  provide zero value and only risk re-introducing the same class of
  bug. Helper and its unit tests stay — still correct for targeted
  future use.
- Add `backend/tests/test_offline_patch.py` covering
  OfflineModeIsEnabled / ConnectionError suppression, success
  pass-through, idempotence, and the missing-method no-op path.

Fixes #526.

* fix(offline): install mistral-regex patch for non-MLX backends

The previous commit left the patch wired only through ``mlx_backend.py``'s
existing import of ``hf_offline_patch``. On Windows/Linux/CUDA users who
never load the MLX backend (everyone who hit #526), the patch module was
never imported, so ``patch_transformers_mistral_regex`` never ran and the
crash persisted.

Hoist the import into ``backends/__init__.py``. Every backend imports from
this package, so the module-level patch install runs before any
``from_pretrained`` call regardless of which engine the user picks.

Caught by CodeRabbit and Cursor Bugbot on #530.
This commit is contained in:
Jamie Pine
2026-04-21 22:01:29 -07:00
committed by GitHub
parent 74e004400f
commit d61e884104
6 changed files with 202 additions and 38 deletions
+52
View File
@@ -142,6 +142,57 @@ def force_offline_if_cached(is_cached: bool, model_label: str = ""):
_saved_transformers_const = None
_mistral_regex_patched = False
def patch_transformers_mistral_regex():
"""Make transformers' tokenizer load robust to HuggingFace metadata failures.
transformers 4.57.x added ``PreTrainedTokenizerBase._patch_mistral_regex``
which unconditionally calls ``huggingface_hub.model_info(repo_id)`` during
every non-local tokenizer load to check whether the model is a Mistral
variant. That call raises on ``HF_HUB_OFFLINE=1`` and on plain network
failures, killing unrelated loads (Qwen TTS, TADA, etc.).
Voicebox never loads Mistral models, so the rewrite the function would
apply is a no-op for us anyway. Wrap the method so any exception from the
metadata lookup returns the tokenizer unchanged — matching the success-path
behavior for non-Mistral repos (transformers 4.57.3,
``tokenization_utils_base.py:2503``).
"""
global _mistral_regex_patched
if _mistral_regex_patched:
return
try:
from transformers.tokenization_utils_base import PreTrainedTokenizerBase
except ImportError:
logger.debug("transformers not available, skipping mistral-regex patch")
return
original = getattr(PreTrainedTokenizerBase, "_patch_mistral_regex", None)
if original is None:
logger.debug(
"transformers has no _patch_mistral_regex attribute, skipping patch",
)
return
def safe_patch_mistral_regex(cls, tokenizer, pretrained_model_name_or_path, *args, **kwargs):
try:
return original(tokenizer, pretrained_model_name_or_path, *args, **kwargs)
except Exception as exc:
logger.debug(
"[mistral-regex-patch] suppressed %s for %r, returning tokenizer as-is",
type(exc).__name__,
pretrained_model_name_or_path,
)
return tokenizer
PreTrainedTokenizerBase._patch_mistral_regex = classmethod(safe_patch_mistral_regex)
_mistral_regex_patched = True
logger.debug("installed _patch_mistral_regex wrapper")
def patch_huggingface_hub_offline():
"""Monkey-patch huggingface_hub to force offline mode."""
try:
@@ -215,4 +266,5 @@ def ensure_original_qwen_config_cached():
if os.environ.get("VOICEBOX_OFFLINE_PATCH", "1") != "0":
patch_huggingface_hub_offline()
patch_transformers_mistral_regex()
ensure_original_qwen_config_cached()