ADDED MLX FOR SUPER FAST GENERATIONS ON APPLE SILICON

- Added support for MLX backend on Apple Silicon, enabling optimized performance for TTS and STT tasks.
- Updated release workflow to include MLX-specific dependencies and configurations for macOS platforms.
- Refactored backend code to dynamically select between MLX and PyTorch based on the runtime environment.
- Enhanced model loading and inference logic to accommodate backend-specific requirements, including updated model IDs and hidden imports.
- Improved health check and model status reporting to reflect the active backend type.
- Streamlined caching mechanisms to support both backend types, ensuring compatibility and performance.
This commit is contained in:
Jamie Pine
2026-01-29 21:50:46 -08:00
parent 86768288ce
commit 081f45e680
13 changed files with 1231 additions and 641 deletions
+33
View File
@@ -0,0 +1,33 @@
"""
Platform detection for backend selection.
"""
import platform
from typing import Literal
def is_apple_silicon() -> bool:
"""
Check if running on Apple Silicon (arm64 macOS).
Returns:
True if on Apple Silicon, False otherwise
"""
return platform.system() == "Darwin" and platform.machine() == "arm64"
def get_backend_type() -> Literal["mlx", "pytorch"]:
"""
Detect the best backend for the current platform.
Returns:
"mlx" on Apple Silicon (if MLX is available), "pytorch" otherwise
"""
if is_apple_silicon():
try:
import mlx
return "mlx"
except ImportError:
# MLX not installed, fallback to PyTorch
return "pytorch"
return "pytorch"