Unloading a TTS/Whisper/LLM model on the MLX backend only dropped the
Python reference (`del self.model`). MLX keeps freed array buffers in
its own allocator pool for reuse instead of returning them to the OS,
so the process's memory footprint never actually shrank after unload
on Apple Silicon (the default backend there) until the process exited.
Add empty_mlx_cache() (backend/backends/base.py), wrapping
mx.clear_cache(), and call it from the three MLX unload_model()
implementations: MLXTTSBackend, MLXSTTBackend, MLXQwenLLMBackend.
Separately, the voice-clone prompt cache (backend/utils/cache.py) is a
process-lifetime dict populated by create_voice_prompt() across every
TTS engine, but nothing ever cleared it on model unload — only the
unrelated /tasks/clear-cache endpoint touched it. Add
clear_voice_prompt_memory_cache() (memory only, disk cache untouched
so a later generation still reloads the prompt instead of recomputing
it) and wire it into every TTS unload path (services/tts.py and the
qwen_custom_voice / generic branches of unload_model_by_config).
Whisper and the LLM backends never produce voice prompts, so their
unload paths are left alone.
Testing:
- New unit tests: backend/tests/test_mlx_unload_clears_cache.py,
backend/tests/test_voice_prompt_cache_unload.py (8 tests, all pass).
- Verified end-to-end on Apple Silicon against real cached models
(Qwen TTS 1.7B, Whisper Turbo, Qwen3 0.6B): loaded each via the
running app, unloaded via the real /models/{name}/unload endpoint,
and confirmed via mx.get_cache_memory()/get_active_memory() that the
MLX allocator's cache drops to 0 on every cycle. Ran a real
voice-clone generation end to end and confirmed the in-memory prompt
cache goes from 1 entry to 0 on unload while the on-disk .prompt
file is left intact.
Two small safety improvements:
1. Voice prompt cache (cache.py): add weights_only=True to torch.load()
so cached .prompt files are loaded using the safe unpickler instead of
the unrestricted pickle deserializer. This follows the PyTorch 2.6+
best practice of opting in to safe loading for all torch.load() calls.
2. SPA catch-all (app.py): replace str.startswith() path guard with
Path.is_relative_to(). The string prefix check passes for sibling
paths like /app/frontend_evil/ that share the /app/frontend prefix.
is_relative_to() correctly tests directory containment.
Replace verbose startup messages with a clean summary:
- App version, Python version, OS/arch
- Database path (fix None display), data directory
- Profile and generation counts
- Backend, GPU, model cache path
- Clean up stale loading_model status on startup
- Remove noisy progress manager log line
- Added `clear_profile_cache` function to manage cache files for specific profiles.
- Integrated cache clearing in `add_profile_sample`, `delete_profile`, and `delete_profile_sample` functions to ensure stale audio caches are invalidated after modifications.
- Enhanced `clear_voice_prompt_cache` to also delete combined audio files, improving overall cache management.
- Updated HistoryTable to implement infinite scrolling for loading history items dynamically.
- Introduced state management for accumulated history and total item count.
- Added Intersection Observer for triggering additional data fetches when scrolling.
- Implemented cache clearing functionality in the backend to manage voice prompt caches effectively.
- Improved loading indicators and user feedback for data fetching states.
- Refactored code for better readability and maintainability.
- Added support for MLX backend on Apple Silicon, enabling optimized performance for TTS and STT tasks.
- Updated release workflow to include MLX-specific dependencies and configurations for macOS platforms.
- Refactored backend code to dynamically select between MLX and PyTorch based on the runtime environment.
- Enhanced model loading and inference logic to accommodate backend-specific requirements, including updated model IDs and hidden imports.
- Improved health check and model status reporting to reflect the active backend type.
- Streamlined caching mechanisms to support both backend types, ensuring compatibility and performance.
