feat: add CosyVoice2/3 TTS engine with instruct and voice cloning

Integrate Alibaba's CosyVoice2-0.5B and Fun-CosyVoice3-0.5B as a new
TTS engine supporting 9 languages, zero-shot voice cloning, and instruct
control (emotions, speed, volume, dialects).

The CosyVoice source is cloned at setup time into backend/vendors/ since
no PyPI package exists. A modelscope→HuggingFace shim redirects model
downloads to the public HF repos, and a lightweight pylogger shim avoids
pulling in pytorch-lightning as a transitive dependency.

Backend: cosyvoice_backend.py, __init__.py registry, models.py regex
Frontend: engine selector, language map, Zod schema, model descriptions
Infra: requirements.txt, justfile, release.yml, Dockerfile, PyInstaller
This commit is contained in:
James Pine
2026-03-17 12:31:20 -07:00
parent c9f38dd496
commit f77dd621e2
15 changed files with 494 additions and 14 deletions
+11
View File
@@ -48,6 +48,12 @@ setup-python:
{{ pip }} install --no-deps chatterbox-tts
# HumeAI TADA pins torch>=2.7,<2.8 which conflicts with our torch>=2.1
{{ pip }} install --no-deps hume-tada
# CosyVoice: clone source into backend/vendors/ (no PyPI package exists)
if [ ! -d "{{ backend_dir }}/vendors/CosyVoice" ]; then
echo "Cloning CosyVoice source..."
mkdir -p {{ backend_dir }}/vendors
git clone --recursive --depth 1 https://github.com/FunAudioLLM/CosyVoice.git {{ backend_dir }}/vendors/CosyVoice
fi
# Apple Silicon: install MLX backend
if [ "$(uname -m)" = "arm64" ] && [ "$(uname)" = "Darwin" ]; then
echo "Detected Apple Silicon — installing MLX dependencies..."
@@ -77,6 +83,11 @@ setup-python:
& "{{ pip }}" install -r {{ backend_dir }}/requirements.txt
& "{{ pip }}" install --no-deps chatterbox-tts
& "{{ pip }}" install --no-deps hume-tada
if (-not (Test-Path "{{ backend_dir }}/vendors/CosyVoice")) { \
Write-Host "Cloning CosyVoice source..."; \
New-Item -ItemType Directory -Force -Path "{{ backend_dir }}/vendors" | Out-Null; \
git clone --recursive --depth 1 https://github.com/FunAudioLLM/CosyVoice.git "{{ backend_dir }}/vendors/CosyVoice"; \
}
& "{{ pip }}" install git+https://github.com/QwenLM/Qwen3-TTS.git
& "{{ pip }}" install pyinstaller ruff pytest pytest-asyncio -q
Write-Host "Python environment ready."