feat: add CosyVoice2/3 TTS engine with instruct and voice cloning

Integrate Alibaba's CosyVoice2-0.5B and Fun-CosyVoice3-0.5B as a new
TTS engine supporting 9 languages, zero-shot voice cloning, and instruct
control (emotions, speed, volume, dialects).

The CosyVoice source is cloned at setup time into backend/vendors/ since
no PyPI package exists. A modelscope→HuggingFace shim redirects model
downloads to the public HF repos, and a lightweight pylogger shim avoids
pulling in pytorch-lightning as a transitive dependency.

Backend: cosyvoice_backend.py, __init__.py registry, models.py regex
Frontend: engine selector, language map, Zod schema, model descriptions
Infra: requirements.txt, justfile, release.yml, Dockerfile, PyInstaller
This commit is contained in:
James Pine
2026-03-17 12:31:20 -07:00
parent c9f38dd496
commit f77dd621e2
15 changed files with 494 additions and 14 deletions
+31
View File
@@ -228,9 +228,40 @@ def build_server(cuda=False):
"torchaudio",
"--collect-submodules",
"tada",
# CosyVoice2/3 — Alibaba TTS with instruct + cloning
"--hidden-import",
"backend.backends.cosyvoice_backend",
# hyperpyyaml dynamically instantiates classes from YAML —
# needs source files and the ruamel.yaml backend
"--collect-all",
"hyperpyyaml",
# onnxruntime ships native shared libraries + provider plugins
"--collect-all",
"onnxruntime",
"--copy-metadata",
"onnxruntime",
# openai-whisper ships mel filter assets and uses tiktoken
"--collect-all",
"whisper",
"--collect-all",
"tiktoken",
# einops used by CosyVoice flow/decoder
"--hidden-import",
"einops",
]
)
# Bundle the vendored CosyVoice source tree for frozen builds.
# The clone lives at backend/vendors/CosyVoice/ at build time.
cosyvoice_vendor = backend_dir / "vendors" / "CosyVoice"
if cosyvoice_vendor.exists():
args.extend([
"--add-data",
f"{cosyvoice_vendor / 'cosyvoice'}{os.pathsep}cosyvoice",
"--add-data",
f"{cosyvoice_vendor / 'third_party' / 'Matcha-TTS' / 'matcha'}{os.pathsep}matcha",
])
# Add CUDA-specific hidden imports
if cuda:
logger.info("Building with CUDA support")