Initialize voicebox project with backend, frontend, and Tauri setup. Added configuration files, dependencies, and basic structure for components, hooks, and utilities. Included README and setup documentation for guidance.

backend/README.md

@@ -0,0 +1,438 @@
+# voicebox Backend
+
+Production-quality FastAPI backend for Qwen3-TTS voice cloning.
+
+## Features
+
+- ✅ **Voice Profile Management** - Create, update, delete voice profiles with multi-sample support
+- ✅ **Voice Cloning** - Generate speech using voice profiles with caching
+- ✅ **Generation History** - Full history tracking with search and filtering
+- ✅ **Transcription** - Whisper-based audio transcription
+- ✅ **Multi-Sample Profiles** - Combine multiple reference samples for better quality
+- ✅ **Voice Prompt Caching** - Dual memory + disk caching for fast generation
+- ✅ **Audio Validation** - Automatic validation of reference audio quality
+- ✅ **Model Management** - Lazy loading and VRAM management
+
+## Architecture
+
+```
+backend/
+├── main.py              # FastAPI app with all routes
+├── models.py            # Pydantic request/response models
+├── tts.py              # Qwen3-TTS inference
+├── transcribe.py       # Whisper ASR
+├── profiles.py         # Voice profile CRUD
+├── history.py          # Generation history
+├── studio.py           # Audio editing (TODO)
+├── database.py         # SQLite ORM
+└── utils/
+    ├── audio.py        # Audio processing utilities
+    ├── cache.py        # Voice prompt caching
+    └── validation.py   # Input validation
+```
+
+## API Endpoints
+
+### Health & Info
+
+#### `GET /`
+Root endpoint with version info.
+
+#### `GET /health`
+Health check with model status.
+
+**Response:**
+```json
+{
+  "status": "healthy",
+  "model_loaded": true,
+  "gpu_available": true,
+  "vram_used_mb": 1024.5
+}
+```
+
+### Voice Profiles
+
+#### `POST /profiles`
+Create a new voice profile.
+
+**Request:**
+```json
+{
+  "name": "My Voice",
+  "description": "Optional description",
+  "language": "en"
+}
+```
+
+**Response:**
+```json
+{
+  "id": "uuid",
+  "name": "My Voice",
+  "description": "Optional description",
+  "language": "en",
+  "created_at": "2024-01-01T00:00:00Z",
+  "updated_at": "2024-01-01T00:00:00Z"
+}
+```
+
+#### `GET /profiles`
+List all voice profiles.
+
+#### `GET /profiles/{profile_id}`
+Get a specific profile.
+
+#### `PUT /profiles/{profile_id}`
+Update a profile.
+
+#### `DELETE /profiles/{profile_id}`
+Delete a profile and all associated samples.
+
+#### `POST /profiles/{profile_id}/samples`
+Add a sample to a profile.
+
+**Form Data:**
+- `file`: Audio file (WAV, MP3, etc.)
+- `reference_text`: Transcript of the audio
+
+**Response:**
+```json
+{
+  "id": "sample-uuid",
+  "profile_id": "profile-uuid",
+  "audio_path": "/path/to/sample.wav",
+  "reference_text": "This is my voice"
+}
+```
+
+#### `GET /profiles/{profile_id}/samples`
+List all samples for a profile.
+
+#### `DELETE /profiles/samples/{sample_id}`
+Delete a specific sample.
+
+### Generation
+
+#### `POST /generate`
+Generate speech from text using a voice profile.
+
+**Request:**
+```json
+{
+  "profile_id": "uuid",
+  "text": "Hello, this is a test.",
+  "language": "en",
+  "seed": 42
+}
+```
+
+**Response:**
+```json
+{
+  "id": "generation-uuid",
+  "profile_id": "profile-uuid",
+  "text": "Hello, this is a test.",
+  "language": "en",
+  "audio_path": "/path/to/audio.wav",
+  "duration": 2.5,
+  "seed": 42,
+  "created_at": "2024-01-01T00:00:00Z"
+}
+```
+
+### History
+
+#### `GET /history`
+List generation history with optional filters.
+
+**Query Parameters:**
+- `profile_id` (optional): Filter by profile
+- `search` (optional): Search in text content
+- `limit` (default: 50): Results per page
+- `offset` (default: 0): Pagination offset
+
+#### `GET /history/{generation_id}`
+Get a specific generation.
+
+#### `DELETE /history/{generation_id}`
+Delete a generation.
+
+#### `GET /history/stats`
+Get generation statistics.
+
+**Response:**
+```json
+{
+  "total_generations": 100,
+  "total_duration_seconds": 250.5,
+  "generations_by_profile": {
+    "profile-uuid-1": 50,
+    "profile-uuid-2": 50
+  }
+}
+```
+
+### Audio Files
+
+#### `GET /audio/{generation_id}`
+Download generated audio file.
+
+Returns WAV file with appropriate headers.
+
+### Transcription
+
+#### `POST /transcribe`
+Transcribe audio file to text.
+
+**Form Data:**
+- `file`: Audio file
+- `language` (optional): Language hint (en or zh)
+
+**Response:**
+```json
+{
+  "text": "Transcribed text here",
+  "duration": 5.5
+}
+```
+
+### Model Management
+
+#### `POST /models/load`
+Manually load TTS model.
+
+**Query Parameters:**
+- `model_size`: Model size (1.7B or 0.6B)
+
+#### `POST /models/unload`
+Unload TTS model to free memory.
+
+## Database Schema
+
+### profiles
+- `id`: UUID primary key
+- `name`: Profile name (unique)
+- `description`: Optional description
+- `language`: Language code (en/zh)
+- `created_at`: Creation timestamp
+- `updated_at`: Last update timestamp
+
+### profile_samples
+- `id`: UUID primary key
+- `profile_id`: Foreign key to profiles
+- `audio_path`: Path to audio file
+- `reference_text`: Transcript
+
+### generations
+- `id`: UUID primary key
+- `profile_id`: Foreign key to profiles
+- `text`: Generated text
+- `language`: Language code
+- `audio_path`: Path to audio file
+- `duration`: Duration in seconds
+- `seed`: Random seed (optional)
+- `created_at`: Creation timestamp
+
+### projects
+- `id`: UUID primary key
+- `name`: Project name
+- `data`: JSON data
+- `created_at`: Creation timestamp
+- `updated_at`: Last update timestamp
+
+## File Structure
+
+```
+data/
+├── profiles/
+│   └── {profile_id}/
+│       ├── {sample_id}.wav
+│       └── ...
+├── generations/
+│   └── {generation_id}.wav
+├── cache/
+│   └── {hash}.prompt
+├── projects/
+│   └── {project_id}.json
+└── voicebox.db
+```
+
+## Setup
+
+### 1. Install Dependencies
+
+```bash
+pip install -r requirements.txt
+```
+
+### 2. Initialize Database
+
+```bash
+python -c "from database import init_db; init_db()"
+```
+
+### 3. Download Models (Automatic)
+
+The Qwen3-TTS models are automatically downloaded from HuggingFace Hub on first use, similar to how Whisper models work.
+
+**No manual download required!** The models will be cached locally after the first download.
+
+Available models:
+- **1.7B** (recommended): `Qwen/Qwen3-TTS-12Hz-1.7B-Base` (~4GB)
+- **0.6B** (faster): `Qwen/Qwen3-TTS-12Hz-0.6B-Base` (~2GB)
+
+**Note:** The first generation will take longer as the model downloads. Subsequent generations will use the cached model.
+
+#### Manual Download (Optional)
+
+If you prefer to download models manually or have limited internet during runtime:
+
+```bash
+# Install huggingface-cli
+pip install huggingface_hub
+
+# Download 1.7B model
+huggingface-cli download Qwen/Qwen3-TTS-12Hz-1.7B-Base
+
+# Or use Python
+python -c "from huggingface_hub import snapshot_download; snapshot_download('Qwen/Qwen3-TTS-12Hz-1.7B-Base')"
+```
+
+Models are cached in `~/.cache/huggingface/hub/` by default.
+
+### 4. Run Server
+
+```bash
+# Development (local only)
+python -m backend.main
+
+# Production (allow remote access)
+python -m backend.main --host 0.0.0.0 --port 8000
+```
+
+## Usage Examples
+
+### Creating a Voice Profile
+
+```bash
+# 1. Create profile
+curl -X POST http://localhost:8000/profiles \
+  -H "Content-Type: application/json" \
+  -d '{"name": "My Voice", "language": "en"}'
+
+# Response: {"id": "abc-123", ...}
+
+# 2. Add sample
+curl -X POST http://localhost:8000/profiles/abc-123/samples \
+  -F "[email protected]" \
+  -F "reference_text=This is my voice sample"
+```
+
+### Generating Speech
+
+```bash
+curl -X POST http://localhost:8000/generate \
+  -H "Content-Type: application/json" \
+  -d '{
+    "profile_id": "abc-123",
+    "text": "Hello, this is a test.",
+    "language": "en",
+    "seed": 42
+  }'
+
+# Response: {"id": "gen-456", "audio_path": "/path/to/audio.wav", ...}
+
+# Download audio
+curl http://localhost:8000/audio/gen-456 -o output.wav
+```
+
+### Transcribing Audio
+
+```bash
+curl -X POST http://localhost:8000/transcribe \
+  -F "[email protected]" \
+  -F "language=en"
+
+# Response: {"text": "Transcribed text", "duration": 5.5}
+```
+
+## Advanced Features
+
+### Multi-Sample Profiles
+
+Add multiple samples to a profile for better quality:
+
+```bash
+# Add first sample
+curl -X POST http://localhost:8000/profiles/abc-123/samples \
+  -F "[email protected]" \
+  -F "reference_text=First sample"
+
+# Add second sample
+curl -X POST http://localhost:8000/profiles/abc-123/samples \
+  -F "[email protected]" \
+  -F "reference_text=Second sample"
+
+# Generation will automatically combine all samples
+```
+
+### Voice Prompt Caching
+
+Voice prompts are automatically cached for faster generation:
+- First generation: ~5-10 seconds (creates prompt)
+- Subsequent generations: ~1-2 seconds (uses cached prompt)
+
+Cache is stored in `data/cache/` and persists across server restarts.
+
+### VRAM Management
+
+Models are lazy-loaded and can be manually unloaded:
+
+```bash
+# Unload TTS model
+curl -X POST http://localhost:8000/models/unload
+
+# Load specific model size
+curl -X POST "http://localhost:8000/models/load?model_size=0.6B"
+```
+
+## Error Handling
+
+All endpoints return proper HTTP status codes:
+
+- `200 OK`: Success
+- `400 Bad Request`: Invalid input
+- `404 Not Found`: Resource not found
+- `500 Internal Server Error`: Server error
+
+Error responses include details:
+
+```json
+{
+  "detail": "Profile not found"
+}
+```
+
+## Performance Tips
+
+1. **Use multi-sample profiles** - Better quality than single sample
+2. **Let caching work** - Voice prompts are cached automatically
+3. **Use 0.6B model on CPU** - Faster than 1.7B with acceptable quality
+4. **Use 1.7B model on GPU** - Best quality, still fast
+5. **Unload Whisper after transcription** - Frees VRAM for TTS
+
+## TODO
+
+- [ ] WebSocket support for generation progress
+- [ ] Batch generation endpoint
+- [ ] Audio effects (M3GAN, etc.)
+- [ ] Voice design (text-to-voice)
+- [ ] Audio studio timeline features
+- [ ] Project management
+- [ ] Authentication & rate limiting
+- [ ] Export/import profiles
+
+## License
+
+See main project LICENSE.

backend/__init__.py

@@ -0,0 +1 @@
+# Backend package

backend/build_binary.py

@@ -0,0 +1,43 @@
+"""
+PyInstaller build script for creating standalone Python server binary.
+"""
+
+import PyInstaller.__main__
+import sys
+import os
+from pathlib import Path
+
+
+def build_server():
+    """Build Python server as standalone binary."""
+    backend_dir = Path(__file__).parent
+    
+    # PyInstaller arguments
+    args = [
+        'main.py',
+        '--onefile',
+        '--name', 'voicebox-server',
+        '--add-data', f'utils{os.pathsep}utils',  # Include utils package
+        '--hidden-import', 'torch',
+        '--hidden-import', 'transformers',
+        '--hidden-import', 'fastapi',
+        '--hidden-import', 'uvicorn',
+        '--hidden-import', 'sqlalchemy',
+        '--hidden-import', 'librosa',
+        '--hidden-import', 'soundfile',
+        '--collect-all', 'qwen-tts',
+        '--noconfirm',
+        '--clean',
+    ]
+    
+    # Change to backend directory
+    os.chdir(backend_dir)
+    
+    # Run PyInstaller
+    PyInstaller.__main__.run(args)
+    
+    print(f"Binary built in {backend_dir / 'dist' / 'voicebox-server'}")
+
+
+if __name__ == '__main__':
+    build_server()

backend/database.py

@@ -0,0 +1,85 @@
+"""
+SQLite database ORM using SQLAlchemy.
+"""
+
+from sqlalchemy import create_engine, Column, String, Integer, Float, DateTime, Text, ForeignKey
+from sqlalchemy.ext.declarative import declarative_base
+from sqlalchemy.orm import sessionmaker, Session
+from datetime import datetime
+import uuid
+from pathlib import Path
+
+Base = declarative_base()
+
+
+class VoiceProfile(Base):
+    """Voice profile database model."""
+    __tablename__ = "profiles"
+    
+    id = Column(String, primary_key=True, default=lambda: str(uuid.uuid4()))
+    name = Column(String, unique=True, nullable=False)
+    description = Column(Text)
+    language = Column(String, default="en")
+    created_at = Column(DateTime, default=datetime.utcnow)
+    updated_at = Column(DateTime, default=datetime.utcnow, onupdate=datetime.utcnow)
+
+
+class ProfileSample(Base):
+    """Voice profile sample database model."""
+    __tablename__ = "profile_samples"
+    
+    id = Column(String, primary_key=True, default=lambda: str(uuid.uuid4()))
+    profile_id = Column(String, ForeignKey("profiles.id"), nullable=False)
+    audio_path = Column(String, nullable=False)
+    reference_text = Column(Text, nullable=False)
+
+
+class Generation(Base):
+    """Generation history database model."""
+    __tablename__ = "generations"
+    
+    id = Column(String, primary_key=True, default=lambda: str(uuid.uuid4()))
+    profile_id = Column(String, ForeignKey("profiles.id"), nullable=False)
+    text = Column(Text, nullable=False)
+    language = Column(String, default="en")
+    audio_path = Column(String, nullable=False)
+    duration = Column(Float, nullable=False)
+    seed = Column(Integer)
+    created_at = Column(DateTime, default=datetime.utcnow)
+
+
+class Project(Base):
+    """Audio studio project database model."""
+    __tablename__ = "projects"
+    
+    id = Column(String, primary_key=True, default=lambda: str(uuid.uuid4()))
+    name = Column(String, nullable=False)
+    data = Column(Text)  # JSON string
+    created_at = Column(DateTime, default=datetime.utcnow)
+    updated_at = Column(DateTime, default=datetime.utcnow, onupdate=datetime.utcnow)
+
+
+# Database setup
+_db_path = Path("data/voicebox.db")
+_db_path.parent.mkdir(parents=True, exist_ok=True)
+
+engine = create_engine(
+    f"sqlite:///{_db_path}",
+    connect_args={"check_same_thread": False},
+)
+
+SessionLocal = sessionmaker(autocommit=False, autoflush=False, bind=engine)
+
+
+def init_db():
+    """Initialize database tables."""
+    Base.metadata.create_all(bind=engine)
+
+
+def get_db():
+    """Get database session (generator for dependency injection)."""
+    db = SessionLocal()
+    try:
+        yield db
+    finally:
+        db.close()

backend/example_usage.py

@@ -0,0 +1,221 @@
+"""
+Example usage of the voicebox backend API.
+
+This script demonstrates how to:
+1. Create a voice profile
+2. Add samples to the profile
+3. Generate speech
+4. List history
+"""
+
+import requests
+import time
+from pathlib import Path
+
+# API base URL
+BASE_URL = "http://localhost:8000"
+
+
+def check_health():
+    """Check if the server is running."""
+    response = requests.get(f"{BASE_URL}/health")
+    data = response.json()
+    print(f"Server status: {data['status']}")
+    print(f"Model loaded: {data['model_loaded']}")
+    print(f"GPU available: {data['gpu_available']}")
+    print()
+    return data
+
+
+def create_profile(name: str, description: str = None, language: str = "en"):
+    """Create a new voice profile."""
+    response = requests.post(
+        f"{BASE_URL}/profiles",
+        json={
+            "name": name,
+            "description": description,
+            "language": language,
+        },
+    )
+    response.raise_for_status()
+    profile = response.json()
+    print(f"Created profile: {profile['name']} (ID: {profile['id']})")
+    return profile
+
+
+def add_sample(profile_id: str, audio_file: str, reference_text: str):
+    """Add a sample to a voice profile."""
+    with open(audio_file, "rb") as f:
+        files = {"file": f}
+        data = {"reference_text": reference_text}
+        response = requests.post(
+            f"{BASE_URL}/profiles/{profile_id}/samples",
+            files=files,
+            data=data,
+        )
+    response.raise_for_status()
+    sample = response.json()
+    print(f"Added sample: {sample['id']}")
+    return sample
+
+
+def generate_speech(profile_id: str, text: str, language: str = "en", seed: int = None):
+    """Generate speech using a voice profile."""
+    print(f"Generating speech: '{text[:50]}...'")
+    start_time = time.time()
+    
+    response = requests.post(
+        f"{BASE_URL}/generate",
+        json={
+            "profile_id": profile_id,
+            "text": text,
+            "language": language,
+            "seed": seed,
+        },
+    )
+    response.raise_for_status()
+    generation = response.json()
+    
+    elapsed = time.time() - start_time
+    print(f"Generated in {elapsed:.2f}s (duration: {generation['duration']:.2f}s)")
+    print(f"Generation ID: {generation['id']}")
+    return generation
+
+
+def download_audio(generation_id: str, output_file: str):
+    """Download generated audio."""
+    response = requests.get(f"{BASE_URL}/audio/{generation_id}")
+    response.raise_for_status()
+    
+    with open(output_file, "wb") as f:
+        f.write(response.content)
+    
+    print(f"Saved audio to: {output_file}")
+
+
+def list_profiles():
+    """List all voice profiles."""
+    response = requests.get(f"{BASE_URL}/profiles")
+    response.raise_for_status()
+    profiles = response.json()
+    
+    print(f"Found {len(profiles)} profiles:")
+    for profile in profiles:
+        print(f"  - {profile['name']} (ID: {profile['id']})")
+    
+    return profiles
+
+
+def list_history(profile_id: str = None, limit: int = 10):
+    """List generation history."""
+    params = {"limit": limit}
+    if profile_id:
+        params["profile_id"] = profile_id
+    
+    response = requests.get(f"{BASE_URL}/history", params=params)
+    response.raise_for_status()
+    history = response.json()
+    
+    print(f"Found {len(history)} generations:")
+    for gen in history:
+        print(f"  - {gen['text'][:50]}... ({gen['duration']:.2f}s)")
+    
+    return history
+
+
+def transcribe_audio(audio_file: str, language: str = None):
+    """Transcribe audio file."""
+    print(f"Transcribing: {audio_file}")
+    
+    with open(audio_file, "rb") as f:
+        files = {"file": f}
+        data = {}
+        if language:
+            data["language"] = language
+        
+        response = requests.post(
+            f"{BASE_URL}/transcribe",
+            files=files,
+            data=data,
+        )
+    
+    response.raise_for_status()
+    result = response.json()
+    
+    print(f"Transcription: {result['text']}")
+    print(f"Duration: {result['duration']:.2f}s")
+    return result
+
+
+def main():
+    """Run example workflow."""
+    print("=" * 60)
+    print("voicebox Backend API Example")
+    print("=" * 60)
+    print()
+    
+    # 1. Check health
+    print("1. Checking server health...")
+    check_health()
+    
+    # 2. Create a profile
+    print("2. Creating voice profile...")
+    profile = create_profile(
+        name="Example Voice",
+        description="A test voice profile",
+        language="en",
+    )
+    profile_id = profile["id"]
+    print()
+    
+    # 3. Add samples (you'll need actual audio files)
+    print("3. Adding samples...")
+    print("   (Skipping - add your own audio files here)")
+    # Uncomment and add your audio file:
+    # sample = add_sample(
+    #     profile_id,
+    #     "path/to/your/sample.wav",
+    #     "This is the transcript of the audio",
+    # )
+    print()
+    
+    # 4. Generate speech (requires samples to be added first)
+    print("4. Generating speech...")
+    print("   (Skipping - add samples first)")
+    # Uncomment after adding samples:
+    # generation = generate_speech(
+    #     profile_id,
+    #     "Hello, this is a test of the voice cloning system.",
+    #     language="en",
+    #     seed=42,
+    # )
+    # 
+    # # 5. Download audio
+    # print("\n5. Downloading audio...")
+    # download_audio(generation["id"], "output.wav")
+    print()
+    
+    # 6. List profiles
+    print("6. Listing all profiles...")
+    list_profiles()
+    print()
+    
+    # 7. List history
+    print("7. Listing generation history...")
+    list_history(limit=5)
+    print()
+    
+    # 8. Transcribe audio (you'll need an audio file)
+    print("8. Transcribing audio...")
+    print("   (Skipping - add your own audio file here)")
+    # Uncomment and add your audio file:
+    # transcribe_audio("path/to/audio.wav", language="en")
+    print()
+    
+    print("=" * 60)
+    print("Example complete!")
+    print("=" * 60)
+
+
+if __name__ == "__main__":
+    main()

backend/history.py

@@ -0,0 +1,219 @@
+"""
+Generation history management module.
+"""
+
+from typing import List, Optional, Tuple
+from datetime import datetime
+import uuid
+import shutil
+from pathlib import Path
+from sqlalchemy.orm import Session
+from sqlalchemy import or_
+
+from .models import GenerationRequest, GenerationResponse, HistoryQuery
+from .database import Generation as DBGeneration
+
+
+# Generations storage directory
+GENERATIONS_DIR = Path("data/generations")
+GENERATIONS_DIR.mkdir(parents=True, exist_ok=True)
+
+
+async def create_generation(
+    profile_id: str,
+    text: str,
+    language: str,
+    audio_path: str,
+    duration: float,
+    seed: Optional[int],
+    db: Session,
+) -> GenerationResponse:
+    """
+    Create a new generation history entry.
+    
+    Args:
+        profile_id: Profile ID used for generation
+        text: Generated text
+        language: Language code
+        audio_path: Path where audio was saved
+        duration: Audio duration in seconds
+        seed: Random seed used (if any)
+        db: Database session
+        
+    Returns:
+        Created generation entry
+    """
+    db_generation = DBGeneration(
+        id=str(uuid.uuid4()),
+        profile_id=profile_id,
+        text=text,
+        language=language,
+        audio_path=audio_path,
+        duration=duration,
+        seed=seed,
+        created_at=datetime.utcnow(),
+    )
+    
+    db.add(db_generation)
+    db.commit()
+    db.refresh(db_generation)
+    
+    return GenerationResponse.model_validate(db_generation)
+
+
+async def get_generation(
+    generation_id: str,
+    db: Session,
+) -> Optional[GenerationResponse]:
+    """
+    Get a generation by ID.
+    
+    Args:
+        generation_id: Generation ID
+        db: Database session
+        
+    Returns:
+        Generation or None if not found
+    """
+    generation = db.query(DBGeneration).filter_by(id=generation_id).first()
+    if not generation:
+        return None
+    
+    return GenerationResponse.model_validate(generation)
+
+
+async def list_generations(
+    query: HistoryQuery,
+    db: Session,
+) -> Tuple[List[GenerationResponse], int]:
+    """
+    List generations with optional filters.
+    
+    Args:
+        query: Query parameters (filters, pagination)
+        db: Database session
+        
+    Returns:
+        Tuple of (generations, total_count)
+    """
+    # Build base query
+    q = db.query(DBGeneration)
+    
+    # Apply profile filter
+    if query.profile_id:
+        q = q.filter(DBGeneration.profile_id == query.profile_id)
+    
+    # Apply search filter (searches in text content)
+    if query.search:
+        search_pattern = f"%{query.search}%"
+        q = q.filter(DBGeneration.text.like(search_pattern))
+    
+    # Get total count before pagination
+    total_count = q.count()
+    
+    # Apply ordering (newest first)
+    q = q.order_by(DBGeneration.created_at.desc())
+    
+    # Apply pagination
+    q = q.offset(query.offset).limit(query.limit)
+    
+    # Execute query
+    generations = q.all()
+    
+    return (
+        [GenerationResponse.model_validate(g) for g in generations],
+        total_count,
+    )
+
+
+async def delete_generation(
+    generation_id: str,
+    db: Session,
+) -> bool:
+    """
+    Delete a generation.
+    
+    Args:
+        generation_id: Generation ID
+        db: Database session
+        
+    Returns:
+        True if deleted, False if not found
+    """
+    generation = db.query(DBGeneration).filter_by(id=generation_id).first()
+    if not generation:
+        return False
+    
+    # Delete audio file
+    audio_path = Path(generation.audio_path)
+    if audio_path.exists():
+        audio_path.unlink()
+    
+    # Delete from database
+    db.delete(generation)
+    db.commit()
+    
+    return True
+
+
+async def delete_generations_by_profile(
+    profile_id: str,
+    db: Session,
+) -> int:
+    """
+    Delete all generations for a profile.
+    
+    Args:
+        profile_id: Profile ID
+        db: Database session
+        
+    Returns:
+        Number of generations deleted
+    """
+    generations = db.query(DBGeneration).filter_by(profile_id=profile_id).all()
+    
+    count = 0
+    for generation in generations:
+        # Delete audio file
+        audio_path = Path(generation.audio_path)
+        if audio_path.exists():
+            audio_path.unlink()
+        
+        # Delete from database
+        db.delete(generation)
+        count += 1
+    
+    db.commit()
+    
+    return count
+
+
+async def get_generation_stats(db: Session) -> dict:
+    """
+    Get generation statistics.
+    
+    Args:
+        db: Database session
+        
+    Returns:
+        Statistics dictionary
+    """
+    from sqlalchemy import func
+    
+    total = db.query(func.count(DBGeneration.id)).scalar()
+    
+    total_duration = db.query(func.sum(DBGeneration.duration)).scalar() or 0
+    
+    # Get generations by profile
+    by_profile = db.query(
+        DBGeneration.profile_id,
+        func.count(DBGeneration.id).label('count')
+    ).group_by(DBGeneration.profile_id).all()
+    
+    return {
+        "total_generations": total,
+        "total_duration_seconds": total_duration,
+        "generations_by_profile": {
+            profile_id: count for profile_id, count in by_profile
+        },
+    }

backend/main.py

@@ -0,0 +1,423 @@
+"""
+FastAPI application for voicebox backend.
+
+Handles voice cloning, generation history, and server mode.
+"""
+
+from fastapi import FastAPI, Depends, UploadFile, File, Form, HTTPException
+from fastapi.middleware.cors import CORSMiddleware
+from fastapi.responses import FileResponse
+from fastapi.staticfiles import StaticFiles
+from sqlalchemy.orm import Session
+from typing import List, Optional
+import uvicorn
+import argparse
+import torch
+import tempfile
+from pathlib import Path
+import uuid
+
+from . import database, models, profiles, history, tts, transcribe
+from .database import get_db, init_db
+
+# Initialize database
+init_db()
+
+app = FastAPI(
+    title="voicebox API",
+    description="Production-quality Qwen3-TTS voice cloning API",
+    version="0.1.0",
+)
+
+# CORS middleware
+app.add_middleware(
+    CORSMiddleware,
+    allow_origins=["*"],  # Configure appropriately for production
+    allow_credentials=True,
+    allow_methods=["*"],
+    allow_headers=["*"],
+)
+
+
+# ============================================
+# ROOT & HEALTH ENDPOINTS
+# ============================================
+
+@app.get("/")
+async def root():
+    """Root endpoint."""
+    return {"message": "voicebox API", "version": "0.1.0"}
+
+
+@app.get("/health", response_model=models.HealthResponse)
+async def health():
+    """Health check endpoint."""
+    tts_model = tts.get_tts_model()
+    gpu_available = torch.cuda.is_available()
+    
+    vram_used = None
+    if gpu_available:
+        vram_used = torch.cuda.memory_allocated() / 1024 / 1024  # MB
+    
+    return models.HealthResponse(
+        status="healthy",
+        model_loaded=tts_model.is_loaded(),
+        gpu_available=gpu_available,
+        vram_used_mb=vram_used,
+    )
+
+
+# ============================================
+# VOICE PROFILE ENDPOINTS
+# ============================================
+
+@app.post("/profiles", response_model=models.VoiceProfileResponse)
+async def create_profile(
+    data: models.VoiceProfileCreate,
+    db: Session = Depends(get_db),
+):
+    """Create a new voice profile."""
+    try:
+        return await profiles.create_profile(data, db)
+    except Exception as e:
+        raise HTTPException(status_code=400, detail=str(e))
+
+
+@app.get("/profiles", response_model=List[models.VoiceProfileResponse])
+async def list_profiles(db: Session = Depends(get_db)):
+    """List all voice profiles."""
+    return await profiles.list_profiles(db)
+
+
+@app.get("/profiles/{profile_id}", response_model=models.VoiceProfileResponse)
+async def get_profile(
+    profile_id: str,
+    db: Session = Depends(get_db),
+):
+    """Get a voice profile by ID."""
+    profile = await profiles.get_profile(profile_id, db)
+    if not profile:
+        raise HTTPException(status_code=404, detail="Profile not found")
+    return profile
+
+
+@app.put("/profiles/{profile_id}", response_model=models.VoiceProfileResponse)
+async def update_profile(
+    profile_id: str,
+    data: models.VoiceProfileCreate,
+    db: Session = Depends(get_db),
+):
+    """Update a voice profile."""
+    profile = await profiles.update_profile(profile_id, data, db)
+    if not profile:
+        raise HTTPException(status_code=404, detail="Profile not found")
+    return profile
+
+
+@app.delete("/profiles/{profile_id}")
+async def delete_profile(
+    profile_id: str,
+    db: Session = Depends(get_db),
+):
+    """Delete a voice profile."""
+    success = await profiles.delete_profile(profile_id, db)
+    if not success:
+        raise HTTPException(status_code=404, detail="Profile not found")
+    return {"message": "Profile deleted successfully"}
+
+
+@app.post("/profiles/{profile_id}/samples", response_model=models.ProfileSampleResponse)
+async def add_profile_sample(
+    profile_id: str,
+    file: UploadFile = File(...),
+    reference_text: str = Form(...),
+    db: Session = Depends(get_db),
+):
+    """Add a sample to a voice profile."""
+    # Save uploaded file to temporary location
+    with tempfile.NamedTemporaryFile(suffix=".wav", delete=False) as tmp:
+        content = await file.read()
+        tmp.write(content)
+        tmp_path = tmp.name
+    
+    try:
+        sample = await profiles.add_profile_sample(
+            profile_id,
+            tmp_path,
+            reference_text,
+            db,
+        )
+        return sample
+    except ValueError as e:
+        raise HTTPException(status_code=400, detail=str(e))
+    finally:
+        # Clean up temp file
+        Path(tmp_path).unlink(missing_ok=True)
+
+
+@app.get("/profiles/{profile_id}/samples", response_model=List[models.ProfileSampleResponse])
+async def get_profile_samples(
+    profile_id: str,
+    db: Session = Depends(get_db),
+):
+    """Get all samples for a profile."""
+    return await profiles.get_profile_samples(profile_id, db)
+
+
+@app.delete("/profiles/samples/{sample_id}")
+async def delete_profile_sample(
+    sample_id: str,
+    db: Session = Depends(get_db),
+):
+    """Delete a profile sample."""
+    success = await profiles.delete_profile_sample(sample_id, db)
+    if not success:
+        raise HTTPException(status_code=404, detail="Sample not found")
+    return {"message": "Sample deleted successfully"}
+
+
+# ============================================
+# GENERATION ENDPOINTS
+# ============================================
+
+@app.post("/generate", response_model=models.GenerationResponse)
+async def generate_speech(
+    data: models.GenerationRequest,
+    db: Session = Depends(get_db),
+):
+    """Generate speech from text using a voice profile."""
+    try:
+        # Get profile
+        profile = await profiles.get_profile(data.profile_id, db)
+        if not profile:
+            raise HTTPException(status_code=404, detail="Profile not found")
+        
+        # Create voice prompt from profile
+        voice_prompt = await profiles.create_voice_prompt_for_profile(
+            data.profile_id,
+            db,
+        )
+        
+        # Generate audio
+        tts_model = tts.get_tts_model()
+        audio, sample_rate = await tts_model.generate(
+            data.text,
+            voice_prompt,
+            data.language,
+            data.seed,
+        )
+        
+        # Calculate duration
+        duration = len(audio) / sample_rate
+        
+        # Save audio
+        generation_id = str(uuid.uuid4())
+        audio_path = history.GENERATIONS_DIR / f"{generation_id}.wav"
+        
+        from .utils.audio import save_audio
+        save_audio(audio, str(audio_path), sample_rate)
+        
+        # Create history entry
+        generation = await history.create_generation(
+            profile_id=data.profile_id,
+            text=data.text,
+            language=data.language,
+            audio_path=str(audio_path),
+            duration=duration,
+            seed=data.seed,
+            db=db,
+        )
+        
+        return generation
+        
+    except ValueError as e:
+        raise HTTPException(status_code=400, detail=str(e))
+    except Exception as e:
+        raise HTTPException(status_code=500, detail=str(e))
+
+
+# ============================================
+# HISTORY ENDPOINTS
+# ============================================
+
+@app.get("/history", response_model=List[models.GenerationResponse])
+async def list_history(
+    profile_id: Optional[str] = None,
+    search: Optional[str] = None,
+    limit: int = 50,
+    offset: int = 0,
+    db: Session = Depends(get_db),
+):
+    """List generation history with optional filters."""
+    query = models.HistoryQuery(
+        profile_id=profile_id,
+        search=search,
+        limit=limit,
+        offset=offset,
+    )
+    generations, total = await history.list_generations(query, db)
+    return generations
+
+
+@app.get("/history/{generation_id}", response_model=models.GenerationResponse)
+async def get_generation(
+    generation_id: str,
+    db: Session = Depends(get_db),
+):
+    """Get a generation by ID."""
+    generation = await history.get_generation(generation_id, db)
+    if not generation:
+        raise HTTPException(status_code=404, detail="Generation not found")
+    return generation
+
+
+@app.delete("/history/{generation_id}")
+async def delete_generation(
+    generation_id: str,
+    db: Session = Depends(get_db),
+):
+    """Delete a generation."""
+    success = await history.delete_generation(generation_id, db)
+    if not success:
+        raise HTTPException(status_code=404, detail="Generation not found")
+    return {"message": "Generation deleted successfully"}
+
+
+@app.get("/history/stats")
+async def get_stats(db: Session = Depends(get_db)):
+    """Get generation statistics."""
+    return await history.get_generation_stats(db)
+
+
+# ============================================
+# TRANSCRIPTION ENDPOINTS
+# ============================================
+
+@app.post("/transcribe", response_model=models.TranscriptionResponse)
+async def transcribe_audio(
+    file: UploadFile = File(...),
+    language: Optional[str] = Form(None),
+):
+    """Transcribe audio file to text."""
+    # Save uploaded file to temporary location
+    with tempfile.NamedTemporaryFile(suffix=".wav", delete=False) as tmp:
+        content = await file.read()
+        tmp.write(content)
+        tmp_path = tmp.name
+    
+    try:
+        # Get audio duration
+        from .utils.audio import load_audio
+        audio, sr = load_audio(tmp_path)
+        duration = len(audio) / sr
+        
+        # Transcribe
+        whisper_model = transcribe.get_whisper_model()
+        text = await whisper_model.transcribe(tmp_path, language)
+        
+        return models.TranscriptionResponse(
+            text=text,
+            duration=duration,
+        )
+        
+    except Exception as e:
+        raise HTTPException(status_code=500, detail=str(e))
+    finally:
+        # Clean up temp file
+        Path(tmp_path).unlink(missing_ok=True)
+
+
+# ============================================
+# FILE SERVING
+# ============================================
+
+@app.get("/audio/{generation_id}")
+async def get_audio(generation_id: str, db: Session = Depends(get_db)):
+    """Serve generated audio file."""
+    generation = await history.get_generation(generation_id, db)
+    if not generation:
+        raise HTTPException(status_code=404, detail="Generation not found")
+    
+    audio_path = Path(generation.audio_path)
+    if not audio_path.exists():
+        raise HTTPException(status_code=404, detail="Audio file not found")
+    
+    return FileResponse(
+        audio_path,
+        media_type="audio/wav",
+        filename=f"generation_{generation_id}.wav",
+    )
+
+
+# ============================================
+# MODEL MANAGEMENT
+# ============================================
+
+@app.post("/models/load")
+async def load_model(model_size: str = "1.7B"):
+    """Manually load TTS model."""
+    try:
+        tts_model = tts.get_tts_model()
+        tts_model.load_model(model_size)
+        return {"message": f"Model {model_size} loaded successfully"}
+    except Exception as e:
+        raise HTTPException(status_code=500, detail=str(e))
+
+
+@app.post("/models/unload")
+async def unload_model():
+    """Unload TTS model to free memory."""
+    try:
+        tts.unload_tts_model()
+        return {"message": "Model unloaded successfully"}
+    except Exception as e:
+        raise HTTPException(status_code=500, detail=str(e))
+
+
+# ============================================
+# STARTUP & SHUTDOWN
+# ============================================
+
+@app.on_event("startup")
+async def startup_event():
+    """Run on application startup."""
+    print("voicebox API starting up...")
+    print(f"Database initialized at {database._db_path}")
+    print(f"GPU available: {torch.cuda.is_available()}")
+
+
+@app.on_event("shutdown")
+async def shutdown_event():
+    """Run on application shutdown."""
+    print("voicebox API shutting down...")
+    # Unload models to free memory
+    tts.unload_tts_model()
+    transcribe.unload_whisper_model()
+
+
+# ============================================
+# MAIN
+# ============================================
+
+if __name__ == "__main__":
+    parser = argparse.ArgumentParser(description="voicebox backend server")
+    parser.add_argument(
+        "--host",
+        type=str,
+        default="127.0.0.1",
+        help="Host to bind to (use 0.0.0.0 for remote access)",
+    )
+    parser.add_argument(
+        "--port",
+        type=int,
+        default=8000,
+        help="Port to bind to",
+    )
+    args = parser.parse_args()
+
+    uvicorn.run(
+        "main:app",
+        host=args.host,
+        port=args.port,
+        reload=False,  # Disable reload in production
+    )

backend/models.py

@@ -0,0 +1,93 @@
+"""
+Pydantic models for request/response validation.
+"""
+
+from pydantic import BaseModel, Field
+from typing import Optional, List
+from datetime import datetime
+
+
+class VoiceProfileCreate(BaseModel):
+    """Request model for creating a voice profile."""
+    name: str = Field(..., min_length=1, max_length=100)
+    description: Optional[str] = Field(None, max_length=500)
+    language: str = Field(default="en", pattern="^(en|zh)$")
+
+
+class VoiceProfileResponse(BaseModel):
+    """Response model for voice profile."""
+    id: str
+    name: str
+    description: Optional[str]
+    language: str
+    created_at: datetime
+    updated_at: datetime
+
+    class Config:
+        from_attributes = True
+
+
+class ProfileSampleCreate(BaseModel):
+    """Request model for adding a sample to a profile."""
+    reference_text: str = Field(..., min_length=1, max_length=1000)
+
+
+class ProfileSampleResponse(BaseModel):
+    """Response model for profile sample."""
+    id: str
+    profile_id: str
+    audio_path: str
+    reference_text: str
+
+    class Config:
+        from_attributes = True
+
+
+class GenerationRequest(BaseModel):
+    """Request model for voice generation."""
+    profile_id: str
+    text: str = Field(..., min_length=1, max_length=5000)
+    language: str = Field(default="en", pattern="^(en|zh)$")
+    seed: Optional[int] = Field(None, ge=0)
+
+
+class GenerationResponse(BaseModel):
+    """Response model for voice generation."""
+    id: str
+    profile_id: str
+    text: str
+    language: str
+    audio_path: str
+    duration: float
+    seed: Optional[int]
+    created_at: datetime
+
+    class Config:
+        from_attributes = True
+
+
+class HistoryQuery(BaseModel):
+    """Query model for generation history."""
+    profile_id: Optional[str] = None
+    search: Optional[str] = None
+    limit: int = Field(default=50, ge=1, le=100)
+    offset: int = Field(default=0, ge=0)
+
+
+class TranscriptionRequest(BaseModel):
+    """Request model for audio transcription."""
+    language: Optional[str] = Field(None, pattern="^(en|zh)$")
+
+
+class TranscriptionResponse(BaseModel):
+    """Response model for transcription."""
+    text: str
+    duration: float
+
+
+class HealthResponse(BaseModel):
+    """Response model for health check."""
+    status: str
+    model_loaded: bool
+    gpu_available: bool
+    vram_used_mb: Optional[float] = None

backend/profiles.py

@@ -0,0 +1,335 @@
+"""
+Voice profile management module.
+"""
+
+from typing import List, Optional
+from datetime import datetime
+import uuid
+import shutil
+from pathlib import Path
+from sqlalchemy.orm import Session
+from sqlalchemy import select
+
+from .models import (
+    VoiceProfileCreate,
+    VoiceProfileResponse,
+    ProfileSampleCreate,
+    ProfileSampleResponse,
+)
+from .database import (
+    VoiceProfile as DBVoiceProfile,
+    ProfileSample as DBProfileSample,
+)
+from .utils.audio import validate_reference_audio, load_audio, save_audio
+from .tts import get_tts_model
+
+
+# Profile storage directory
+PROFILES_DIR = Path("data/profiles")
+PROFILES_DIR.mkdir(parents=True, exist_ok=True)
+
+
+async def create_profile(
+    data: VoiceProfileCreate,
+    db: Session,
+) -> VoiceProfileResponse:
+    """
+    Create a new voice profile.
+    
+    Args:
+        data: Profile creation data
+        db: Database session
+        
+    Returns:
+        Created profile
+    """
+    # Create profile in database
+    db_profile = DBVoiceProfile(
+        id=str(uuid.uuid4()),
+        name=data.name,
+        description=data.description,
+        language=data.language,
+        created_at=datetime.utcnow(),
+        updated_at=datetime.utcnow(),
+    )
+    
+    db.add(db_profile)
+    db.commit()
+    db.refresh(db_profile)
+    
+    # Create profile directory
+    profile_dir = PROFILES_DIR / db_profile.id
+    profile_dir.mkdir(parents=True, exist_ok=True)
+    
+    return VoiceProfileResponse.model_validate(db_profile)
+
+
+async def add_profile_sample(
+    profile_id: str,
+    audio_path: str,
+    reference_text: str,
+    db: Session,
+) -> ProfileSampleResponse:
+    """
+    Add a sample to a voice profile.
+    
+    Args:
+        profile_id: Profile ID
+        audio_path: Path to temporary audio file
+        reference_text: Transcript of audio
+        db: Database session
+        
+    Returns:
+        Created sample
+    """
+    # Validate profile exists
+    profile = db.query(DBVoiceProfile).filter_by(id=profile_id).first()
+    if not profile:
+        raise ValueError(f"Profile {profile_id} not found")
+    
+    # Validate audio
+    is_valid, error_msg = validate_reference_audio(audio_path)
+    if not is_valid:
+        raise ValueError(f"Invalid reference audio: {error_msg}")
+    
+    # Create sample ID and directory
+    sample_id = str(uuid.uuid4())
+    profile_dir = PROFILES_DIR / profile_id
+    profile_dir.mkdir(parents=True, exist_ok=True)
+    
+    # Copy audio file to profile directory
+    dest_path = profile_dir / f"{sample_id}.wav"
+    audio, sr = load_audio(audio_path)
+    save_audio(audio, str(dest_path), sr)
+    
+    # Create database entry
+    db_sample = DBProfileSample(
+        id=sample_id,
+        profile_id=profile_id,
+        audio_path=str(dest_path),
+        reference_text=reference_text,
+    )
+    
+    db.add(db_sample)
+    
+    # Update profile timestamp
+    profile.updated_at = datetime.utcnow()
+    
+    db.commit()
+    db.refresh(db_sample)
+    
+    return ProfileSampleResponse.model_validate(db_sample)
+
+
+async def get_profile(
+    profile_id: str,
+    db: Session,
+) -> Optional[VoiceProfileResponse]:
+    """
+    Get a voice profile by ID.
+    
+    Args:
+        profile_id: Profile ID
+        db: Database session
+        
+    Returns:
+        Profile or None if not found
+    """
+    profile = db.query(DBVoiceProfile).filter_by(id=profile_id).first()
+    if not profile:
+        return None
+    
+    return VoiceProfileResponse.model_validate(profile)
+
+
+async def get_profile_samples(
+    profile_id: str,
+    db: Session,
+) -> List[ProfileSampleResponse]:
+    """
+    Get all samples for a profile.
+    
+    Args:
+        profile_id: Profile ID
+        db: Database session
+        
+    Returns:
+        List of samples
+    """
+    samples = db.query(DBProfileSample).filter_by(profile_id=profile_id).all()
+    return [ProfileSampleResponse.model_validate(s) for s in samples]
+
+
+async def list_profiles(db: Session) -> List[VoiceProfileResponse]:
+    """
+    List all voice profiles.
+    
+    Args:
+        db: Database session
+        
+    Returns:
+        List of profiles
+    """
+    profiles = db.query(DBVoiceProfile).order_by(
+        DBVoiceProfile.created_at.desc()
+    ).all()
+    
+    return [VoiceProfileResponse.model_validate(p) for p in profiles]
+
+
+async def update_profile(
+    profile_id: str,
+    data: VoiceProfileCreate,
+    db: Session,
+) -> Optional[VoiceProfileResponse]:
+    """
+    Update a voice profile.
+    
+    Args:
+        profile_id: Profile ID
+        data: Updated profile data
+        db: Database session
+        
+    Returns:
+        Updated profile or None if not found
+    """
+    profile = db.query(DBVoiceProfile).filter_by(id=profile_id).first()
+    if not profile:
+        return None
+    
+    # Update fields
+    profile.name = data.name
+    profile.description = data.description
+    profile.language = data.language
+    profile.updated_at = datetime.utcnow()
+    
+    db.commit()
+    db.refresh(profile)
+    
+    return VoiceProfileResponse.model_validate(profile)
+
+
+async def delete_profile(
+    profile_id: str,
+    db: Session,
+) -> bool:
+    """
+    Delete a voice profile and all associated data.
+    
+    Args:
+        profile_id: Profile ID
+        db: Database session
+        
+    Returns:
+        True if deleted, False if not found
+    """
+    profile = db.query(DBVoiceProfile).filter_by(id=profile_id).first()
+    if not profile:
+        return False
+    
+    # Delete samples from database
+    db.query(DBProfileSample).filter_by(profile_id=profile_id).delete()
+    
+    # Delete profile from database
+    db.delete(profile)
+    db.commit()
+    
+    # Delete profile directory
+    profile_dir = PROFILES_DIR / profile_id
+    if profile_dir.exists():
+        shutil.rmtree(profile_dir)
+    
+    return True
+
+
+async def delete_profile_sample(
+    sample_id: str,
+    db: Session,
+) -> bool:
+    """
+    Delete a profile sample.
+    
+    Args:
+        sample_id: Sample ID
+        db: Database session
+        
+    Returns:
+        True if deleted, False if not found
+    """
+    sample = db.query(DBProfileSample).filter_by(id=sample_id).first()
+    if not sample:
+        return False
+    
+    # Delete audio file
+    audio_path = Path(sample.audio_path)
+    if audio_path.exists():
+        audio_path.unlink()
+    
+    # Delete from database
+    db.delete(sample)
+    db.commit()
+    
+    return True
+
+
+async def create_voice_prompt_for_profile(
+    profile_id: str,
+    db: Session,
+    use_cache: bool = True,
+) -> dict:
+    """
+    Create a combined voice prompt from all samples in a profile.
+    
+    Args:
+        profile_id: Profile ID
+        db: Database session
+        use_cache: Whether to use cached prompts
+        
+    Returns:
+        Voice prompt dictionary
+    """
+    # Get all samples for profile
+    samples = db.query(DBProfileSample).filter_by(profile_id=profile_id).all()
+    
+    if not samples:
+        raise ValueError(f"No samples found for profile {profile_id}")
+    
+    tts_model = get_tts_model()
+    
+    if len(samples) == 1:
+        # Single sample - use directly
+        sample = samples[0]
+        voice_prompt, _ = await tts_model.create_voice_prompt(
+            sample.audio_path,
+            sample.reference_text,
+            use_cache=use_cache,
+        )
+        return voice_prompt
+    else:
+        # Multiple samples - combine them
+        audio_paths = [s.audio_path for s in samples]
+        reference_texts = [s.reference_text for s in samples]
+        
+        # Combine audio
+        combined_audio, combined_text = await tts_model.combine_voice_prompts(
+            audio_paths,
+            reference_texts,
+        )
+        
+        # Save combined audio temporarily
+        import tempfile
+        with tempfile.NamedTemporaryFile(suffix=".wav", delete=False) as tmp:
+            save_audio(combined_audio, tmp.name, 24000)
+            tmp_path = tmp.name
+        
+        try:
+            # Create prompt from combined audio
+            voice_prompt, _ = await tts_model.create_voice_prompt(
+                tmp_path,
+                combined_text,
+                use_cache=use_cache,
+            )
+            return voice_prompt
+        finally:
+            # Clean up temp file
+            Path(tmp_path).unlink(missing_ok=True)

backend/requirements.txt

@@ -0,0 +1,22 @@
+# FastAPI and server
+fastapi>=0.109.0
+uvicorn[standard]>=0.27.0
+pydantic>=2.5.0
+
+# Database
+sqlalchemy>=2.0.0
+alembic>=1.13.0
+
+# ML models
+torch>=2.1.0
+transformers>=4.36.0
+accelerate>=0.26.0
+huggingface_hub>=0.20.0
+
+# Audio processing
+librosa>=0.10.0
+soundfile>=0.12.0
+numpy>=1.24.0
+
+# Utilities
+python-multipart>=0.0.6

backend/studio.py

@@ -0,0 +1,66 @@
+"""
+Audio studio module for timeline editing.
+"""
+
+from typing import List, Dict, Optional
+import numpy as np
+
+
+class AudioStudio:
+    """Audio editing and timeline management."""
+    
+    async def get_word_timestamps(
+        self,
+        audio_path: str,
+        text: str,
+    ) -> List[Dict[str, float]]:
+        """
+        Get word-level timestamps for audio.
+        
+        Args:
+            audio_path: Path to audio file
+            text: Corresponding text
+            
+        Returns:
+            List of word timestamps: [{"word": "...", "start": 0.0, "end": 0.5}, ...]
+        """
+        # TODO: Implement Whisper alignment
+        raise NotImplementedError("Word timestamps not yet implemented")
+    
+    async def mix_audio(
+        self,
+        audio_paths: List[str],
+        volumes: Optional[List[float]] = None,
+    ) -> bytes:
+        """
+        Mix multiple audio files together.
+        
+        Args:
+            audio_paths: List of audio file paths
+            volumes: Optional volume levels (0.0-1.0) for each track
+            
+        Returns:
+            Mixed audio bytes (WAV format)
+        """
+        # TODO: Implement audio mixing
+        raise NotImplementedError("Audio mixing not yet implemented")
+    
+    async def trim_audio(
+        self,
+        audio_path: str,
+        start: float,
+        end: float,
+    ) -> bytes:
+        """
+        Trim audio to specified time range.
+        
+        Args:
+            audio_path: Path to audio file
+            start: Start time in seconds
+            end: End time in seconds
+            
+        Returns:
+            Trimmed audio bytes (WAV format)
+        """
+        # TODO: Implement audio trimming
+        raise NotImplementedError("Audio trimming not yet implemented")

backend/transcribe.py

@@ -0,0 +1,218 @@
+"""
+Whisper ASR module for transcription.
+"""
+
+from typing import Optional, List, Dict
+import torch
+import numpy as np
+from pathlib import Path
+
+
+class WhisperModel:
+    """Manages Whisper model loading and transcription."""
+    
+    def __init__(self, model_size: str = "base"):
+        self.model = None
+        self.processor = None
+        self.model_size = model_size
+        self.device = self._get_device()
+    
+    def _get_device(self) -> str:
+        """Get the best available device."""
+        if torch.cuda.is_available():
+            return "cuda"
+        elif hasattr(torch.backends, 'mps') and torch.backends.mps.is_available():
+            # MPS support for Whisper
+            return "cpu"  # Use CPU for stability
+        return "cpu"
+    
+    def is_loaded(self) -> bool:
+        """Check if model is loaded."""
+        return self.model is not None
+    
+    def load_model(self, model_size: Optional[str] = None):
+        """
+        Lazy load the Whisper model.
+        
+        Args:
+            model_size: Model size (tiny, base, small, medium, large)
+        """
+        if model_size is None:
+            model_size = self.model_size
+        
+        if self.model is not None and self.model_size == model_size:
+            return
+        
+        try:
+            from transformers import WhisperProcessor, WhisperForConditionalGeneration
+            
+            model_name = f"openai/whisper-{model_size}"
+            
+            print(f"Loading Whisper model {model_size} on {self.device}...")
+            
+            self.processor = WhisperProcessor.from_pretrained(model_name)
+            self.model = WhisperForConditionalGeneration.from_pretrained(model_name)
+            self.model.to(self.device)
+            
+            self.model_size = model_size
+            
+            print(f"Whisper model {model_size} loaded successfully")
+            
+        except Exception as e:
+            print(f"Error loading Whisper model: {e}")
+            raise
+    
+    def unload_model(self):
+        """Unload the model to free memory."""
+        if self.model is not None:
+            del self.model
+            del self.processor
+            self.model = None
+            self.processor = None
+            
+            if torch.cuda.is_available():
+                torch.cuda.empty_cache()
+            
+            print("Whisper model unloaded")
+    
+    async def transcribe(
+        self,
+        audio_path: str,
+        language: Optional[str] = None,
+    ) -> str:
+        """
+        Transcribe audio to text.
+        
+        Args:
+            audio_path: Path to audio file
+            language: Optional language hint (en or zh)
+            
+        Returns:
+            Transcribed text
+        """
+        self.load_model()
+        
+        from .utils.audio import load_audio
+        
+        # Load audio
+        audio, sr = load_audio(audio_path, sample_rate=16000)
+        
+        # Process audio
+        inputs = self.processor(
+            audio,
+            sampling_rate=16000,
+            return_tensors="pt",
+        )
+        inputs = inputs.to(self.device)
+        
+        # Set language if provided
+        forced_decoder_ids = None
+        if language:
+            lang_code = "en" if language == "en" else "zh"
+            forced_decoder_ids = self.processor.get_decoder_prompt_ids(
+                language=lang_code,
+                task="transcribe",
+            )
+        
+        # Generate transcription
+        with torch.no_grad():
+            predicted_ids = self.model.generate(
+                inputs["input_features"],
+                forced_decoder_ids=forced_decoder_ids,
+            )
+        
+        # Decode
+        transcription = self.processor.batch_decode(
+            predicted_ids,
+            skip_special_tokens=True,
+        )[0]
+        
+        return transcription.strip()
+    
+    async def transcribe_with_timestamps(
+        self,
+        audio_path: str,
+        language: Optional[str] = None,
+    ) -> List[Dict[str, any]]:
+        """
+        Transcribe audio with word-level timestamps.
+        
+        Args:
+            audio_path: Path to audio file
+            language: Optional language hint
+            
+        Returns:
+            List of word segments with timestamps
+        """
+        self.load_model()
+        
+        from .utils.audio import load_audio
+        
+        # Load audio
+        audio, sr = load_audio(audio_path, sample_rate=16000)
+        
+        # Process audio
+        inputs = self.processor(
+            audio,
+            sampling_rate=16000,
+            return_tensors="pt",
+        )
+        inputs = inputs.to(self.device)
+        
+        # Set language if provided
+        forced_decoder_ids = None
+        if language:
+            lang_code = "en" if language == "en" else "zh"
+            forced_decoder_ids = self.processor.get_decoder_prompt_ids(
+                language=lang_code,
+                task="transcribe",
+            )
+        
+        # Generate with timestamps
+        with torch.no_grad():
+            predicted_ids = self.model.generate(
+                inputs["input_features"],
+                forced_decoder_ids=forced_decoder_ids,
+                return_timestamps=True,
+            )
+        
+        # Decode with timestamps
+        result = self.processor.batch_decode(
+            predicted_ids,
+            skip_special_tokens=False,
+        )[0]
+        
+        # Parse timestamps (simplified - would need more robust parsing)
+        # For now, return basic transcription
+        # TODO: Implement proper timestamp parsing
+        transcription = self.processor.batch_decode(
+            predicted_ids,
+            skip_special_tokens=True,
+        )[0]
+        
+        return [
+            {
+                "text": transcription,
+                "start": 0.0,
+                "end": len(audio) / sr,
+            }
+        ]
+
+
+# Global model instance
+_whisper_model: Optional[WhisperModel] = None
+
+
+def get_whisper_model() -> WhisperModel:
+    """Get or create Whisper model instance."""
+    global _whisper_model
+    if _whisper_model is None:
+        _whisper_model = WhisperModel()
+    return _whisper_model
+
+
+def unload_whisper_model():
+    """Unload Whisper model to free memory."""
+    global _whisper_model
+    if _whisper_model is not None:
+        _whisper_model.unload_model()

backend/tts.py

@@ -0,0 +1,297 @@
+"""
+TTS inference module using Qwen3-TTS.
+"""
+
+from typing import Optional, List, Tuple
+import torch
+import numpy as np
+import io
+import soundfile as sf
+from pathlib import Path
+
+from .utils.cache import get_cache_key, get_cached_voice_prompt, cache_voice_prompt
+from .utils.audio import normalize_audio
+
+
+class TTSModel:
+    """Manages Qwen3-TTS model loading and inference."""
+    
+    def __init__(self, model_size: str = "1.7B"):
+        self.model = None
+        self.model_size = model_size
+        self.device = self._get_device()
+        self._current_model_size = None
+    
+    def _get_device(self) -> str:
+        """Get the best available device."""
+        if torch.cuda.is_available():
+            return "cuda"
+        elif hasattr(torch.backends, 'mps') and torch.backends.mps.is_available():
+            # MPS can have issues, use CPU for stability
+            return "cpu"
+        return "cpu"
+    
+    def is_loaded(self) -> bool:
+        """Check if model is loaded."""
+        return self.model is not None
+    
+    def _get_model_path(self, model_size: str) -> str:
+        """
+        Get the model path, downloading from HuggingFace Hub if needed.
+        
+        Args:
+            model_size: Model size (1.7B or 0.6B)
+            
+        Returns:
+            Path to model (either local or HuggingFace Hub ID)
+        """
+        # HuggingFace Hub model IDs
+        hf_model_map = {
+            "1.7B": "Qwen/Qwen3-TTS-12Hz-1.7B-Base",
+            "0.6B": "Qwen/Qwen3-TTS-12Hz-0.6B-Base",
+        }
+        
+        # Local directory names (for backwards compatibility)
+        local_model_map = {
+            "1.7B": "Qwen--Qwen3-TTS-12Hz-1.7B-Base",
+            "0.6B": "Qwen--Qwen3-TTS-12Hz-0.6B-Base",
+        }
+        
+        if model_size not in hf_model_map:
+            raise ValueError(f"Unknown model size: {model_size}")
+        
+        # Check if model exists locally (backwards compatibility)
+        local_path = Path("data/models") / local_model_map[model_size]
+        if local_path.exists():
+            print(f"Found local model at {local_path}")
+            return str(local_path)
+        
+        # Use HuggingFace Hub model ID (will auto-download)
+        hf_model_id = hf_model_map[model_size]
+        print(f"Will download model from HuggingFace Hub: {hf_model_id}")
+        
+        return hf_model_id
+    
+    def load_model(self, model_size: Optional[str] = None):
+        """
+        Lazy load the TTS model with automatic downloading from HuggingFace Hub.
+        
+        The model will be automatically downloaded on first use and cached locally.
+        This works similar to how Whisper models are loaded.
+        
+        Args:
+            model_size: Model size to load (1.7B or 0.6B)
+        """
+        if model_size is None:
+            model_size = self.model_size
+            
+        # If already loaded with correct size, return
+        if self.model is not None and self._current_model_size == model_size:
+            return
+        
+        # Unload existing model if different size requested
+        if self.model is not None and self._current_model_size != model_size:
+            self.unload_model()
+        
+        try:
+            from qwen_tts import Qwen3TTSModel
+            
+            # Get model path (local or HuggingFace Hub ID)
+            model_path = self._get_model_path(model_size)
+            
+            print(f"Loading TTS model {model_size} on {self.device}...")
+            
+            # Load the model - from_pretrained handles both local paths and HF Hub IDs
+            self.model = Qwen3TTSModel.from_pretrained(
+                model_path,
+                device_map=self.device,
+                torch_dtype=torch.float32 if self.device == "cpu" else torch.bfloat16,
+            )
+            
+            self._current_model_size = model_size
+            self.model_size = model_size
+            
+            print(f"TTS model {model_size} loaded successfully")
+            
+        except ImportError as e:
+            print(f"Error: qwen_tts package not found. Install with: pip install git+https://github.com/QwenLM/Qwen3-TTS.git")
+            raise
+        except Exception as e:
+            print(f"Error loading TTS model: {e}")
+            print(f"Tip: The model will be automatically downloaded from HuggingFace Hub on first use.")
+            raise
+    
+    def unload_model(self):
+        """Unload the model to free memory."""
+        if self.model is not None:
+            del self.model
+            self.model = None
+            self._current_model_size = None
+            
+            if torch.cuda.is_available():
+                torch.cuda.empty_cache()
+            
+            print("TTS model unloaded")
+    
+    async def create_voice_prompt(
+        self,
+        audio_path: str,
+        reference_text: str,
+        use_cache: bool = True,
+    ) -> Tuple[dict, bool]:
+        """
+        Create voice prompt from reference audio.
+        
+        Args:
+            audio_path: Path to reference audio file
+            reference_text: Transcript of reference audio
+            use_cache: Whether to use cached prompt if available
+            
+        Returns:
+            Tuple of (voice_prompt_dict, was_cached)
+        """
+        self.load_model()
+        
+        # Check cache if enabled
+        if use_cache:
+            cache_key = get_cache_key(audio_path, reference_text)
+            cached_prompt = get_cached_voice_prompt(cache_key)
+            if cached_prompt is not None:
+                return cached_prompt, True
+        
+        # Create new voice prompt
+        voice_prompt_items = self.model.create_voice_clone_prompt(
+            ref_audio=str(audio_path),
+            ref_text=reference_text,
+            x_vector_only_mode=False,
+        )
+        
+        # Cache if enabled
+        if use_cache:
+            cache_voice_prompt(cache_key, voice_prompt_items)
+        
+        return voice_prompt_items, False
+    
+    async def combine_voice_prompts(
+        self,
+        audio_paths: List[str],
+        reference_texts: List[str],
+    ) -> Tuple[np.ndarray, str]:
+        """
+        Combine multiple reference samples for better quality.
+        
+        Args:
+            audio_paths: List of audio file paths
+            reference_texts: List of reference texts
+            
+        Returns:
+            Tuple of (combined_audio, combined_text)
+        """
+        from .utils.audio import load_audio
+        
+        combined_audio = []
+        
+        for audio_path in audio_paths:
+            audio, sr = load_audio(audio_path)
+            audio = normalize_audio(audio)
+            combined_audio.append(audio)
+        
+        # Concatenate audio
+        mixed = np.concatenate(combined_audio)
+        mixed = normalize_audio(mixed)
+        
+        # Combine texts
+        combined_text = " ".join(reference_texts)
+        
+        return mixed, combined_text
+    
+    async def generate(
+        self,
+        text: str,
+        voice_prompt: dict,
+        language: str = "en",
+        seed: Optional[int] = None,
+    ) -> Tuple[np.ndarray, int]:
+        """
+        Generate audio from text using voice prompt.
+        
+        Args:
+            text: Text to synthesize
+            voice_prompt: Voice prompt dictionary from create_voice_prompt
+            language: Language code (en or zh)
+            seed: Random seed for reproducibility
+            
+        Returns:
+            Tuple of (audio_array, sample_rate)
+        """
+        self.load_model()
+        
+        # Set seed if provided
+        if seed is not None:
+            torch.manual_seed(seed)
+            if torch.cuda.is_available():
+                torch.cuda.manual_seed(seed)
+        
+        # Generate audio
+        wavs, sample_rate = self.model.generate_voice_clone(
+            text=text,
+            voice_clone_prompt=voice_prompt,
+        )
+        
+        audio = wavs[0]  # Get first result
+        
+        return audio, sample_rate
+    
+    async def generate_from_reference(
+        self,
+        text: str,
+        audio_path: str,
+        reference_text: str,
+        language: str = "en",
+        seed: Optional[int] = None,
+    ) -> Tuple[np.ndarray, int]:
+        """
+        Generate audio directly from reference (convenience method).
+        
+        Args:
+            text: Text to synthesize
+            audio_path: Path to reference audio
+            reference_text: Transcript of reference audio
+            language: Language code
+            seed: Random seed
+            
+        Returns:
+            Tuple of (audio_array, sample_rate)
+        """
+        # Create voice prompt (with caching)
+        voice_prompt, _ = await self.create_voice_prompt(audio_path, reference_text)
+        
+        # Generate
+        return await self.generate(text, voice_prompt, language, seed)
+
+
+# Global model instance
+_tts_model: Optional[TTSModel] = None
+
+
+def get_tts_model() -> TTSModel:
+    """Get or create TTS model instance."""
+    global _tts_model
+    if _tts_model is None:
+        _tts_model = TTSModel()
+    return _tts_model
+
+
+def unload_tts_model():
+    """Unload TTS model to free memory."""
+    global _tts_model
+    if _tts_model is not None:
+        _tts_model.unload_model()
+
+
+def audio_to_wav_bytes(audio: np.ndarray, sample_rate: int) -> bytes:
+    """Convert audio array to WAV bytes."""
+    buffer = io.BytesIO()
+    sf.write(buffer, audio, sample_rate, format="WAV")
+    buffer.seek(0)
+    return buffer.read()

backend/utils/__init__.py

@@ -0,0 +1 @@
+# Utils package

backend/utils/audio.py

@@ -0,0 +1,119 @@
+"""
+Audio processing utilities.
+"""
+
+import numpy as np
+import soundfile as sf
+import librosa
+from typing import Tuple, Optional
+
+
+def normalize_audio(
+    audio: np.ndarray,
+    target_db: float = -20.0,
+    peak_limit: float = 0.85,
+) -> np.ndarray:
+    """
+    Normalize audio to target loudness with peak limiting.
+    
+    Args:
+        audio: Input audio array
+        target_db: Target RMS level in dB
+        peak_limit: Peak limit (0.0-1.0)
+        
+    Returns:
+        Normalized audio array
+    """
+    # Convert to float32
+    audio = audio.astype(np.float32)
+    
+    # Calculate current RMS
+    rms = np.sqrt(np.mean(audio**2))
+    
+    # Calculate target RMS
+    target_rms = 10**(target_db / 20)
+    
+    # Apply gain
+    if rms > 0:
+        gain = target_rms / rms
+        audio = audio * gain
+    
+    # Peak limiting
+    audio = np.clip(audio, -peak_limit, peak_limit)
+    
+    return audio
+
+
+def load_audio(
+    path: str,
+    sample_rate: int = 24000,
+    mono: bool = True,
+) -> Tuple[np.ndarray, int]:
+    """
+    Load audio file with normalization.
+    
+    Args:
+        path: Path to audio file
+        sample_rate: Target sample rate
+        mono: Convert to mono
+        
+    Returns:
+        Tuple of (audio_array, sample_rate)
+    """
+    audio, sr = librosa.load(path, sr=sample_rate, mono=mono)
+    return audio, sr
+
+
+def save_audio(
+    audio: np.ndarray,
+    path: str,
+    sample_rate: int = 24000,
+) -> None:
+    """
+    Save audio file.
+    
+    Args:
+        audio: Audio array
+        path: Output path
+        sample_rate: Sample rate
+    """
+    sf.write(path, audio, sample_rate)
+
+
+def validate_reference_audio(
+    audio_path: str,
+    min_duration: float = 2.0,
+    max_duration: float = 30.0,
+    min_rms: float = 0.01,
+) -> Tuple[bool, Optional[str]]:
+    """
+    Validate reference audio for voice cloning.
+    
+    Args:
+        audio_path: Path to audio file
+        min_duration: Minimum duration in seconds
+        max_duration: Maximum duration in seconds
+        min_rms: Minimum RMS level
+        
+    Returns:
+        Tuple of (is_valid, error_message)
+    """
+    try:
+        audio, sr = load_audio(audio_path)
+        duration = len(audio) / sr
+        
+        if duration < min_duration:
+            return False, f"Audio too short (minimum {min_duration} seconds)"
+        if duration > max_duration:
+            return False, f"Audio too long (maximum {max_duration} seconds)"
+        
+        rms = np.sqrt(np.mean(audio**2))
+        if rms < min_rms:
+            return False, "Audio is too quiet or silent"
+        
+        if np.abs(audio).max() > 0.99:
+            return False, "Audio is clipping (reduce input gain)"
+        
+        return True, None
+    except Exception as e:
+        return False, f"Error validating audio: {str(e)}"

backend/utils/cache.py

@@ -0,0 +1,87 @@
+"""
+Voice prompt caching utilities.
+"""
+
+import hashlib
+import torch
+from pathlib import Path
+from typing import Optional, Tuple
+import soundfile as sf
+
+
+_cache_dir = Path("data/cache")
+_cache_dir.mkdir(parents=True, exist_ok=True)
+
+# In-memory cache
+_memory_cache: dict[str, torch.Tensor] = {}
+
+
+def get_cache_key(audio_path: str, reference_text: str) -> str:
+    """
+    Generate cache key from audio file and reference text.
+    
+    Args:
+        audio_path: Path to audio file
+        reference_text: Reference text
+        
+    Returns:
+        Cache key (MD5 hash)
+    """
+    # Read audio file
+    with open(audio_path, "rb") as f:
+        audio_bytes = f.read()
+    
+    # Combine audio bytes and text
+    combined = audio_bytes + reference_text.encode("utf-8")
+    
+    # Generate hash
+    return hashlib.md5(combined).hexdigest()
+
+
+def get_cached_voice_prompt(
+    cache_key: str,
+) -> Optional[torch.Tensor]:
+    """
+    Get cached voice prompt if available.
+    
+    Args:
+        cache_key: Cache key
+        
+    Returns:
+        Cached voice prompt tensor or None
+    """
+    # Check in-memory cache
+    if cache_key in _memory_cache:
+        return _memory_cache[cache_key]
+    
+    # Check disk cache
+    cache_file = _cache_dir / f"{cache_key}.prompt"
+    if cache_file.exists():
+        try:
+            prompt = torch.load(cache_file)
+            _memory_cache[cache_key] = prompt
+            return prompt
+        except Exception:
+            # Cache file corrupted, delete it
+            cache_file.unlink()
+    
+    return None
+
+
+def cache_voice_prompt(
+    cache_key: str,
+    voice_prompt: torch.Tensor,
+) -> None:
+    """
+    Cache voice prompt to memory and disk.
+    
+    Args:
+        cache_key: Cache key
+        voice_prompt: Voice prompt tensor
+    """
+    # Store in memory
+    _memory_cache[cache_key] = voice_prompt
+    
+    # Store on disk
+    cache_file = _cache_dir / f"{cache_key}.prompt"
+    torch.save(voice_prompt, cache_file)

backend/utils/validation.py

@@ -0,0 +1,63 @@
+"""
+Input validation utilities.
+"""
+
+from typing import Tuple, Optional
+from pathlib import Path
+
+
+def validate_text(text: str, max_length: int = 5000) -> Tuple[bool, Optional[str]]:
+    """
+    Validate text input.
+    
+    Args:
+        text: Text to validate
+        max_length: Maximum length
+        
+    Returns:
+        Tuple of (is_valid, error_message)
+    """
+    if not text or not text.strip():
+        return False, "Text cannot be empty"
+    
+    if len(text) > max_length:
+        return False, f"Text too long (maximum {max_length} characters)"
+    
+    return True, None
+
+
+def validate_language(language: str) -> Tuple[bool, Optional[str]]:
+    """
+    Validate language code.
+    
+    Args:
+        language: Language code
+        
+    Returns:
+        Tuple of (is_valid, error_message)
+    """
+    valid_languages = ["en", "zh"]
+    if language not in valid_languages:
+        return False, f"Invalid language (must be one of: {', '.join(valid_languages)})"
+    
+    return True, None
+
+
+def validate_file_path(path: str) -> Tuple[bool, Optional[str]]:
+    """
+    Validate file path exists.
+    
+    Args:
+        path: File path
+        
+    Returns:
+        Tuple of (is_valid, error_message)
+    """
+    file_path = Path(path)
+    if not file_path.exists():
+        return False, f"File not found: {path}"
+    
+    if not file_path.is_file():
+        return False, f"Path is not a file: {path}"
+    
+    return True, None