generated from Labyricorn/labyricorn-project-template
rebrand: rename VoiceBox to TalkBox throughout codebase
CI / frontend-quality (push) Canceled after 0s
CI / frontend-quality (push) Canceled after 0s
- All 'voicebox'/'Voicebox'/'VOICEBOX' strings replaced with 'talkbox'/'TalkBox'/'TALKBOX' - Port changed from 17493 to 17494 (avoids conflict with upstream VoiceBox) - MCP tool namespace: voicebox.* -> talkbox.* - App bundle ID: sh.voicebox.app -> com.talkbox.app - Binary names: voicebox-server -> talkbox-server, voicebox-mcp -> talkbox-mcp - Docker user/group: voicebox -> talkbox - Database: voicebox.db -> talkbox.db - Env vars: VOICEBOX_* -> TALKBOX_* - Asset files renamed: voicebox-logo.* -> talkbox-logo.*, etc. - External binaries in tauri.conf.json updated to talkbox-server/talkbox-mcp
This commit is contained in:
+11
-11
@@ -1,8 +1,8 @@
|
||||
# Voicebox Cloud Roadmap
|
||||
# TalkBox Cloud Roadmap
|
||||
|
||||
The post-mobile commercial trajectory. Captures the strategic arc beyond `mobile/PLAN.md` — what Voicebox becomes once the mobile companion ships and we start layering optional cloud services on top of the local-first base.
|
||||
The post-mobile commercial trajectory. Captures the strategic arc beyond `mobile/PLAN.md` — what TalkBox becomes once the mobile companion ships and we start layering optional cloud services on top of the local-first base.
|
||||
|
||||
The desktop app stays free. Paid surface is the cloud layer, gated behind a Voicebox account, designed so the server sees as little as possible.
|
||||
The desktop app stays free. Paid surface is the cloud layer, gated behind a TalkBox account, designed so the server sees as little as possible.
|
||||
|
||||
---
|
||||
|
||||
@@ -14,7 +14,7 @@ See [`mobile/PLAN.md`](../../mobile/PLAN.md). Entirely local: paired-device keys
|
||||
|
||||
### Phase 1 — Backup & Sync (next big feature)
|
||||
|
||||
First introduction of a Voicebox cloud account. Server stores **only encrypted blobs**.
|
||||
First introduction of a TalkBox cloud account. Server stores **only encrypted blobs**.
|
||||
|
||||
- **E2E encryption keyed off the device key** from the mobile pairing flow. Audio + transcript blobs are encrypted client-side before upload; the server never has the plaintext or the key.
|
||||
- **Quota by number of generations**, not by storage GB. Avoids "how many GB do you offer" framing and keeps tiering legible. (Word-count quotas are an alternative — closer to the ElevenLabs model — but generations are simpler to communicate.)
|
||||
@@ -28,7 +28,7 @@ The privacy framing is load-bearing. "We see encrypted blobs and that's it" is t
|
||||
|
||||
The big bet. Today there is no major neutral voice-inference provider — every cloud TTS service ships its own proprietary models. Open-source TTS models exist and keep getting better, but nobody runs them as a paid hosted catalog at scale.
|
||||
|
||||
Voicebox already has the distribution. The thesis is: the same users who chose local-first specifically to avoid sending voice data to ElevenLabs will pay a fair markup to run open-source voices on hosted GPUs **when they don't have local hardware** (mobile-only users, low-end laptops, "I just don't want to manage CUDA"), provided the privacy story stays consistent.
|
||||
TalkBox already has the distribution. The thesis is: the same users who chose local-first specifically to avoid sending voice data to ElevenLabs will pay a fair markup to run open-source voices on hosted GPUs **when they don't have local hardware** (mobile-only users, low-end laptops, "I just don't want to manage CUDA"), provided the privacy story stays consistent.
|
||||
|
||||
- **Catalog-first positioning.** Cloud can offer more voices than the desktop binary bundles (the bundle is already 500MB without CUDA, ~3GB with — there's a hard ceiling on what we can ship locally). Catalog grows over time.
|
||||
- **Pricing tiers (rough first cut):** $5 / $15 / $25 / month, plus Enterprise. Final numbers depend on benchmarking — see below.
|
||||
@@ -71,16 +71,16 @@ Scale-to-zero where cold start fits the budget. Hot engines need warm pools size
|
||||
|
||||
**Profile pipeline.** Cloned voice → encrypted blob with user-account-key → uploaded to R2 cold storage → fetched into worker memory at job start → decrypted in memory only, never written to worker disk → discarded on worker idle. TTL applies at the R2 layer (cold storage retention); worker hot-path retention is bounded by warmup window. Embedding-only caching where the engine exposes a stable embedding interface; raw-audio caching is the fallback. Per-engine audit needed before launch — see open questions.
|
||||
|
||||
**Hybrid routing on the client.** Desktop, mobile, and MCP clients already speak `127.0.0.1:17493`. Add `VOICEBOX_API_URL` + `VOICEBOX_API_KEY` plus a routing function:
|
||||
**Hybrid routing on the client.** Desktop, mobile, and MCP clients already speak `127.0.0.1:17494`. Add `TALKBOX_API_URL` + `TALKBOX_API_KEY` plus a routing function:
|
||||
|
||||
```
|
||||
if local_backend_reachable() and engine in local_engines:
|
||||
→ 127.0.0.1:17493
|
||||
→ 127.0.0.1:17494
|
||||
else:
|
||||
→ api.voicebox.sh/v1
|
||||
→ api.talkbox.sh/v1
|
||||
```
|
||||
|
||||
Mobile-without-paired-desktop falls through to cloud automatically. Desktop without a usable GPU falls through for big engines, stays local for Kokoro. Same `voicebox.speak()` MCP call works either way. This is the differentiator versus ElevenLabs (cloud-only) and pure local-first competitors (no fallback).
|
||||
Mobile-without-paired-desktop falls through to cloud automatically. Desktop without a usable GPU falls through for big engines, stays local for Kokoro. Same `talkbox.speak()` MCP call works either way. This is the differentiator versus ElevenLabs (cloud-only) and pure local-first competitors (no fallback).
|
||||
|
||||
#### Cloud-cached voice profiles
|
||||
|
||||
@@ -99,7 +99,7 @@ Inference latency makes it untenable to re-upload reference samples per call. Cl
|
||||
|
||||
A marketplace where voice owners license their cloned voices for others to use, with revenue sharing. Possibly: "rent out your AI voice."
|
||||
|
||||
This is the only phase that requires hosting voice profiles, and it requires real licensing infrastructure first — consent verification, takedown flow, identity claims, revenue accounting. Until that exists, **Voicebox does not host voice profiles in cloud at all** (see constraint below). Marketplace is the long-term endgame, not the next quarter.
|
||||
This is the only phase that requires hosting voice profiles, and it requires real licensing infrastructure first — consent verification, takedown flow, identity claims, revenue accounting. Until that exists, **TalkBox does not host voice profiles in cloud at all** (see constraint below). Marketplace is the long-term endgame, not the next quarter.
|
||||
|
||||
---
|
||||
|
||||
@@ -119,7 +119,7 @@ This protects two things at once:
|
||||
|
||||
### Privacy is the moat, not a feature
|
||||
|
||||
The "private LLM users → ElevenLabs voice" workflow is incoherent: people pay to keep their text private and then hand their speech to a cloud vendor that trains on it. Voicebox is the consistent answer for that audience. Every cloud feature should be designed so a privacy-conscious user can adopt it without breaking that internal consistency — which is why Phase 1 is fully E2E and Phase 2 is "audited no-log" rather than "we have your audio but trust us."
|
||||
The "private LLM users → ElevenLabs voice" workflow is incoherent: people pay to keep their text private and then hand their speech to a cloud vendor that trains on it. TalkBox is the consistent answer for that audience. Every cloud feature should be designed so a privacy-conscious user can adopt it without breaking that internal consistency — which is why Phase 1 is fully E2E and Phase 2 is "audited no-log" rather than "we have your audio but trust us."
|
||||
|
||||
### Revenue stack is multi-source
|
||||
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
|
||||
## Overview
|
||||
|
||||
Docker support makes Voicebox easier to deploy, especially for:
|
||||
Docker support makes TalkBox easier to deploy, especially for:
|
||||
|
||||
- **Consistent Environments**: Same setup across dev/staging/prod
|
||||
- **GPU Passthrough**: Easy NVIDIA/AMD GPU access
|
||||
@@ -19,17 +19,17 @@ Docker support makes Voicebox easier to deploy, especially for:
|
||||
|
||||
```bash
|
||||
# CPU-only version
|
||||
docker run -p 8000:8000 -v voicebox-data:/app/data \
|
||||
ghcr.io/jamiepine/voicebox:latest
|
||||
docker run -p 8000:8000 -v talkbox-data:/app/data \
|
||||
ghcr.io/jamiepine/talkbox:latest
|
||||
|
||||
# NVIDIA GPU version
|
||||
docker run --gpus all -p 8000:8000 -v voicebox-data:/app/data \
|
||||
ghcr.io/jamiepine/voicebox:latest-cuda
|
||||
docker run --gpus all -p 8000:8000 -v talkbox-data:/app/data \
|
||||
ghcr.io/jamiepine/talkbox:latest-cuda
|
||||
|
||||
# AMD GPU version (experimental)
|
||||
docker run --device=/dev/kfd --device=/dev/dri -p 8000:8000 \
|
||||
-v voicebox-data:/app/data \
|
||||
ghcr.io/jamiepine/voicebox:latest-rocm
|
||||
-v talkbox-data:/app/data \
|
||||
ghcr.io/jamiepine/talkbox:latest-rocm
|
||||
```
|
||||
|
||||
Then open: `http://localhost:8000`
|
||||
@@ -42,12 +42,12 @@ Create `docker-compose.yml`:
|
||||
version: '3.8'
|
||||
|
||||
services:
|
||||
voicebox:
|
||||
image: ghcr.io/jamiepine/voicebox:latest-cuda
|
||||
talkbox:
|
||||
image: ghcr.io/jamiepine/talkbox:latest-cuda
|
||||
ports:
|
||||
- "8000:8000"
|
||||
volumes:
|
||||
- voicebox-data:/app/data
|
||||
- talkbox-data:/app/data
|
||||
- huggingface-cache:/root/.cache/huggingface
|
||||
environment:
|
||||
- GPU_MEMORY_FRACTION=0.8 # Use 80% of GPU memory
|
||||
@@ -62,7 +62,7 @@ services:
|
||||
capabilities: [gpu]
|
||||
|
||||
volumes:
|
||||
voicebox-data:
|
||||
talkbox-data:
|
||||
huggingface-cache:
|
||||
```
|
||||
|
||||
@@ -108,8 +108,8 @@ CMD ["uvicorn", "backend.main:app", "--host", "0.0.0.0", "--port", "8000"]
|
||||
|
||||
Build and run:
|
||||
```bash
|
||||
docker build -t voicebox .
|
||||
docker run -p 8000:8000 -v $(pwd)/data:/app/data voicebox
|
||||
docker build -t talkbox .
|
||||
docker run -p 8000:8000 -v $(pwd)/data:/app/data talkbox
|
||||
```
|
||||
|
||||
### Multi-Stage Build (Optimized)
|
||||
@@ -160,7 +160,7 @@ CMD ["uvicorn", "backend.main:app", "--host", "0.0.0.0", "--port", "8000"]
|
||||
|
||||
Build:
|
||||
```bash
|
||||
docker build -f Dockerfile.optimized -t voicebox:slim .
|
||||
docker build -f Dockerfile.optimized -t talkbox:slim .
|
||||
```
|
||||
|
||||
## GPU Support
|
||||
@@ -195,15 +195,15 @@ CMD ["uvicorn", "backend.main:app", "--host", "0.0.0.0", "--port", "8000"]
|
||||
**Run with GPU:**
|
||||
```bash
|
||||
docker run --gpus all -p 8000:8000 \
|
||||
-v voicebox-data:/app/data \
|
||||
voicebox:cuda
|
||||
-v talkbox-data:/app/data \
|
||||
talkbox:cuda
|
||||
```
|
||||
|
||||
**Docker Compose with GPU:**
|
||||
```yaml
|
||||
services:
|
||||
voicebox:
|
||||
image: voicebox:cuda
|
||||
talkbox:
|
||||
image: talkbox:cuda
|
||||
deploy:
|
||||
resources:
|
||||
reservations:
|
||||
@@ -249,8 +249,8 @@ CMD ["uvicorn", "backend.main:app", "--host", "0.0.0.0", "--port", "8000"]
|
||||
docker run --device=/dev/kfd --device=/dev/dri \
|
||||
--group-add video --ipc=host --cap-add=SYS_PTRACE \
|
||||
--security-opt seccomp=unconfined \
|
||||
-p 8000:8000 -v voicebox-data:/app/data \
|
||||
voicebox:rocm
|
||||
-p 8000:8000 -v talkbox-data:/app/data \
|
||||
talkbox:rocm
|
||||
```
|
||||
|
||||
**Note:** ROCm support varies by GPU model. Works best on Linux. See [AMD ROCm docs](https://rocm.docs.amd.com) for compatibility.
|
||||
@@ -260,9 +260,9 @@ docker run --device=/dev/kfd --device=/dev/dri \
|
||||
### Essential Volumes
|
||||
|
||||
```bash
|
||||
docker run -v voicebox-data:/app/data \ # Profiles, generations, history
|
||||
docker run -v talkbox-data:/app/data \ # Profiles, generations, history
|
||||
-v huggingface-cache:/root/.cache/huggingface \ # Downloaded models
|
||||
-p 8000:8000 voicebox
|
||||
-p 8000:8000 talkbox
|
||||
```
|
||||
|
||||
### Development Volume Mounts
|
||||
@@ -271,9 +271,9 @@ For development with hot-reload:
|
||||
|
||||
```bash
|
||||
docker run -v $(pwd)/backend:/app/backend \ # Live code changes
|
||||
-v voicebox-data:/app/data \
|
||||
-v talkbox-data:/app/data \
|
||||
-e RELOAD=true \
|
||||
-p 8000:8000 voicebox
|
||||
-p 8000:8000 talkbox
|
||||
```
|
||||
|
||||
### Custom Model Storage
|
||||
@@ -283,13 +283,13 @@ Use external model directory:
|
||||
```bash
|
||||
docker run -v /path/to/models:/models \
|
||||
-e MODELS_DIR=/models \
|
||||
-v voicebox-data:/app/data \
|
||||
-p 8000:8000 voicebox
|
||||
-v talkbox-data:/app/data \
|
||||
-p 8000:8000 talkbox
|
||||
```
|
||||
|
||||
## Environment Variables
|
||||
|
||||
Configure Voicebox via environment variables:
|
||||
Configure TalkBox via environment variables:
|
||||
|
||||
```bash
|
||||
docker run -e TTS_MODE=local \
|
||||
@@ -297,7 +297,7 @@ docker run -e TTS_MODE=local \
|
||||
-e OPENAI_API_KEY=sk-... \
|
||||
-e GPU_MEMORY_FRACTION=0.8 \
|
||||
-e LOG_LEVEL=info \
|
||||
-p 8000:8000 voicebox
|
||||
-p 8000:8000 talkbox
|
||||
```
|
||||
|
||||
### Available Variables
|
||||
@@ -324,14 +324,14 @@ docker run -e TTS_MODE=local \
|
||||
version: '3.8'
|
||||
|
||||
services:
|
||||
voicebox:
|
||||
image: ghcr.io/jamiepine/voicebox:latest-cuda
|
||||
container_name: voicebox
|
||||
talkbox:
|
||||
image: ghcr.io/jamiepine/talkbox:latest-cuda
|
||||
container_name: talkbox
|
||||
restart: unless-stopped
|
||||
ports:
|
||||
- "8000:8000"
|
||||
volumes:
|
||||
- voicebox-data:/app/data
|
||||
- talkbox-data:/app/data
|
||||
- huggingface-cache:/root/.cache/huggingface
|
||||
environment:
|
||||
- TTS_MODE=local
|
||||
@@ -353,7 +353,7 @@ services:
|
||||
start_period: 40s
|
||||
|
||||
volumes:
|
||||
voicebox-data:
|
||||
talkbox-data:
|
||||
driver: local
|
||||
huggingface-cache:
|
||||
driver: local
|
||||
@@ -371,7 +371,7 @@ docker compose -f docker-compose.prod.yml up -d
|
||||
version: '3.8'
|
||||
|
||||
services:
|
||||
voicebox:
|
||||
talkbox:
|
||||
build:
|
||||
context: .
|
||||
dockerfile: Dockerfile
|
||||
@@ -379,7 +379,7 @@ services:
|
||||
- "8000:8000"
|
||||
volumes:
|
||||
- ./backend:/app/backend:ro
|
||||
- voicebox-data:/app/data
|
||||
- talkbox-data:/app/data
|
||||
- huggingface-cache:/root/.cache/huggingface
|
||||
environment:
|
||||
- RELOAD=true
|
||||
@@ -388,7 +388,7 @@ services:
|
||||
command: uvicorn backend.main:app --host 0.0.0.0 --port 8000 --reload
|
||||
|
||||
volumes:
|
||||
voicebox-data:
|
||||
talkbox-data:
|
||||
huggingface-cache:
|
||||
```
|
||||
|
||||
@@ -401,12 +401,12 @@ Full stack with reverse proxy and monitoring:
|
||||
version: '3.8'
|
||||
|
||||
services:
|
||||
# Main Voicebox app
|
||||
voicebox:
|
||||
image: ghcr.io/jamiepine/voicebox:latest-cuda
|
||||
# Main TalkBox app
|
||||
talkbox:
|
||||
image: ghcr.io/jamiepine/talkbox:latest-cuda
|
||||
restart: unless-stopped
|
||||
volumes:
|
||||
- voicebox-data:/app/data
|
||||
- talkbox-data:/app/data
|
||||
- huggingface-cache:/root/.cache/huggingface
|
||||
environment:
|
||||
- TTS_MODE=local
|
||||
@@ -429,7 +429,7 @@ services:
|
||||
- ./nginx.conf:/etc/nginx/nginx.conf:ro
|
||||
- ./ssl:/etc/nginx/ssl:ro
|
||||
depends_on:
|
||||
- voicebox
|
||||
- talkbox
|
||||
|
||||
# Prometheus monitoring (optional)
|
||||
prometheus:
|
||||
@@ -441,7 +441,7 @@ services:
|
||||
- prometheus-data:/prometheus
|
||||
|
||||
volumes:
|
||||
voicebox-data:
|
||||
talkbox-data:
|
||||
huggingface-cache:
|
||||
prometheus-data:
|
||||
```
|
||||
@@ -466,9 +466,9 @@ volumes:
|
||||
3. **Deploy:**
|
||||
```bash
|
||||
docker run --gpus all -d -p 80:8000 \
|
||||
-v voicebox-data:/app/data \
|
||||
-v talkbox-data:/app/data \
|
||||
--restart unless-stopped \
|
||||
ghcr.io/jamiepine/voicebox:latest-cuda
|
||||
ghcr.io/jamiepine/talkbox:latest-cuda
|
||||
```
|
||||
|
||||
### DigitalOcean
|
||||
@@ -477,7 +477,7 @@ Use GPU Droplet + Docker:
|
||||
|
||||
```bash
|
||||
# Create droplet via CLI
|
||||
doctl compute droplet create voicebox \
|
||||
doctl compute droplet create talkbox \
|
||||
--size gpu-h100x1-80gb \
|
||||
--image ubuntu-22-04-x64 \
|
||||
--region nyc3
|
||||
@@ -486,19 +486,19 @@ doctl compute droplet create voicebox \
|
||||
ssh root@<droplet-ip>
|
||||
curl -fsSL https://get.docker.com -o get-docker.sh
|
||||
sh get-docker.sh
|
||||
docker run --gpus all -d -p 80:8000 voicebox:cuda
|
||||
docker run --gpus all -d -p 80:8000 talkbox:cuda
|
||||
```
|
||||
|
||||
### Google Cloud Run (CPU-only)
|
||||
|
||||
```bash
|
||||
# Build and push
|
||||
docker build -t gcr.io/your-project/voicebox .
|
||||
docker push gcr.io/your-project/voicebox
|
||||
docker build -t gcr.io/your-project/talkbox .
|
||||
docker push gcr.io/your-project/talkbox
|
||||
|
||||
# Deploy to Cloud Run
|
||||
gcloud run deploy voicebox \
|
||||
--image gcr.io/your-project/voicebox \
|
||||
gcloud run deploy talkbox \
|
||||
--image gcr.io/your-project/talkbox \
|
||||
--platform managed \
|
||||
--region us-central1 \
|
||||
--memory 4Gi \
|
||||
@@ -510,10 +510,10 @@ gcloud run deploy voicebox \
|
||||
|
||||
Create `fly.toml`:
|
||||
```toml
|
||||
app = "voicebox"
|
||||
app = "talkbox"
|
||||
|
||||
[build]
|
||||
image = "ghcr.io/jamiepine/voicebox:latest"
|
||||
image = "ghcr.io/jamiepine/talkbox:latest"
|
||||
|
||||
[[services]]
|
||||
http_checks = []
|
||||
@@ -529,7 +529,7 @@ app = "voicebox"
|
||||
handlers = ["tls", "http"]
|
||||
|
||||
[mounts]
|
||||
source = "voicebox_data"
|
||||
source = "talkbox_data"
|
||||
destination = "/app/data"
|
||||
```
|
||||
|
||||
@@ -560,38 +560,38 @@ docker run --rm --device=/dev/kfd --device=/dev/dri rocm/dev-ubuntu-22.04:6.0 ro
|
||||
Container can't write to volumes:
|
||||
```bash
|
||||
# Fix permissions
|
||||
docker run --user $(id -u):$(id -g) -v $(pwd)/data:/app/data voicebox
|
||||
docker run --user $(id -u):$(id -g) -v $(pwd)/data:/app/data talkbox
|
||||
```
|
||||
|
||||
### Out of Memory
|
||||
|
||||
Reduce GPU memory usage:
|
||||
```bash
|
||||
docker run -e GPU_MEMORY_FRACTION=0.5 voicebox
|
||||
docker run -e GPU_MEMORY_FRACTION=0.5 talkbox
|
||||
```
|
||||
|
||||
Or use CPU-only:
|
||||
```bash
|
||||
docker run -e DEVICE=cpu voicebox
|
||||
docker run -e DEVICE=cpu talkbox
|
||||
```
|
||||
|
||||
### Model Download Fails
|
||||
|
||||
Ensure HuggingFace cache is writable:
|
||||
```bash
|
||||
docker run -v huggingface-cache:/root/.cache/huggingface voicebox
|
||||
docker run -v huggingface-cache:/root/.cache/huggingface talkbox
|
||||
```
|
||||
|
||||
Or use host cache:
|
||||
```bash
|
||||
docker run -v ~/.cache/huggingface:/root/.cache/huggingface voicebox
|
||||
docker run -v ~/.cache/huggingface:/root/.cache/huggingface talkbox
|
||||
```
|
||||
|
||||
### Port Already in Use
|
||||
|
||||
Change host port:
|
||||
```bash
|
||||
docker run -p 8080:8000 voicebox # Use port 8080 instead
|
||||
docker run -p 8080:8000 talkbox # Use port 8080 instead
|
||||
```
|
||||
|
||||
## Security Best Practices
|
||||
@@ -600,8 +600,8 @@ docker run -p 8080:8000 voicebox # Use port 8080 instead
|
||||
|
||||
Create non-root user in Dockerfile:
|
||||
```dockerfile
|
||||
RUN useradd -m -u 1000 voicebox
|
||||
USER voicebox
|
||||
RUN useradd -m -u 1000 talkbox
|
||||
USER talkbox
|
||||
```
|
||||
|
||||
### 2. Use Secrets for API Keys
|
||||
@@ -615,7 +615,7 @@ echo "sk-your-key" | docker secret create openai_key -
|
||||
docker service create \
|
||||
--secret openai_key \
|
||||
-e OPENAI_API_KEY_FILE=/run/secrets/openai_key \
|
||||
voicebox
|
||||
talkbox
|
||||
```
|
||||
|
||||
### 3. Network Isolation
|
||||
@@ -624,7 +624,7 @@ Use internal networks for multi-container setups:
|
||||
|
||||
```yaml
|
||||
services:
|
||||
voicebox:
|
||||
talkbox:
|
||||
networks:
|
||||
- internal
|
||||
nginx:
|
||||
@@ -646,7 +646,7 @@ Prevent resource exhaustion:
|
||||
|
||||
```yaml
|
||||
services:
|
||||
voicebox:
|
||||
talkbox:
|
||||
deploy:
|
||||
resources:
|
||||
limits:
|
||||
@@ -663,10 +663,10 @@ services:
|
||||
|
||||
```bash
|
||||
# Use 80% of GPU (default 90%)
|
||||
docker run -e GPU_MEMORY_FRACTION=0.8 voicebox
|
||||
docker run -e GPU_MEMORY_FRACTION=0.8 talkbox
|
||||
|
||||
# Allow GPU memory growth (prevents OOM)
|
||||
docker run -e TF_FORCE_GPU_ALLOW_GROWTH=true voicebox
|
||||
docker run -e TF_FORCE_GPU_ALLOW_GROWTH=true talkbox
|
||||
```
|
||||
|
||||
### Model Caching
|
||||
@@ -676,14 +676,14 @@ Pre-download models to volume:
|
||||
```bash
|
||||
# Download models first
|
||||
docker run --rm -v huggingface-cache:/root/.cache/huggingface \
|
||||
voicebox python -c "
|
||||
talkbox python -c "
|
||||
from transformers import WhisperProcessor, WhisperForConditionalGeneration
|
||||
WhisperProcessor.from_pretrained('openai/whisper-base')
|
||||
WhisperForConditionalGeneration.from_pretrained('openai/whisper-base')
|
||||
"
|
||||
|
||||
# Then run normally
|
||||
docker run -v huggingface-cache:/root/.cache/huggingface voicebox
|
||||
docker run -v huggingface-cache:/root/.cache/huggingface talkbox
|
||||
```
|
||||
|
||||
### Multi-Worker Setup
|
||||
@@ -728,10 +728,10 @@ Then scrape `/metrics` with Prometheus.
|
||||
|
||||
View container logs:
|
||||
```bash
|
||||
docker logs -f voicebox
|
||||
docker logs -f talkbox
|
||||
|
||||
# Or with compose
|
||||
docker compose logs -f voicebox
|
||||
docker compose logs -f talkbox
|
||||
```
|
||||
|
||||
## Next Steps
|
||||
@@ -748,7 +748,7 @@ Help improve Docker support:
|
||||
1. Test on different platforms (AMD GPU, ARM64, etc.)
|
||||
2. Submit Dockerfile optimizations
|
||||
3. Share deployment configurations
|
||||
4. Report issues: [GitHub Issues](https://github.com/jamiepine/voicebox/issues)
|
||||
4. Report issues: [GitHub Issues](https://github.com/jamiepine/talkbox/issues)
|
||||
|
||||
## Resources
|
||||
|
||||
|
||||
@@ -6,13 +6,13 @@
|
||||
|
||||
## Context
|
||||
|
||||
Homebrew Cask PR [#260314](https://github.com/Homebrew/homebrew-cask/pull/260314) adds `brew install --cask voicebox`. CI is green on macOS 14 and macOS 26 (arm + intel) but fails on macOS 15 (arm + intel). The 0.4.3 release added DMG-level stapling to address this, and it didn't move CI — 0.4.5 still fails. A maintainer reproduced the failure in a fresh Sequoia VM.
|
||||
Homebrew Cask PR [#260314](https://github.com/Homebrew/homebrew-cask/pull/260314) adds `brew install --cask talkbox`. CI is green on macOS 14 and macOS 26 (arm + intel) but fails on macOS 15 (arm + intel). The 0.4.3 release added DMG-level stapling to address this, and it didn't move CI — 0.4.5 still fails. A maintainer reproduced the failure in a fresh Sequoia VM.
|
||||
|
||||
This document is the working diagnosis plus the ordered fix plan.
|
||||
|
||||
## What the failing check actually does
|
||||
|
||||
The failing step is `brew audit --cask --online --signing --new voicebox`, not `brew install`. `brew install` succeeds end-to-end in CI (the log shows `Uninstalling Cask voicebox` after the install phase). The `--signing` audit:
|
||||
The failing step is `brew audit --cask --online --signing --new talkbox`, not `brew install`. `brew install` succeeds end-to-end in CI (the log shows `Uninstalling Cask talkbox` after the install phase). The `--signing` audit:
|
||||
|
||||
1. Downloads the cask's `url`
|
||||
2. Mounts the DMG
|
||||
@@ -32,16 +32,16 @@ Local dev machines pass `spctl` because the first-party developer context and ca
|
||||
|
||||
## Where the gap is likely to be
|
||||
|
||||
Voicebox ships PyInstaller sidecars declared in `tauri.conf.json` under `externalBin`:
|
||||
TalkBox ships PyInstaller sidecars declared in `tauri.conf.json` under `externalBin`:
|
||||
|
||||
- **0.4.x:** `voicebox-server` only (single `--onefile` Mach-O on macOS)
|
||||
- **0.5.0+:** `voicebox-server` and `voicebox-mcp` (`voicebox-mcp` is new in 0.5.0)
|
||||
- **0.4.x:** `talkbox-server` only (single `--onefile` Mach-O on macOS)
|
||||
- **0.5.0+:** `talkbox-server` and `talkbox-mcp` (`talkbox-mcp` is new in 0.5.0)
|
||||
|
||||
Tauri's bundler signs each `externalBin` with the configured identity but does not apply `--options=runtime` or `--timestamp` automatically, and does not merge the outer app's entitlements into the sidecar signature. The outer `Voicebox` binary is correctly signed with hardened runtime + `disable-library-validation`; the sidecars likely are not.
|
||||
Tauri's bundler signs each `externalBin` with the configured identity but does not apply `--options=runtime` or `--timestamp` automatically, and does not merge the outer app's entitlements into the sidecar signature. The outer `TalkBox` binary is correctly signed with hardened runtime + `disable-library-validation`; the sidecars likely are not.
|
||||
|
||||
Order of likelihood:
|
||||
|
||||
1. Sidecar `voicebox-server` lacks hardened runtime or a secure timestamp in its signature.
|
||||
1. Sidecar `talkbox-server` lacks hardened runtime or a secure timestamp in its signature.
|
||||
2. The sidecar inherits the identity but was signed before tauri-action's final notarization pass, so the notarization ticket doesn't actually cover it.
|
||||
3. Something inside the sidecar's PyInstaller archive unpacks to a `.dylib` at runtime that Gatekeeper inspects during assessment.
|
||||
|
||||
@@ -52,11 +52,11 @@ The 0.5.0 fix must cover both sidecars.
|
||||
Run against a freshly downloaded release DMG (not a dev build, and from a machine that has never opened the app before):
|
||||
|
||||
```
|
||||
hdiutil attach Voicebox_0.4.5_aarch64.dmg
|
||||
xcrun stapler validate "/Volumes/Voicebox 0.4.5/Voicebox.app"
|
||||
spctl -a -vvv -t open --context context:primary-signature "/Volumes/Voicebox 0.4.5/Voicebox.app"
|
||||
codesign --verify --deep --strict --verbose=2 "/Volumes/Voicebox 0.4.5/Voicebox.app"
|
||||
codesign -dv --verbose=4 "/Volumes/Voicebox 0.4.5/Voicebox.app/Contents/MacOS/voicebox-server"
|
||||
hdiutil attach TalkBox_0.4.5_aarch64.dmg
|
||||
xcrun stapler validate "/Volumes/TalkBox 0.4.5/TalkBox.app"
|
||||
spctl -a -vvv -t open --context context:primary-signature "/Volumes/TalkBox 0.4.5/TalkBox.app"
|
||||
codesign --verify --deep --strict --verbose=2 "/Volumes/TalkBox 0.4.5/TalkBox.app"
|
||||
codesign -dv --verbose=4 "/Volumes/TalkBox 0.4.5/TalkBox.app/Contents/MacOS/talkbox-server"
|
||||
```
|
||||
|
||||
The last command is the tell — look for `flags=0x10000(runtime)` and a `Timestamp=` line. If either is missing, the sidecar is the failure.
|
||||
@@ -71,7 +71,7 @@ Pull the 0.4.5 DMG on a fresh Sequoia environment or a VM snapshot with no trust
|
||||
|
||||
### Phase 2 — Sign sidecars explicitly in the release workflow
|
||||
|
||||
Between tauri-action's build step and the DMG-notarization step already in `release.yml`, add a step that re-signs every `externalBin` present under `Voicebox.app/Contents/MacOS/` with:
|
||||
Between tauri-action's build step and the DMG-notarization step already in `release.yml`, add a step that re-signs every `externalBin` present under `TalkBox.app/Contents/MacOS/` with:
|
||||
|
||||
- `--options=runtime` (hardened runtime)
|
||||
- `--timestamp` (secure timestamp)
|
||||
@@ -80,11 +80,11 @@ Between tauri-action's build step and the DMG-notarization step already in `rele
|
||||
|
||||
Re-sign the outer `.app` afterward so its seal covers the updated nested signatures.
|
||||
|
||||
Covers `voicebox-server` on 0.4.x and both sidecars from 0.5.0 forward.
|
||||
Covers `talkbox-server` on 0.4.x and both sidecars from 0.5.0 forward.
|
||||
|
||||
### Phase 3 — Re-notarize and staple the `.app`
|
||||
|
||||
After sidecars are re-signed the outer bundle's notarization ticket is stale. Submit the `.app` (zipped) to `notarytool`, wait, then `xcrun stapler staple Voicebox.app`. This puts the ticket directly on the `.app` so the `spctl -t open` audit passes without any online ticket lookup.
|
||||
After sidecars are re-signed the outer bundle's notarization ticket is stale. Submit the `.app` (zipped) to `notarytool`, wait, then `xcrun stapler staple TalkBox.app`. This puts the ticket directly on the `.app` so the `spctl -t open` audit passes without any online ticket lookup.
|
||||
|
||||
Then rebuild the DMG from the stapled `.app` and keep the existing DMG-level notarize/staple step — it still helps Finder drag-install.
|
||||
|
||||
@@ -94,7 +94,7 @@ Before upload, run the same four diagnostic commands against the built artifact
|
||||
|
||||
### Phase 5 — Re-request Homebrew CI
|
||||
|
||||
Once a tagged release passes Phase 4 locally, push a cask update to #260314. Expect `test voicebox (macos-15, arm)` and `test voicebox (macos-15-intel, intel)` to go green.
|
||||
Once a tagged release passes Phase 4 locally, push a cask update to #260314. Expect `test talkbox (macos-15, arm)` and `test talkbox (macos-15-intel, intel)` to go green.
|
||||
|
||||
## Open questions
|
||||
|
||||
|
||||
+57
-57
@@ -1,4 +1,4 @@
|
||||
# MCP Server — Voicebox Speed Run
|
||||
# MCP Server — TalkBox Speed Run
|
||||
|
||||
**Status:** v1 shipped — HTTP transport, all 4 tools, per-client bindings, `POST /speak`, stdio shim (binary built, bundled into Tauri sidecar), Settings UI, speak-pill via SSE with Rust-side `dictate:show` handler so agent-initiated speech surfaces the pill on screen. `cargo check` clean, `tsc` clean, full Inspector round-trip verified.
|
||||
**Last reviewed:** 2026-04-23
|
||||
@@ -8,21 +8,21 @@
|
||||
### Shipped (backend)
|
||||
- **`fastmcp` + `sse-starlette`** pinned in `backend/requirements.txt`.
|
||||
- **`backend/mcp_server/`** package with `server.py`, `tools.py`, `context.py`, `resolve.py`, `events.py`, `README.md`. Named `mcp_server` (not `mcp`) to sidestep a shadowing conflict with the installed `mcp` PyPI package that FastMCP imports internally.
|
||||
- **Streamable HTTP mount at `/mcp`** via FastMCP's `http_app(transport='http')`. Sub-app lifespan composed with Voicebox's own startup/shutdown through an `@asynccontextmanager lifespan=` in `backend/app.py` (migrated away from the deprecated `@app.on_event` handlers).
|
||||
- **Streamable HTTP mount at `/mcp`** via FastMCP's `http_app(transport='http')`. Sub-app lifespan composed with TalkBox's own startup/shutdown through an `@asynccontextmanager lifespan=` in `backend/app.py` (migrated away from the deprecated `@app.on_event` handlers).
|
||||
- **Four MCP tools**, dot-named to match the landing and ecosystem convention:
|
||||
- `voicebox.speak(text, profile?, engine?, personality?, language?)`
|
||||
- `voicebox.transcribe(audio_base64?, audio_path?, language?, model?)`
|
||||
- `voicebox.list_captures(limit, offset)`
|
||||
- `voicebox.list_profiles()`
|
||||
- **`ClientIdMiddleware`** pulls `X-Voicebox-Client-Id` into a `ContextVar` on every `/mcp*` request; auto-stamps `MCPClientBinding.last_seen_at`, auto-creating the row if the client is new.
|
||||
- `talkbox.speak(text, profile?, engine?, personality?, language?)`
|
||||
- `talkbox.transcribe(audio_base64?, audio_path?, language?, model?)`
|
||||
- `talkbox.list_captures(limit, offset)`
|
||||
- `talkbox.list_profiles()`
|
||||
- **`ClientIdMiddleware`** pulls `X-TalkBox-Client-Id` into a `ContextVar` on every `/mcp*` request; auto-stamps `MCPClientBinding.last_seen_at`, auto-creating the row if the client is new.
|
||||
- **Profile resolution precedence** `explicit → per-client binding → capture_settings.default_playback_voice_id → error`. `services/profiles.get_profile_orm_by_name_or_id()` lets agents pass a voice by name ("Morgan") instead of UUID.
|
||||
- **`MCPClientBinding` table** (new) via `Base.metadata.create_all` — no migration needed.
|
||||
- **Bindings REST:** `GET|PUT /mcp/bindings`, `DELETE /mcp/bindings/{client_id}`.
|
||||
- **`POST /speak`** REST wrapper for non-MCP callers (shell / ACP / A2A). Same `resolve_profile` precedence, same code path as the MCP tool.
|
||||
- **Stdio shim** at `backend/mcp_shim/__main__.py` — ~200 lines of `httpx` proxy; reads env (`VOICEBOX_PORT`, `VOICEBOX_HOST`, `VOICEBOX_CLIENT_ID`), waits for `/health`, then streams JSON-RPC ↔ SSE. Rolled our own after the `mcp` SDK's session-management helpers mis-shook-hands. Smoke-tested: `initialize`, `tools/list`, and `tools/call` all round-trip cleanly.
|
||||
- **Stdio shim** at `backend/mcp_shim/__main__.py` — ~200 lines of `httpx` proxy; reads env (`TALKBOX_PORT`, `TALKBOX_HOST`, `TALKBOX_CLIENT_ID`), waits for `/health`, then streams JSON-RPC ↔ SSE. Rolled our own after the `mcp` SDK's session-management helpers mis-shook-hands. Smoke-tested: `initialize`, `tools/list`, and `tools/call` all round-trip cleanly.
|
||||
- **Pill SSE:** `GET /events/speak` (`sse-starlette`) emits `speak-start` from the MCP tool and `POST /speak`, `speak-end` from `services/generation.run_generation`'s finally block.
|
||||
- **PyInstaller:**
|
||||
- `backend/build_binary.py` `--shim` flag builds a minimal `voicebox-mcp` binary (torch/transformers/mlx/etc. explicitly excluded, target <20 MB).
|
||||
- `backend/build_binary.py` `--shim` flag builds a minimal `talkbox-mcp` binary (torch/transformers/mlx/etc. explicitly excluded, target <20 MB).
|
||||
- The main server spec picks up `fastmcp`, `mcp`, `sse_starlette`, and `backend.mcp_server.*` via `--collect-all` / `--hidden-import`.
|
||||
- **`backend/mcp_server/README.md`** quickstart (Inspector, `.mcp.json` snippets, tool reference).
|
||||
|
||||
@@ -39,12 +39,12 @@
|
||||
- Router + `ServerTab` tab bar wired to `/settings/mcp`.
|
||||
|
||||
### Shipped (native shell)
|
||||
- **`tauri.conf.json`** — `voicebox-mcp` added to `externalBin` (alongside `voicebox-server`).
|
||||
- **`tauri.conf.json`** — `talkbox-mcp` added to `externalBin` (alongside `talkbox-server`).
|
||||
- **`dictate:show` listener** in `tauri/src-tauri/src/main.rs` — invokes a new `show_dictate_window(app_handle)` helper that mirrors the hotkey-monitor's position+show logic (undo click-through, reposition to top-center of the current monitor, show). Agent-initiated speech now pops the pill visible on screen.
|
||||
|
||||
### Validated end-to-end (this session, via curl)
|
||||
- `/mcp/` init → `tools/list` → `tools/call voicebox.speak` → actual audio plays (Jarvis, 1.68 s).
|
||||
- `POST /speak` with `X-Voicebox-Client-Id: claude-code` resolves to the bound Jarvis profile without passing `profile`.
|
||||
- `/mcp/` init → `tools/list` → `tools/call talkbox.speak` → actual audio plays (Jarvis, 1.68 s).
|
||||
- `POST /speak` with `X-TalkBox-Client-Id: claude-code` resolves to the bound Jarvis profile without passing `profile`.
|
||||
- `/events/speak` emits `ready`, `speak-start`, `speak-end` in order, generation_id threads through both.
|
||||
- Stdio shim: `echo {…} | python -m backend.mcp_shim` returns valid JSON-RPC for all 4 methods.
|
||||
- `last_seen_at` auto-stamps on first call; binding row auto-creates.
|
||||
@@ -52,8 +52,8 @@
|
||||
- `cargo check` on the Tauri crate: clean.
|
||||
|
||||
### Outstanding (must-do before release)
|
||||
- **CI build for shim on Windows/Linux** — `python backend/build_binary.py --shim` is wired up and built cleanly for `aarch64-apple-darwin` (18 MB, installed at `tauri/src-tauri/binaries/voicebox-mcp-aarch64-apple-darwin`, Tauri `cargo check` green). The Windows and Linux triples (`x86_64-pc-windows-msvc`, `x86_64-unknown-linux-gnu`) need the same build in their respective CI runners and artifacts dropped alongside the macOS binary.
|
||||
- **Windows/Linux paths in the stdio snippet** — the Settings page hardcodes the macOS path (`/Applications/Voicebox.app/Contents/MacOS/voicebox-mcp`). Needs a per-OS switch (`%LOCALAPPDATA%\Programs\Voicebox\voicebox-mcp.exe`, Linux bundled-path), ideally with the Tauri shell resolving its own app path at runtime and injecting it into the snippet.
|
||||
- **CI build for shim on Windows/Linux** — `python backend/build_binary.py --shim` is wired up and built cleanly for `aarch64-apple-darwin` (18 MB, installed at `tauri/src-tauri/binaries/talkbox-mcp-aarch64-apple-darwin`, Tauri `cargo check` green). The Windows and Linux triples (`x86_64-pc-windows-msvc`, `x86_64-unknown-linux-gnu`) need the same build in their respective CI runners and artifacts dropped alongside the macOS binary.
|
||||
- **Windows/Linux paths in the stdio snippet** — the Settings page hardcodes the macOS path (`/Applications/TalkBox.app/Contents/MacOS/talkbox-mcp`). Needs a per-OS switch (`%LOCALAPPDATA%\Programs\TalkBox\talkbox-mcp.exe`, Linux bundled-path), ideally with the Tauri shell resolving its own app path at runtime and injecting it into the snippet.
|
||||
|
||||
### Nice-to-have (follow-up passes)
|
||||
- **One-click install buttons** — write/merge into `~/.claude/settings.json`, `~/.cursor/mcp.json`, etc. via a Tauri command. Copy-paste works today; this is pure ergonomics.
|
||||
@@ -63,9 +63,9 @@
|
||||
|
||||
## Context
|
||||
|
||||
Voicebox already ships the I/O surface (Captures, Generate, personality-driven `/profiles/{id}/speak`), but local AI agents can't reach any of it. This plan adds a Model Context Protocol server so Claude Code / Cursor / Cline can call `voicebox.speak`, `voicebox.transcribe`, `voicebox.list_captures`, and `voicebox.list_profiles` — turning Voicebox into the local voice layer for every agent on the user's machine (Phase 5 of `docs/plans/VOICE_IO.md`).
|
||||
TalkBox already ships the I/O surface (Captures, Generate, personality-driven `/profiles/{id}/speak`), but local AI agents can't reach any of it. This plan adds a Model Context Protocol server so Claude Code / Cursor / Cline can call `talkbox.speak`, `talkbox.transcribe`, `talkbox.list_captures`, and `talkbox.list_profiles` — turning TalkBox into the local voice layer for every agent on the user's machine (Phase 5 of `docs/plans/VOICE_IO.md`).
|
||||
|
||||
The shortest path to "Claude Code speaks in a cloned voice": mount **FastMCP** inside the existing FastAPI/uvicorn process at `/mcp` (Streamable HTTP), and users install it as a URL (`{"url": "http://127.0.0.1:17493/mcp"}`) — the ecosystem-idiomatic shape for a long-running local service. Per-client voice binding via a new `mcp_client_bindings` table + Settings UI, resolved from an `X-Voicebox-Client-Id` header. A **stdio shim binary** `voicebox-mcp` is bundled as a fallback sidecar for clients that can't speak HTTP MCP. A public `POST /speak` REST wrapper covers non-MCP callers (shell scripts, ACP, A2A). A `speaking` pill state gives agent-initiated audio visibility — trust-critical, non-negotiable.
|
||||
The shortest path to "Claude Code speaks in a cloned voice": mount **FastMCP** inside the existing FastAPI/uvicorn process at `/mcp` (Streamable HTTP), and users install it as a URL (`{"url": "http://127.0.0.1:17494/mcp"}`) — the ecosystem-idiomatic shape for a long-running local service. Per-client voice binding via a new `mcp_client_bindings` table + Settings UI, resolved from an `X-TalkBox-Client-Id` header. A **stdio shim binary** `talkbox-mcp` is bundled as a fallback sidecar for clients that can't speak HTTP MCP. A public `POST /speak` REST wrapper covers non-MCP callers (shell scripts, ACP, A2A). A `speaking` pill state gives agent-initiated audio visibility — trust-critical, non-negotiable.
|
||||
|
||||
## Architecture
|
||||
|
||||
@@ -75,22 +75,22 @@ Claude Code / Cursor / Windsurf / VS Code MCP
|
||||
├─ HTTP (primary) ────────────────────┐
|
||||
│ {"url": ".../mcp"} │
|
||||
│ │
|
||||
└─ stdio (fallback) ───────────────▶ [voicebox-mcp shim binary]
|
||||
{"command": "/abs/path/voicebox-mcp"} (absolute path;
|
||||
└─ stdio (fallback) ───────────────▶ [talkbox-mcp shim binary]
|
||||
{"command": "/abs/path/talkbox-mcp"} (absolute path;
|
||||
│ Settings page
|
||||
│ copies it for you)
|
||||
▼
|
||||
uvicorn + FastAPI (port 17493)
|
||||
uvicorn + FastAPI (port 17494)
|
||||
├─ /mcp (FastMCP, Streamable HTTP)
|
||||
└─ /speak (REST wrapper for non-MCP callers)
|
||||
└─ tools call existing services
|
||||
```
|
||||
|
||||
- **Transport:** Streamable HTTP as primary (Nov-2025 spec, post-SSE). Claude Code, Cursor, Windsurf, and the VS Code MCP extensions all support HTTP — it's the idiomatic shape for a long-running local service, which Voicebox already is.
|
||||
- **Stdio fallback:** `voicebox-mcp` binary bundled inside the app for clients that can't speak HTTP MCP. The Settings page renders the exact snippet with the detected absolute path — user copies, pastes, done. No PATH manipulation, no custom CLI wrapper.
|
||||
- **Identity:** HTTP clients set `X-Voicebox-Client-Id` header in their MCP config's `headers` block. Stdio clients set `VOICEBOX_CLIENT_ID` env var, which the shim forwards as the same HTTP header. Server reads it into a `ContextVar`.
|
||||
- **Transport:** Streamable HTTP as primary (Nov-2025 spec, post-SSE). Claude Code, Cursor, Windsurf, and the VS Code MCP extensions all support HTTP — it's the idiomatic shape for a long-running local service, which TalkBox already is.
|
||||
- **Stdio fallback:** `talkbox-mcp` binary bundled inside the app for clients that can't speak HTTP MCP. The Settings page renders the exact snippet with the detected absolute path — user copies, pastes, done. No PATH manipulation, no custom CLI wrapper.
|
||||
- **Identity:** HTTP clients set `X-TalkBox-Client-Id` header in their MCP config's `headers` block. Stdio clients set `TALKBOX_CLIENT_ID` env var, which the shim forwards as the same HTTP header. Server reads it into a `ContextVar`.
|
||||
- **Profile resolution precedence:** explicit tool arg → per-client `MCPClientBinding.profile_id` → `capture_settings.default_playback_voice_id` → error.
|
||||
- **Port:** `17493`, matching `tauri/src-tauri/src/main.rs:63` (`SERVER_PORT` constant). Shim default with `VOICEBOX_PORT` env override.
|
||||
- **Port:** `17494`, matching `tauri/src-tauri/src/main.rs:63` (`SERVER_PORT` constant). Shim default with `TALKBOX_PORT` env override.
|
||||
- **Non-MCP access:** `POST /speak` is a thin REST wrapper around the same tool path — one endpoint for shell scripts, ACP, A2A, and anything that isn't MCP-native.
|
||||
|
||||
## Library choice
|
||||
@@ -132,7 +132,7 @@ Global default stays in `capture_settings.default_playback_voice_id` — no dupl
|
||||
| `backend/mcp/events.py` | In-memory `asyncio.Queue` pub/sub for speak-start / speak-end |
|
||||
| `backend/mcp/README.md` | MCP Inspector quickstart + `.mcp.json` snippets |
|
||||
| `backend/mcp_shim/__init__.py`, `__main__.py` | Stdio ↔ Streamable HTTP proxy (~150 lines) |
|
||||
| `backend/voicebox-mcp.spec` | PyInstaller spec for the shim (strips torch/transformers from `hiddenimports`) |
|
||||
| `backend/talkbox-mcp.spec` | PyInstaller spec for the shim (strips torch/transformers from `hiddenimports`) |
|
||||
| `backend/routes/speak.py` | `POST /speak {text, profile?, engine?, personality?, language?}` — REST wrapper around `resolve_profile()` + `generate_speech()` for non-MCP agents |
|
||||
|
||||
### Backend — modified
|
||||
@@ -150,8 +150,8 @@ Global default stays in `capture_settings.default_playback_voice_id` — no dupl
|
||||
| `backend/routes/mcp_bindings.py` (new) | REST CRUD for bindings (list, upsert, delete). |
|
||||
| `backend/routes/events.py` (new) | `GET /events/speak` — `EventSourceResponse` subscribed to the events queue. |
|
||||
| `backend/requirements.txt` | `+ fastmcp` (or `mcp>=1.0`), `+ sse-starlette` |
|
||||
| `backend/voicebox-server.spec` | `hiddenimports += ['mcp', 'mcp.server', 'fastmcp']` |
|
||||
| `backend/build_binary.py` | Second PyInstaller invocation for `voicebox-mcp.spec`; copy to `tauri/src-tauri/binaries/` with target-triple suffix |
|
||||
| `backend/talkbox-server.spec` | `hiddenimports += ['mcp', 'mcp.server', 'fastmcp']` |
|
||||
| `backend/build_binary.py` | Second PyInstaller invocation for `talkbox-mcp.spec`; copy to `tauri/src-tauri/binaries/` with target-triple suffix |
|
||||
|
||||
### Frontend — new
|
||||
|
||||
@@ -175,19 +175,19 @@ Global default stays in `capture_settings.default_playback_voice_id` — no dupl
|
||||
|
||||
| File | Change |
|
||||
|---|---|
|
||||
| `tauri/src-tauri/tauri.conf.json` | `"externalBin": ["binaries/voicebox-server", "binaries/voicebox-mcp"]` |
|
||||
| `tauri/src-tauri/binaries/voicebox-mcp-<triple>` | Build artifact from PyInstaller |
|
||||
| `tauri/src-tauri/tauri.conf.json` | `"externalBin": ["binaries/talkbox-server", "binaries/talkbox-mcp"]` |
|
||||
| `tauri/src-tauri/binaries/talkbox-mcp-<triple>` | Build artifact from PyInstaller |
|
||||
|
||||
## Tool signatures
|
||||
|
||||
All tools read `current_client_id.get()` (from middleware). Return JSON-serializable dicts.
|
||||
|
||||
Tools are registered with **dotted names** (`voicebox.speak`, etc.) to match the landing page and the industry convention (`filesystem.read_file`, `github.create_issue`). Python function names stay snake_case; the dot goes in the `name=` kwarg.
|
||||
Tools are registered with **dotted names** (`talkbox.speak`, etc.) to match the landing page and the industry convention (`filesystem.read_file`, `github.create_issue`). Python function names stay snake_case; the dot goes in the `name=` kwarg.
|
||||
|
||||
```python
|
||||
# backend/mcp/tools.py
|
||||
|
||||
@mcp.tool(name="voicebox.speak")
|
||||
@mcp.tool(name="talkbox.speak")
|
||||
async def speak(text: str,
|
||||
profile: str | None = None, # name OR id
|
||||
engine: str | None = None,
|
||||
@@ -198,7 +198,7 @@ async def speak(text: str,
|
||||
# route honors `personality=True` by running rewrite_as_profile on
|
||||
# the input before running the normal TTS pipeline.
|
||||
|
||||
@mcp.tool(name="voicebox.transcribe")
|
||||
@mcp.tool(name="talkbox.transcribe")
|
||||
async def transcribe(audio_base64: str | None = None,
|
||||
audio_path: str | None = None, # absolute local path
|
||||
language: str | None = None,
|
||||
@@ -206,11 +206,11 @@ async def transcribe(audio_base64: str | None = None,
|
||||
"""Transcribe audio. Exactly one of audio_base64/audio_path. Returns {text, duration, language}."""
|
||||
# validate path readable, size < 200 MB, then call services.transcribe.transcribe_bytes
|
||||
|
||||
@mcp.tool(name="voicebox.list_captures")
|
||||
@mcp.tool(name="talkbox.list_captures")
|
||||
async def list_captures(limit: int = 20, offset: int = 0) -> dict:
|
||||
"""Recent captures with transcripts. Returns {captures: [...]}"""
|
||||
|
||||
@mcp.tool(name="voicebox.list_profiles")
|
||||
@mcp.tool(name="talkbox.list_profiles")
|
||||
async def list_profiles() -> dict:
|
||||
"""Available voice profiles. Returns {profiles: [{id, name, voice_type, has_personality}]}"""
|
||||
```
|
||||
@@ -222,7 +222,7 @@ async def list_profiles() -> dict:
|
||||
@router.post("/speak", response_model=GenerationResponse)
|
||||
async def speak(data: SpeakRequest, request: Request, db: Session = Depends(get_db)):
|
||||
"""Same behavior as the MCP tool — for shell scripts, ACP, A2A, or anything non-MCP."""
|
||||
client_id = request.headers.get("X-Voicebox-Client-Id")
|
||||
client_id = request.headers.get("X-TalkBox-Client-Id")
|
||||
profile = resolve_profile(data.profile, client_id, db)
|
||||
if profile is None: raise HTTPException(400, "No voice profile resolved.")
|
||||
req = GenerationRequest(profile_id=profile.id, text=data.text,
|
||||
@@ -244,14 +244,14 @@ mount_into(application)
|
||||
|
||||
`mount_into` installs `ClientIdMiddleware` and calls `app.mount("/mcp", mcp.streamable_http_app())`.
|
||||
|
||||
**Lifespan migration is load-bearing** — FastMCP's session manager requires the `lifespan=` kwarg, not `@app.on_event`. Wrap the existing startup/shutdown bodies in an `@asynccontextmanager` using `contextlib.AsyncExitStack` so both Voicebox's init and FastMCP's session manager run. Verify dev + packaged build after the migration.
|
||||
**Lifespan migration is load-bearing** — FastMCP's session manager requires the `lifespan=` kwarg, not `@app.on_event`. Wrap the existing startup/shutdown bodies in an `@asynccontextmanager` using `contextlib.AsyncExitStack` so both TalkBox's init and FastMCP's session manager run. Verify dev + packaged build after the migration.
|
||||
|
||||
## Stdio shim (`backend/mcp_shim/__main__.py`)
|
||||
|
||||
1. Port: `int(os.environ.get("VOICEBOX_PORT", "17493"))`.
|
||||
2. Client id: `os.environ.get("VOICEBOX_CLIENT_ID", "unknown")`.
|
||||
1. Port: `int(os.environ.get("TALKBOX_PORT", "17494"))`.
|
||||
2. Client id: `os.environ.get("TALKBOX_CLIENT_ID", "unknown")`.
|
||||
3. Health probe `GET /health` with 30 s tolerance (torch imports slowly). On failure, emit JSON-RPC error on stdout, exit 1.
|
||||
4. Connect Streamable HTTP MCP client to `http://127.0.0.1:{port}/mcp` with `X-Voicebox-Client-Id: {client_id}` header.
|
||||
4. Connect Streamable HTTP MCP client to `http://127.0.0.1:{port}/mcp` with `X-TalkBox-Client-Id: {client_id}` header.
|
||||
5. Proxy JSON-RPC bidirectionally — stdin → HTTP, SSE → stdout. Use `mcp` SDK's built-in stdio↔HTTP bridge if available; otherwise ~40 lines of asyncio.
|
||||
6. Stdout = JSON-RPC only. All logs to stderr.
|
||||
|
||||
@@ -273,23 +273,23 @@ PyInstaller spec keeps only `mcp`, `httpx`, `anyio`, `click` — target binary <
|
||||
|
||||
HTTP form (primary):
|
||||
```json
|
||||
{"mcpServers": {"voicebox": {
|
||||
"url": "http://127.0.0.1:17493/mcp",
|
||||
"headers": {"X-Voicebox-Client-Id": "claude-code"}
|
||||
{"mcpServers": {"talkbox": {
|
||||
"url": "http://127.0.0.1:17494/mcp",
|
||||
"headers": {"X-TalkBox-Client-Id": "claude-code"}
|
||||
}}}
|
||||
```
|
||||
|
||||
Stdio form (fallback, absolute path auto-filled from detected app location):
|
||||
```json
|
||||
{"mcpServers": {"voicebox": {
|
||||
"command": "/Applications/Voicebox.app/Contents/MacOS/voicebox-mcp",
|
||||
"env": {"VOICEBOX_CLIENT_ID": "claude-code"}
|
||||
{"mcpServers": {"talkbox": {
|
||||
"command": "/Applications/TalkBox.app/Contents/MacOS/talkbox-mcp",
|
||||
"env": {"TALKBOX_CLIENT_ID": "claude-code"}
|
||||
}}}
|
||||
```
|
||||
|
||||
Plus the Claude-Code-specific one-liner:
|
||||
```
|
||||
claude mcp add voicebox --transport http --url http://127.0.0.1:17493/mcp --header "X-Voicebox-Client-Id: claude-code"
|
||||
claude mcp add talkbox --transport http --url http://127.0.0.1:17494/mcp --header "X-TalkBox-Client-Id: claude-code"
|
||||
```
|
||||
- **One-click install buttons** for known clients (v1: Claude Code via `claude mcp add` invocation, and a config-file writer for Cursor/Windsurf whose config locations are known). Each has a matching "Remove" button. Hide buttons for clients not detected on disk.
|
||||
- **Connection status** — small indicator next to each binding showing the last time that `client_id` actually called the server (rolling timestamp recorded by middleware), so users can tell their install worked.
|
||||
@@ -297,26 +297,26 @@ PyInstaller spec keeps only `mcp`, `httpx`, `anyio`, `click` — target binary <
|
||||
## Ordered task list (shortest path first)
|
||||
|
||||
1. `fastmcp` + `sse-starlette` → `backend/requirements.txt`; install.
|
||||
2. Add `backend/mcp/{server,tools,context,resolve}.py` with the 4 tools registered as `voicebox.speak` etc. (no middleware yet — global default profile only).
|
||||
2. Add `backend/mcp/{server,tools,context,resolve}.py` with the 4 tools registered as `talkbox.speak` etc. (no middleware yet — global default profile only).
|
||||
3. Migrate `app.py` to `lifespan=`; mount FastMCP at `/mcp`.
|
||||
4. **Milestone:** `npx @modelcontextprotocol/inspector http://127.0.0.1:17493/mcp` — call `voicebox.speak`, hear audio.
|
||||
4. **Milestone:** `npx @modelcontextprotocol/inspector http://127.0.0.1:17494/mcp` — call `talkbox.speak`, hear audio.
|
||||
5. Add `get_profile_by_name_or_id`; wire the tool's `profile` arg.
|
||||
6. `MCPClientBinding` model + migration; middleware; full `resolve_profile` precedence.
|
||||
7. `backend/routes/speak.py` — `POST /speak` REST wrapper, reusing `resolve_profile` + `speak_in_character`.
|
||||
8. `/mcp/bindings` REST + `MCPBindings.tsx` UI with HTTP and stdio copy-snippets, one-click install for detected clients, and connection-status indicators. **Users can install Voicebox as an MCP server after this step.**
|
||||
9. `backend/mcp_shim/__main__.py` + PyInstaller spec + `build_binary.py` second pass; register `voicebox-mcp` as a Tauri sidecar. (Fallback path goes live.)
|
||||
8. `/mcp/bindings` REST + `MCPBindings.tsx` UI with HTTP and stdio copy-snippets, one-click install for detected clients, and connection-status indicators. **Users can install TalkBox as an MCP server after this step.**
|
||||
9. `backend/mcp_shim/__main__.py` + PyInstaller spec + `build_binary.py` second pass; register `talkbox-mcp` as a Tauri sidecar. (Fallback path goes live.)
|
||||
10. Events queue + `/events/speak` SSE + `DictateWindow` `speaking` pill state.
|
||||
11. `backend/mcp/README.md` quickstart.
|
||||
|
||||
Claude Code can call `voicebox.speak` after step 4 (direct HTTP, manual config). Step 8 makes that a one-click experience. Step 9 adds the stdio fallback for clients that don't speak HTTP MCP.
|
||||
Claude Code can call `talkbox.speak` after step 4 (direct HTTP, manual config). Step 8 makes that a one-click experience. Step 9 adds the stdio fallback for clients that don't speak HTTP MCP.
|
||||
|
||||
## Verification
|
||||
|
||||
- **Step 4 smoke:** `npx @modelcontextprotocol/inspector http://127.0.0.1:17493/mcp`. Call `voicebox.list_profiles`, then `voicebox.speak(text="hello from mcp")`. Audio plays; generation appears in History with `source="personality_speak"` (or new `source="mcp"` if we add one).
|
||||
- **REST wrapper:** `curl -X POST http://127.0.0.1:17493/speak -d '{"text":"hi","profile":"Morgan"}'` — same behavior, same pill surface.
|
||||
- **Per-client:** open two Inspector sessions with different `X-Voicebox-Client-Id` headers, bind each to a different profile in Settings, verify distinct voices without `profile` arg.
|
||||
- **Claude Code end-to-end (HTTP):** `claude mcp add voicebox --transport http --url http://127.0.0.1:17493/mcp --header "X-Voicebox-Client-Id: claude-code"`, then ask Claude Code to speak. Pill shows `speaking: <profile>`, audio plays, capture appears in history.
|
||||
- **Stdio fallback:** manually paste the stdio snippet from Settings into a client's config, verify same behavior. `VOICEBOX_CLIENT_ID=claude-code python -m backend.mcp_shim` while backend is up; pipe a tools/list JSON-RPC in, verify response over stdout.
|
||||
- **Step 4 smoke:** `npx @modelcontextprotocol/inspector http://127.0.0.1:17494/mcp`. Call `talkbox.list_profiles`, then `talkbox.speak(text="hello from mcp")`. Audio plays; generation appears in History with `source="personality_speak"` (or new `source="mcp"` if we add one).
|
||||
- **REST wrapper:** `curl -X POST http://127.0.0.1:17494/speak -d '{"text":"hi","profile":"Morgan"}'` — same behavior, same pill surface.
|
||||
- **Per-client:** open two Inspector sessions with different `X-TalkBox-Client-Id` headers, bind each to a different profile in Settings, verify distinct voices without `profile` arg.
|
||||
- **Claude Code end-to-end (HTTP):** `claude mcp add talkbox --transport http --url http://127.0.0.1:17494/mcp --header "X-TalkBox-Client-Id: claude-code"`, then ask Claude Code to speak. Pill shows `speaking: <profile>`, audio plays, capture appears in history.
|
||||
- **Stdio fallback:** manually paste the stdio snippet from Settings into a client's config, verify same behavior. `TALKBOX_CLIENT_ID=claude-code python -m backend.mcp_shim` while backend is up; pipe a tools/list JSON-RPC in, verify response over stdout.
|
||||
- **Transcribe:** point at `/tmp/test.wav`; diff against `POST /transcribe` response.
|
||||
- **Failure modes:** kill backend mid-speak — shim must surface a JSON-RPC error, not deadlock. When backend isn't running, HTTP clients should get a clear connection-refused surfaced by the client.
|
||||
|
||||
@@ -326,8 +326,8 @@ Claude Code can call `voicebox.speak` after step 4 (direct HTTP, manual config).
|
||||
- **Lifespan migration** touches critical path (DB init, task queue, watchdog). Dev + packaged build both need a smoke after.
|
||||
- **Shim binary size** — if `mcp` pulls in enough dep weight that PyInstaller output is awkward, fall back to a Rust shim (Tauri shell is already Rust; JSON-RPC framing is trivial).
|
||||
- **Source attribution** — consider `source="mcp"` on the `Generation` model, or a dedicated `originator_client` column, if the Captures tab should filter MCP-originated generations.
|
||||
- **`audio_path` in `voicebox_transcribe`** — local-only today, but if the server ever binds beyond 127.0.0.1 we need to restrict reads to `data_dir` + user-whitelist.
|
||||
- **Auth** — none for now (127.0.0.1 only). If we bind outside, bearer token via `~/.voicebox/secret` + plumb through shim.
|
||||
- **`audio_path` in `talkbox_transcribe`** — local-only today, but if the server ever binds beyond 127.0.0.1 we need to restrict reads to `data_dir` + user-whitelist.
|
||||
- **Auth** — none for now (127.0.0.1 only). If we bind outside, bearer token via `~/.talkbox/secret` + plumb through shim.
|
||||
- **HTTP MCP client support** — the plan leads with direct HTTP. Claude Code, Cursor, Windsurf, and VS Code MCP extensions all support it as of 2026, but if we discover an important client is stdio-only we still have the shim fallback ready.
|
||||
- **`.mcpb` desktop extension for Claude Desktop** (v2 polish) — Claude Desktop supports a double-clickable extension bundle format. Worth revisiting after v1 ships for an even cleaner install; skipped for now since Claude Desktop isn't the primary user (Claude Code + IDE users are).
|
||||
|
||||
|
||||
@@ -2,11 +2,11 @@
|
||||
|
||||
**Status:** Planned for v0.2.0
|
||||
|
||||
**Issue:** [#10 OpenAI API compatibility](https://github.com/jamiepine/voicebox/issues/10)
|
||||
**Issue:** [#10 OpenAI API compatibility](https://github.com/jamiepine/talkbox/issues/10)
|
||||
|
||||
## Overview
|
||||
|
||||
This feature exposes OpenAI-compatible endpoints from Voicebox, allowing any tool, library, or application that speaks the OpenAI Audio API to use Voicebox as a drop-in local replacement.
|
||||
This feature exposes OpenAI-compatible endpoints from TalkBox, allowing any tool, library, or application that speaks the OpenAI Audio API to use TalkBox as a drop-in local replacement.
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
@@ -16,7 +16,7 @@ flowchart LR
|
||||
Apps[Third-party Apps]
|
||||
end
|
||||
|
||||
subgraph voicebox [Voicebox Server]
|
||||
subgraph talkbox [TalkBox Server]
|
||||
OpenAI["/v1/audio/* endpoints"]
|
||||
TTS[TTSModel]
|
||||
Whisper[WhisperModel]
|
||||
@@ -33,8 +33,8 @@ flowchart LR
|
||||
|
||||
## Use Cases
|
||||
|
||||
- **OpenAI SDK users**: `openai.audio.speech.create()` works with Voicebox
|
||||
- **LLM frameworks**: LangChain, AutoGen, etc. can use Voicebox for TTS
|
||||
- **OpenAI SDK users**: `openai.audio.speech.create()` works with TalkBox
|
||||
- **LLM frameworks**: LangChain, AutoGen, etc. can use TalkBox for TTS
|
||||
- **Shell scripts**: `curl` commands copy-pasted from OpenAI docs work
|
||||
- **Existing integrations**: Any tool expecting OpenAI's API works without code changes
|
||||
|
||||
@@ -60,7 +60,7 @@ OpenAI spec: https://platform.openai.com/docs/api-reference/audio/createSpeech
|
||||
|
||||
**Voice Mapping Strategy:**
|
||||
|
||||
- `voice` parameter maps to Voicebox profile names (case-insensitive)
|
||||
- `voice` parameter maps to TalkBox profile names (case-insensitive)
|
||||
- If no match, use a configurable default profile
|
||||
- Support special syntax: `voice: "profile:uuid"` for explicit profile ID
|
||||
|
||||
@@ -132,7 +132,7 @@ Add helper in [backend/profiles.py](backend/profiles.py):
|
||||
```python
|
||||
async def resolve_voice_for_openai(voice: str, db: Session) -> Optional[VoiceProfile]:
|
||||
"""
|
||||
Resolve OpenAI voice parameter to a Voicebox profile.
|
||||
Resolve OpenAI voice parameter to a TalkBox profile.
|
||||
|
||||
Priority:
|
||||
1. Exact profile name match (case-insensitive)
|
||||
|
||||
+30
-30
@@ -64,7 +64,7 @@ Paste half (macOS):
|
||||
`NSRunningApplication.activateWithOptions:` for re-activation.
|
||||
- `accessibility.rs` — `AXIsProcessTrusted` gate.
|
||||
- `paste_final_text` command — activate → 120 ms settle → save clip →
|
||||
write text → ⌘V → 400 ms → restore. Skips when focus was in Voicebox
|
||||
write text → ⌘V → 400 ms → restore. Skips when focus was in TalkBox
|
||||
itself.
|
||||
- Focus rides the `dictate:start` event payload; `DictateWindow` holds the
|
||||
snapshot in a ref and consume-once-nulls on paste so a late-arriving
|
||||
@@ -75,13 +75,13 @@ Paste half (macOS):
|
||||
Outstanding: Windows `SendInput` / UIAutomation / `SetForegroundWindow`
|
||||
equivalents, Linux `uinput` / AT-SPI equivalents (and the Wayland story),
|
||||
first-run Accessibility prompt UI with deep-link to System Settings,
|
||||
direct-injection path for focus-was-inside-Voicebox (step 6 — dictating
|
||||
direct-injection path for focus-was-inside-TalkBox (step 6 — dictating
|
||||
into our own Generate tab currently falls back to the capture list).
|
||||
|
||||
### Not started
|
||||
|
||||
- **Phase 5 — Agent voice output + persona loop.** No `/speak` endpoint, no
|
||||
`voicebox.speak` MCP tool, no per-agent voice binding, no persona metadata
|
||||
`talkbox.speak` MCP tool, no per-agent voice binding, no persona metadata
|
||||
on profiles.
|
||||
- **Phase 6 — STT engine expansion.** Only Whisper (`mlx_backend.py`).
|
||||
Parakeet v3, Qwen3-ASR, Kyutai — all unregistered.
|
||||
@@ -120,7 +120,7 @@ Called out in recent sessions but not yet in a phase:
|
||||
|
||||
## Overview
|
||||
|
||||
Voicebox ships the output half of a voice I/O loop: clone a voice, generate
|
||||
TalkBox ships the output half of a voice I/O loop: clone a voice, generate
|
||||
speech, apply effects, compose multi-voice projects. The input half — speech to
|
||||
text, dictation, routing — exists today as a single Whisper model wired into the
|
||||
Recording & Transcription panel. This doc proposes making voice *input* a
|
||||
@@ -128,18 +128,18 @@ first-class pillar: more STT engines, a dictation shell (global hotkey, audio
|
||||
capture, paste, streaming), a local LLM backend, and a user-configurable
|
||||
pipeline from captured audio to whatever the user wants to do with it.
|
||||
|
||||
Positioning is the key move. **Voicebox becomes the local voice I/O layer for
|
||||
Positioning is the key move. **TalkBox becomes the local voice I/O layer for
|
||||
humans and AI agents** — a local alternative to cloud dictation tools, with the
|
||||
differentiator that we also do TTS and voice cloning. The same app that
|
||||
captures your voice can generate a response in any voice profile you've
|
||||
cloned. "Anything voice is Voicebox."
|
||||
cloned. "Anything voice is TalkBox."
|
||||
|
||||
### Positioning shift
|
||||
|
||||
Before this plan, Voicebox was **"the open-source AI voice cloning studio."**
|
||||
Before this plan, TalkBox was **"the open-source AI voice cloning studio."**
|
||||
Cloning was the headline capability.
|
||||
|
||||
After this plan, Voicebox is **"the open-source AI voice studio."** Cloning is
|
||||
After this plan, TalkBox is **"the open-source AI voice studio."** Cloning is
|
||||
one capability in a broader category that now spans input (STT, dictation),
|
||||
intelligence (local LLM, refinement, persona), output (TTS, cloning, effects,
|
||||
Stories), and routing. The word "cloning" drops out of the top-line descriptor
|
||||
@@ -147,7 +147,7 @@ because it's become a feature rather than the thesis.
|
||||
|
||||
### Competitive frame
|
||||
|
||||
Voicebox ends up covering the territory of two separately-funded, separately
|
||||
TalkBox ends up covering the territory of two separately-funded, separately
|
||||
branded cloud incumbents that operate on opposite sides of the same voice I/O
|
||||
loop:
|
||||
|
||||
@@ -155,7 +155,7 @@ loop:
|
||||
- **WisprFlow** (~$70M raised): voice dictation for agents and power users —
|
||||
the "users talk" side
|
||||
|
||||
Both are cloud-only. Voicebox becomes the only local alternative to either,
|
||||
Both are cloud-only. TalkBox becomes the only local alternative to either,
|
||||
running in one app, with a single model directory and LLM shared between input
|
||||
and output. That bridging — dictation → LLM → TTS with a cloned voice in the
|
||||
middle — is the thing no single incumbent can match, because neither has the
|
||||
@@ -167,10 +167,10 @@ These are not engineering tasks but should ride the Phase 4 ship so marketing
|
||||
and positioning stay in sync with the product.
|
||||
|
||||
- **README.md** — drop "cloning" from the top-line descriptor. Add a section
|
||||
that explicitly frames Voicebox as "the open-source local alternative to
|
||||
that explicitly frames TalkBox as "the open-source local alternative to
|
||||
WisprFlow and ElevenLabs." Competitive framing belongs in the README and on
|
||||
the landing page — not in-app (reads as defensive).
|
||||
- **voicebox.sh landing page** — same positioning shift.
|
||||
- **talkbox.sh landing page** — same positioning shift.
|
||||
- **GitHub About / repo topics** — swap "voice-cloning" or similar tags for
|
||||
broader "voice-io," "local-tts," "local-stt," etc.
|
||||
- **Release notes** — the Phase 4 launch note is the "we're now voice I/O" moment.
|
||||
@@ -187,11 +187,11 @@ and positioning stay in sync with the product.
|
||||
- The **persona loop** — speak to an agent, have it reply in a cloned voice —
|
||||
is a feature only we can ship. Nobody with a dictation product has TTS; nobody
|
||||
with a TTS product has good dictation. The full duplex is ours.
|
||||
- Agent harnesses already pipe Voicebox TTS into their stacks. Giving those
|
||||
users STT from the same app closes the loop and makes Voicebox the default
|
||||
- Agent harnesses already pipe TalkBox TTS into their stacks. Giving those
|
||||
users STT from the same app closes the loop and makes TalkBox the default
|
||||
voice I/O layer for the agentic dev-tool crowd.
|
||||
- **Typing a 2,000-character TTS script is user-hostile.** The most immediate
|
||||
internal win is dictating directly into Voicebox's own generation form —
|
||||
internal win is dictating directly into TalkBox's own generation form —
|
||||
speak the script, generate the voice. This dogfoods the whole STT pipeline
|
||||
without touching a single OS-level API.
|
||||
- **Voice-to-voice models are landing.** Moshi (Kyutai), GLM-4-Voice, Qwen2.5
|
||||
@@ -202,7 +202,7 @@ and positioning stay in sync with the product.
|
||||
## Non-goals
|
||||
|
||||
- Cloud fallback or "bring your own API key" STT/LLM. Local is the product.
|
||||
- A separate tray-only dictation app. We extend Voicebox, not fork it.
|
||||
- A separate tray-only dictation app. We extend TalkBox, not fork it.
|
||||
- Replacing the Stories editor with a notes layout. Long-form capture is a
|
||||
preset on top of the pipeline, not a new product surface.
|
||||
- Real-time translation UI. It can exist as a transform later, but it's not in
|
||||
@@ -262,7 +262,7 @@ API call (WS / HTTP) ──┘ MCP server sink
|
||||
|
||||
`Source → Transform → Sink` is internal, dataflow-style vocabulary (same shape
|
||||
as Unix pipes, Apache Beam, Kafka) — not user-facing. The UI surface will use
|
||||
Voicebox-native language (see open questions).
|
||||
TalkBox-native language (see open questions).
|
||||
|
||||
Concrete preset examples this shape enables:
|
||||
|
||||
@@ -310,7 +310,7 @@ branching behavior, not four separate sinks.
|
||||
|
||||
| Target | Delivery strategy |
|
||||
|---|---|
|
||||
| Focused text field inside Voicebox | Direct React state update via event. No clipboard involved. |
|
||||
| Focused text field inside TalkBox | Direct React state update via event. No clipboard involved. |
|
||||
| Focused text field in another app | Accessibility-verified paste: save clipboard, write transcript, simulate paste, restore clipboard. |
|
||||
| No text focus detected | Clipboard only, toast notification ("Transcript copied — no text field focused"). |
|
||||
| Platform-specific special cases (terminal apps, specific editors) | Per-app overrides where the generic path misbehaves. |
|
||||
@@ -355,7 +355,7 @@ Generate · Stories · Captures · Voices · Effects · Models · Settings.
|
||||
### Parallel explainer on the Generate tab
|
||||
|
||||
The Captures settings page gets a "What's different" aside that introduces
|
||||
Voicebox's dictation story. The Generate tab deserves a parallel — first-time
|
||||
TalkBox's dictation story. The Generate tab deserves a parallel — first-time
|
||||
users need to be told what voice generation is *for* in a post-Voice-I/O
|
||||
world, not just handed a text field.
|
||||
|
||||
@@ -394,13 +394,13 @@ Dictation is one half of the loop — user speaks, agent listens. The other half
|
||||
first-class primitive rather than being buried as a TTS loopback sink or a
|
||||
consumer read-aloud button.
|
||||
|
||||
The shape is a single new capability: any agent can call Voicebox to speak
|
||||
The shape is a single new capability: any agent can call TalkBox to speak
|
||||
arbitrary text in a user-configured voice. The same pill that surfaces during
|
||||
dictation surfaces during agent speech, so the user always sees what's coming
|
||||
out of their machine.
|
||||
|
||||
```
|
||||
MCP tool: voicebox.speak({ text, profile?, style? })
|
||||
MCP tool: talkbox.speak({ text, profile?, style? })
|
||||
REST: POST /speak { text, profile_id?, style? }
|
||||
```
|
||||
|
||||
@@ -422,7 +422,7 @@ audio through system output, and surface the pill in a `speaking` state.
|
||||
- **Mute + rate limits.** One-toggle mute for all agent speech. Per-source
|
||||
rate limits prevent a runaway agent from monologuing.
|
||||
|
||||
This primitive is what makes "Voicebox as voice layer for every agent on your
|
||||
This primitive is what makes "TalkBox as voice layer for every agent on your
|
||||
machine" a concrete shipping capability rather than marketing language. MCP,
|
||||
ACP, and A2A integrations all slot into it — none of those agent protocols
|
||||
need to know anything about TTS models, GPU placement, or voice profiles.
|
||||
@@ -517,7 +517,7 @@ tracking these models.
|
||||
1. **Tab name.** Leaning **Captures** — neutral, extensible across dictation,
|
||||
long-form recordings, and uploaded audio without repainting the tab later.
|
||||
"Dictations" is narrower (office-productivity coded, doesn't fit meeting
|
||||
recordings). "Notes" is the wrong mental model — nobody opens Voicebox to
|
||||
recordings). "Notes" is the wrong mental model — nobody opens TalkBox to
|
||||
write notes. "Transcriptions" is flat.
|
||||
2. **Refinement vocabulary.** The LLM-post-STT step needs a user-facing name.
|
||||
"Refine," "polish," "rewrite," "smart edit" are candidates. "Refinement" in
|
||||
@@ -532,7 +532,7 @@ tracking these models.
|
||||
5. **Long-form capture product surface.** Pure preset, or dedicated entry point
|
||||
in the new tab? Leaning preset, but long-form is the feature that most
|
||||
justifies its own landing page.
|
||||
6. **Hotkey primitive naming.** Hold-vs-tap needs Voicebox-native phrasing in
|
||||
6. **Hotkey primitive naming.** Hold-vs-tap needs TalkBox-native phrasing in
|
||||
UI copy. Settings can still use industry-standard terms.
|
||||
|
||||
## Ordered phases
|
||||
@@ -565,9 +565,9 @@ No new runtime. No `llama.cpp`, no `ollama`, no fragmented model cache.
|
||||
|
||||
### Phase 3 — In-app voice input
|
||||
|
||||
A universal mic button on every Voicebox text input. Hold, speak, release —
|
||||
A universal mic button on every TalkBox text input. Hold, speak, release —
|
||||
text lands in the focused field via direct React state update. No OS APIs
|
||||
involved; Voicebox owns the input.
|
||||
involved; TalkBox owns the input.
|
||||
|
||||
Marquee use cases:
|
||||
|
||||
@@ -593,7 +593,7 @@ of the persona loop and it lands here for free — no LLM involved, no new
|
||||
backend endpoints, just a Captures-tab button that sends the transcript text
|
||||
to the existing `/generate` endpoint with a user-selected voice profile and
|
||||
plays the result. Category-defining differentiator from the v1 prototype
|
||||
onward: Superwhisper and WisprFlow cannot do this because they have no TTS. Voicebox can, with one day of frontend wiring.
|
||||
onward: Superwhisper and WisprFlow cannot do this because they have no TTS. TalkBox can, with one day of frontend wiring.
|
||||
|
||||
Keep it aggressively minimal on day one. A capture list, a detail view, a
|
||||
model picker, a Play-as-voice dropdown. Refinement prompt editing, correction
|
||||
@@ -602,11 +602,11 @@ Tier-2 work when someone actually asks for them.
|
||||
|
||||
### Phase 5 — Agent voice output + persona loop
|
||||
|
||||
Two features that together make "Voicebox as the voice layer for every agent
|
||||
Two features that together make "TalkBox as the voice layer for every agent
|
||||
on your machine" a shipping reality:
|
||||
|
||||
1. **`speak()` primitive.** New `POST /speak` endpoint and `voicebox.speak`
|
||||
MCP tool. Any agent calls Voicebox to speak arbitrary text in a
|
||||
1. **`speak()` primitive.** New `POST /speak` endpoint and `talkbox.speak`
|
||||
MCP tool. Any agent calls TalkBox to speak arbitrary text in a
|
||||
user-configured voice; the pill surfaces in a `speaking` state. Settings
|
||||
UI for default voice, per-agent voice binding (Claude Code → Morgan,
|
||||
Cursor → Scarlett), and a global mute.
|
||||
|
||||
Reference in New Issue
Block a user