rebrand: rename VoiceBox to TalkBox throughout codebase
CI / frontend-quality (push) Canceled after 0s

- All 'voicebox'/'Voicebox'/'VOICEBOX' strings replaced with 'talkbox'/'TalkBox'/'TALKBOX'
- Port changed from 17493 to 17494 (avoids conflict with upstream VoiceBox)
- MCP tool namespace: voicebox.* -> talkbox.*
- App bundle ID: sh.voicebox.app -> com.talkbox.app
- Binary names: voicebox-server -> talkbox-server, voicebox-mcp -> talkbox-mcp
- Docker user/group: voicebox -> talkbox
- Database: voicebox.db -> talkbox.db
- Env vars: VOICEBOX_* -> TALKBOX_*
- Asset files renamed: voicebox-logo.* -> talkbox-logo.*, etc.
- External binaries in tauri.conf.json updated to talkbox-server/talkbox-mcp
This commit is contained in:
2026-08-24 19:45:56 -07:00
parent eaef8dd838
commit b8815e94ea
205 changed files with 1593 additions and 1593 deletions
+11 -11
View File
@@ -1,8 +1,8 @@
# Voicebox Cloud Roadmap
# TalkBox Cloud Roadmap
The post-mobile commercial trajectory. Captures the strategic arc beyond `mobile/PLAN.md` — what Voicebox becomes once the mobile companion ships and we start layering optional cloud services on top of the local-first base.
The post-mobile commercial trajectory. Captures the strategic arc beyond `mobile/PLAN.md` — what TalkBox becomes once the mobile companion ships and we start layering optional cloud services on top of the local-first base.
The desktop app stays free. Paid surface is the cloud layer, gated behind a Voicebox account, designed so the server sees as little as possible.
The desktop app stays free. Paid surface is the cloud layer, gated behind a TalkBox account, designed so the server sees as little as possible.
---
@@ -14,7 +14,7 @@ See [`mobile/PLAN.md`](../../mobile/PLAN.md). Entirely local: paired-device keys
### Phase 1 — Backup & Sync (next big feature)
First introduction of a Voicebox cloud account. Server stores **only encrypted blobs**.
First introduction of a TalkBox cloud account. Server stores **only encrypted blobs**.
- **E2E encryption keyed off the device key** from the mobile pairing flow. Audio + transcript blobs are encrypted client-side before upload; the server never has the plaintext or the key.
- **Quota by number of generations**, not by storage GB. Avoids "how many GB do you offer" framing and keeps tiering legible. (Word-count quotas are an alternative — closer to the ElevenLabs model — but generations are simpler to communicate.)
@@ -28,7 +28,7 @@ The privacy framing is load-bearing. "We see encrypted blobs and that's it" is t
The big bet. Today there is no major neutral voice-inference provider — every cloud TTS service ships its own proprietary models. Open-source TTS models exist and keep getting better, but nobody runs them as a paid hosted catalog at scale.
Voicebox already has the distribution. The thesis is: the same users who chose local-first specifically to avoid sending voice data to ElevenLabs will pay a fair markup to run open-source voices on hosted GPUs **when they don't have local hardware** (mobile-only users, low-end laptops, "I just don't want to manage CUDA"), provided the privacy story stays consistent.
TalkBox already has the distribution. The thesis is: the same users who chose local-first specifically to avoid sending voice data to ElevenLabs will pay a fair markup to run open-source voices on hosted GPUs **when they don't have local hardware** (mobile-only users, low-end laptops, "I just don't want to manage CUDA"), provided the privacy story stays consistent.
- **Catalog-first positioning.** Cloud can offer more voices than the desktop binary bundles (the bundle is already 500MB without CUDA, ~3GB with — there's a hard ceiling on what we can ship locally). Catalog grows over time.
- **Pricing tiers (rough first cut):** $5 / $15 / $25 / month, plus Enterprise. Final numbers depend on benchmarking — see below.
@@ -71,16 +71,16 @@ Scale-to-zero where cold start fits the budget. Hot engines need warm pools size
**Profile pipeline.** Cloned voice → encrypted blob with user-account-key → uploaded to R2 cold storage → fetched into worker memory at job start → decrypted in memory only, never written to worker disk → discarded on worker idle. TTL applies at the R2 layer (cold storage retention); worker hot-path retention is bounded by warmup window. Embedding-only caching where the engine exposes a stable embedding interface; raw-audio caching is the fallback. Per-engine audit needed before launch — see open questions.
**Hybrid routing on the client.** Desktop, mobile, and MCP clients already speak `127.0.0.1:17493`. Add `VOICEBOX_API_URL` + `VOICEBOX_API_KEY` plus a routing function:
**Hybrid routing on the client.** Desktop, mobile, and MCP clients already speak `127.0.0.1:17494`. Add `TALKBOX_API_URL` + `TALKBOX_API_KEY` plus a routing function:
```
if local_backend_reachable() and engine in local_engines:
→ 127.0.0.1:17493
→ 127.0.0.1:17494
else:
→ api.voicebox.sh/v1
→ api.talkbox.sh/v1
```
Mobile-without-paired-desktop falls through to cloud automatically. Desktop without a usable GPU falls through for big engines, stays local for Kokoro. Same `voicebox.speak()` MCP call works either way. This is the differentiator versus ElevenLabs (cloud-only) and pure local-first competitors (no fallback).
Mobile-without-paired-desktop falls through to cloud automatically. Desktop without a usable GPU falls through for big engines, stays local for Kokoro. Same `talkbox.speak()` MCP call works either way. This is the differentiator versus ElevenLabs (cloud-only) and pure local-first competitors (no fallback).
#### Cloud-cached voice profiles
@@ -99,7 +99,7 @@ Inference latency makes it untenable to re-upload reference samples per call. Cl
A marketplace where voice owners license their cloned voices for others to use, with revenue sharing. Possibly: "rent out your AI voice."
This is the only phase that requires hosting voice profiles, and it requires real licensing infrastructure first — consent verification, takedown flow, identity claims, revenue accounting. Until that exists, **Voicebox does not host voice profiles in cloud at all** (see constraint below). Marketplace is the long-term endgame, not the next quarter.
This is the only phase that requires hosting voice profiles, and it requires real licensing infrastructure first — consent verification, takedown flow, identity claims, revenue accounting. Until that exists, **TalkBox does not host voice profiles in cloud at all** (see constraint below). Marketplace is the long-term endgame, not the next quarter.
---
@@ -119,7 +119,7 @@ This protects two things at once:
### Privacy is the moat, not a feature
The "private LLM users → ElevenLabs voice" workflow is incoherent: people pay to keep their text private and then hand their speech to a cloud vendor that trains on it. Voicebox is the consistent answer for that audience. Every cloud feature should be designed so a privacy-conscious user can adopt it without breaking that internal consistency — which is why Phase 1 is fully E2E and Phase 2 is "audited no-log" rather than "we have your audio but trust us."
The "private LLM users → ElevenLabs voice" workflow is incoherent: people pay to keep their text private and then hand their speech to a cloud vendor that trains on it. TalkBox is the consistent answer for that audience. Every cloud feature should be designed so a privacy-conscious user can adopt it without breaking that internal consistency — which is why Phase 1 is fully E2E and Phase 2 is "audited no-log" rather than "we have your audio but trust us."
### Revenue stack is multi-source
+71 -71
View File
@@ -5,7 +5,7 @@
## Overview
Docker support makes Voicebox easier to deploy, especially for:
Docker support makes TalkBox easier to deploy, especially for:
- **Consistent Environments**: Same setup across dev/staging/prod
- **GPU Passthrough**: Easy NVIDIA/AMD GPU access
@@ -19,17 +19,17 @@ Docker support makes Voicebox easier to deploy, especially for:
```bash
# CPU-only version
docker run -p 8000:8000 -v voicebox-data:/app/data \
ghcr.io/jamiepine/voicebox:latest
docker run -p 8000:8000 -v talkbox-data:/app/data \
ghcr.io/jamiepine/talkbox:latest
# NVIDIA GPU version
docker run --gpus all -p 8000:8000 -v voicebox-data:/app/data \
ghcr.io/jamiepine/voicebox:latest-cuda
docker run --gpus all -p 8000:8000 -v talkbox-data:/app/data \
ghcr.io/jamiepine/talkbox:latest-cuda
# AMD GPU version (experimental)
docker run --device=/dev/kfd --device=/dev/dri -p 8000:8000 \
-v voicebox-data:/app/data \
ghcr.io/jamiepine/voicebox:latest-rocm
-v talkbox-data:/app/data \
ghcr.io/jamiepine/talkbox:latest-rocm
```
Then open: `http://localhost:8000`
@@ -42,12 +42,12 @@ Create `docker-compose.yml`:
version: '3.8'
services:
voicebox:
image: ghcr.io/jamiepine/voicebox:latest-cuda
talkbox:
image: ghcr.io/jamiepine/talkbox:latest-cuda
ports:
- "8000:8000"
volumes:
- voicebox-data:/app/data
- talkbox-data:/app/data
- huggingface-cache:/root/.cache/huggingface
environment:
- GPU_MEMORY_FRACTION=0.8 # Use 80% of GPU memory
@@ -62,7 +62,7 @@ services:
capabilities: [gpu]
volumes:
voicebox-data:
talkbox-data:
huggingface-cache:
```
@@ -108,8 +108,8 @@ CMD ["uvicorn", "backend.main:app", "--host", "0.0.0.0", "--port", "8000"]
Build and run:
```bash
docker build -t voicebox .
docker run -p 8000:8000 -v $(pwd)/data:/app/data voicebox
docker build -t talkbox .
docker run -p 8000:8000 -v $(pwd)/data:/app/data talkbox
```
### Multi-Stage Build (Optimized)
@@ -160,7 +160,7 @@ CMD ["uvicorn", "backend.main:app", "--host", "0.0.0.0", "--port", "8000"]
Build:
```bash
docker build -f Dockerfile.optimized -t voicebox:slim .
docker build -f Dockerfile.optimized -t talkbox:slim .
```
## GPU Support
@@ -195,15 +195,15 @@ CMD ["uvicorn", "backend.main:app", "--host", "0.0.0.0", "--port", "8000"]
**Run with GPU:**
```bash
docker run --gpus all -p 8000:8000 \
-v voicebox-data:/app/data \
voicebox:cuda
-v talkbox-data:/app/data \
talkbox:cuda
```
**Docker Compose with GPU:**
```yaml
services:
voicebox:
image: voicebox:cuda
talkbox:
image: talkbox:cuda
deploy:
resources:
reservations:
@@ -249,8 +249,8 @@ CMD ["uvicorn", "backend.main:app", "--host", "0.0.0.0", "--port", "8000"]
docker run --device=/dev/kfd --device=/dev/dri \
--group-add video --ipc=host --cap-add=SYS_PTRACE \
--security-opt seccomp=unconfined \
-p 8000:8000 -v voicebox-data:/app/data \
voicebox:rocm
-p 8000:8000 -v talkbox-data:/app/data \
talkbox:rocm
```
**Note:** ROCm support varies by GPU model. Works best on Linux. See [AMD ROCm docs](https://rocm.docs.amd.com) for compatibility.
@@ -260,9 +260,9 @@ docker run --device=/dev/kfd --device=/dev/dri \
### Essential Volumes
```bash
docker run -v voicebox-data:/app/data \ # Profiles, generations, history
docker run -v talkbox-data:/app/data \ # Profiles, generations, history
-v huggingface-cache:/root/.cache/huggingface \ # Downloaded models
-p 8000:8000 voicebox
-p 8000:8000 talkbox
```
### Development Volume Mounts
@@ -271,9 +271,9 @@ For development with hot-reload:
```bash
docker run -v $(pwd)/backend:/app/backend \ # Live code changes
-v voicebox-data:/app/data \
-v talkbox-data:/app/data \
-e RELOAD=true \
-p 8000:8000 voicebox
-p 8000:8000 talkbox
```
### Custom Model Storage
@@ -283,13 +283,13 @@ Use external model directory:
```bash
docker run -v /path/to/models:/models \
-e MODELS_DIR=/models \
-v voicebox-data:/app/data \
-p 8000:8000 voicebox
-v talkbox-data:/app/data \
-p 8000:8000 talkbox
```
## Environment Variables
Configure Voicebox via environment variables:
Configure TalkBox via environment variables:
```bash
docker run -e TTS_MODE=local \
@@ -297,7 +297,7 @@ docker run -e TTS_MODE=local \
-e OPENAI_API_KEY=sk-... \
-e GPU_MEMORY_FRACTION=0.8 \
-e LOG_LEVEL=info \
-p 8000:8000 voicebox
-p 8000:8000 talkbox
```
### Available Variables
@@ -324,14 +324,14 @@ docker run -e TTS_MODE=local \
version: '3.8'
services:
voicebox:
image: ghcr.io/jamiepine/voicebox:latest-cuda
container_name: voicebox
talkbox:
image: ghcr.io/jamiepine/talkbox:latest-cuda
container_name: talkbox
restart: unless-stopped
ports:
- "8000:8000"
volumes:
- voicebox-data:/app/data
- talkbox-data:/app/data
- huggingface-cache:/root/.cache/huggingface
environment:
- TTS_MODE=local
@@ -353,7 +353,7 @@ services:
start_period: 40s
volumes:
voicebox-data:
talkbox-data:
driver: local
huggingface-cache:
driver: local
@@ -371,7 +371,7 @@ docker compose -f docker-compose.prod.yml up -d
version: '3.8'
services:
voicebox:
talkbox:
build:
context: .
dockerfile: Dockerfile
@@ -379,7 +379,7 @@ services:
- "8000:8000"
volumes:
- ./backend:/app/backend:ro
- voicebox-data:/app/data
- talkbox-data:/app/data
- huggingface-cache:/root/.cache/huggingface
environment:
- RELOAD=true
@@ -388,7 +388,7 @@ services:
command: uvicorn backend.main:app --host 0.0.0.0 --port 8000 --reload
volumes:
voicebox-data:
talkbox-data:
huggingface-cache:
```
@@ -401,12 +401,12 @@ Full stack with reverse proxy and monitoring:
version: '3.8'
services:
# Main Voicebox app
voicebox:
image: ghcr.io/jamiepine/voicebox:latest-cuda
# Main TalkBox app
talkbox:
image: ghcr.io/jamiepine/talkbox:latest-cuda
restart: unless-stopped
volumes:
- voicebox-data:/app/data
- talkbox-data:/app/data
- huggingface-cache:/root/.cache/huggingface
environment:
- TTS_MODE=local
@@ -429,7 +429,7 @@ services:
- ./nginx.conf:/etc/nginx/nginx.conf:ro
- ./ssl:/etc/nginx/ssl:ro
depends_on:
- voicebox
- talkbox
# Prometheus monitoring (optional)
prometheus:
@@ -441,7 +441,7 @@ services:
- prometheus-data:/prometheus
volumes:
voicebox-data:
talkbox-data:
huggingface-cache:
prometheus-data:
```
@@ -466,9 +466,9 @@ volumes:
3. **Deploy:**
```bash
docker run --gpus all -d -p 80:8000 \
-v voicebox-data:/app/data \
-v talkbox-data:/app/data \
--restart unless-stopped \
ghcr.io/jamiepine/voicebox:latest-cuda
ghcr.io/jamiepine/talkbox:latest-cuda
```
### DigitalOcean
@@ -477,7 +477,7 @@ Use GPU Droplet + Docker:
```bash
# Create droplet via CLI
doctl compute droplet create voicebox \
doctl compute droplet create talkbox \
--size gpu-h100x1-80gb \
--image ubuntu-22-04-x64 \
--region nyc3
@@ -486,19 +486,19 @@ doctl compute droplet create voicebox \
ssh root@<droplet-ip>
curl -fsSL https://get.docker.com -o get-docker.sh
sh get-docker.sh
docker run --gpus all -d -p 80:8000 voicebox:cuda
docker run --gpus all -d -p 80:8000 talkbox:cuda
```
### Google Cloud Run (CPU-only)
```bash
# Build and push
docker build -t gcr.io/your-project/voicebox .
docker push gcr.io/your-project/voicebox
docker build -t gcr.io/your-project/talkbox .
docker push gcr.io/your-project/talkbox
# Deploy to Cloud Run
gcloud run deploy voicebox \
--image gcr.io/your-project/voicebox \
gcloud run deploy talkbox \
--image gcr.io/your-project/talkbox \
--platform managed \
--region us-central1 \
--memory 4Gi \
@@ -510,10 +510,10 @@ gcloud run deploy voicebox \
Create `fly.toml`:
```toml
app = "voicebox"
app = "talkbox"
[build]
image = "ghcr.io/jamiepine/voicebox:latest"
image = "ghcr.io/jamiepine/talkbox:latest"
[[services]]
http_checks = []
@@ -529,7 +529,7 @@ app = "voicebox"
handlers = ["tls", "http"]
[mounts]
source = "voicebox_data"
source = "talkbox_data"
destination = "/app/data"
```
@@ -560,38 +560,38 @@ docker run --rm --device=/dev/kfd --device=/dev/dri rocm/dev-ubuntu-22.04:6.0 ro
Container can't write to volumes:
```bash
# Fix permissions
docker run --user $(id -u):$(id -g) -v $(pwd)/data:/app/data voicebox
docker run --user $(id -u):$(id -g) -v $(pwd)/data:/app/data talkbox
```
### Out of Memory
Reduce GPU memory usage:
```bash
docker run -e GPU_MEMORY_FRACTION=0.5 voicebox
docker run -e GPU_MEMORY_FRACTION=0.5 talkbox
```
Or use CPU-only:
```bash
docker run -e DEVICE=cpu voicebox
docker run -e DEVICE=cpu talkbox
```
### Model Download Fails
Ensure HuggingFace cache is writable:
```bash
docker run -v huggingface-cache:/root/.cache/huggingface voicebox
docker run -v huggingface-cache:/root/.cache/huggingface talkbox
```
Or use host cache:
```bash
docker run -v ~/.cache/huggingface:/root/.cache/huggingface voicebox
docker run -v ~/.cache/huggingface:/root/.cache/huggingface talkbox
```
### Port Already in Use
Change host port:
```bash
docker run -p 8080:8000 voicebox # Use port 8080 instead
docker run -p 8080:8000 talkbox # Use port 8080 instead
```
## Security Best Practices
@@ -600,8 +600,8 @@ docker run -p 8080:8000 voicebox # Use port 8080 instead
Create non-root user in Dockerfile:
```dockerfile
RUN useradd -m -u 1000 voicebox
USER voicebox
RUN useradd -m -u 1000 talkbox
USER talkbox
```
### 2. Use Secrets for API Keys
@@ -615,7 +615,7 @@ echo "sk-your-key" | docker secret create openai_key -
docker service create \
--secret openai_key \
-e OPENAI_API_KEY_FILE=/run/secrets/openai_key \
voicebox
talkbox
```
### 3. Network Isolation
@@ -624,7 +624,7 @@ Use internal networks for multi-container setups:
```yaml
services:
voicebox:
talkbox:
networks:
- internal
nginx:
@@ -646,7 +646,7 @@ Prevent resource exhaustion:
```yaml
services:
voicebox:
talkbox:
deploy:
resources:
limits:
@@ -663,10 +663,10 @@ services:
```bash
# Use 80% of GPU (default 90%)
docker run -e GPU_MEMORY_FRACTION=0.8 voicebox
docker run -e GPU_MEMORY_FRACTION=0.8 talkbox
# Allow GPU memory growth (prevents OOM)
docker run -e TF_FORCE_GPU_ALLOW_GROWTH=true voicebox
docker run -e TF_FORCE_GPU_ALLOW_GROWTH=true talkbox
```
### Model Caching
@@ -676,14 +676,14 @@ Pre-download models to volume:
```bash
# Download models first
docker run --rm -v huggingface-cache:/root/.cache/huggingface \
voicebox python -c "
talkbox python -c "
from transformers import WhisperProcessor, WhisperForConditionalGeneration
WhisperProcessor.from_pretrained('openai/whisper-base')
WhisperForConditionalGeneration.from_pretrained('openai/whisper-base')
"
# Then run normally
docker run -v huggingface-cache:/root/.cache/huggingface voicebox
docker run -v huggingface-cache:/root/.cache/huggingface talkbox
```
### Multi-Worker Setup
@@ -728,10 +728,10 @@ Then scrape `/metrics` with Prometheus.
View container logs:
```bash
docker logs -f voicebox
docker logs -f talkbox
# Or with compose
docker compose logs -f voicebox
docker compose logs -f talkbox
```
## Next Steps
@@ -748,7 +748,7 @@ Help improve Docker support:
1. Test on different platforms (AMD GPU, ARM64, etc.)
2. Submit Dockerfile optimizations
3. Share deployment configurations
4. Report issues: [GitHub Issues](https://github.com/jamiepine/voicebox/issues)
4. Report issues: [GitHub Issues](https://github.com/jamiepine/talkbox/issues)
## Resources
+16 -16
View File
@@ -6,13 +6,13 @@
## Context
Homebrew Cask PR [#260314](https://github.com/Homebrew/homebrew-cask/pull/260314) adds `brew install --cask voicebox`. CI is green on macOS 14 and macOS 26 (arm + intel) but fails on macOS 15 (arm + intel). The 0.4.3 release added DMG-level stapling to address this, and it didn't move CI — 0.4.5 still fails. A maintainer reproduced the failure in a fresh Sequoia VM.
Homebrew Cask PR [#260314](https://github.com/Homebrew/homebrew-cask/pull/260314) adds `brew install --cask talkbox`. CI is green on macOS 14 and macOS 26 (arm + intel) but fails on macOS 15 (arm + intel). The 0.4.3 release added DMG-level stapling to address this, and it didn't move CI — 0.4.5 still fails. A maintainer reproduced the failure in a fresh Sequoia VM.
This document is the working diagnosis plus the ordered fix plan.
## What the failing check actually does
The failing step is `brew audit --cask --online --signing --new voicebox`, not `brew install`. `brew install` succeeds end-to-end in CI (the log shows `Uninstalling Cask voicebox` after the install phase). The `--signing` audit:
The failing step is `brew audit --cask --online --signing --new talkbox`, not `brew install`. `brew install` succeeds end-to-end in CI (the log shows `Uninstalling Cask talkbox` after the install phase). The `--signing` audit:
1. Downloads the cask's `url`
2. Mounts the DMG
@@ -32,16 +32,16 @@ Local dev machines pass `spctl` because the first-party developer context and ca
## Where the gap is likely to be
Voicebox ships PyInstaller sidecars declared in `tauri.conf.json` under `externalBin`:
TalkBox ships PyInstaller sidecars declared in `tauri.conf.json` under `externalBin`:
- **0.4.x:** `voicebox-server` only (single `--onefile` Mach-O on macOS)
- **0.5.0+:** `voicebox-server` and `voicebox-mcp` (`voicebox-mcp` is new in 0.5.0)
- **0.4.x:** `talkbox-server` only (single `--onefile` Mach-O on macOS)
- **0.5.0+:** `talkbox-server` and `talkbox-mcp` (`talkbox-mcp` is new in 0.5.0)
Tauri's bundler signs each `externalBin` with the configured identity but does not apply `--options=runtime` or `--timestamp` automatically, and does not merge the outer app's entitlements into the sidecar signature. The outer `Voicebox` binary is correctly signed with hardened runtime + `disable-library-validation`; the sidecars likely are not.
Tauri's bundler signs each `externalBin` with the configured identity but does not apply `--options=runtime` or `--timestamp` automatically, and does not merge the outer app's entitlements into the sidecar signature. The outer `TalkBox` binary is correctly signed with hardened runtime + `disable-library-validation`; the sidecars likely are not.
Order of likelihood:
1. Sidecar `voicebox-server` lacks hardened runtime or a secure timestamp in its signature.
1. Sidecar `talkbox-server` lacks hardened runtime or a secure timestamp in its signature.
2. The sidecar inherits the identity but was signed before tauri-action's final notarization pass, so the notarization ticket doesn't actually cover it.
3. Something inside the sidecar's PyInstaller archive unpacks to a `.dylib` at runtime that Gatekeeper inspects during assessment.
@@ -52,11 +52,11 @@ The 0.5.0 fix must cover both sidecars.
Run against a freshly downloaded release DMG (not a dev build, and from a machine that has never opened the app before):
```
hdiutil attach Voicebox_0.4.5_aarch64.dmg
xcrun stapler validate "/Volumes/Voicebox 0.4.5/Voicebox.app"
spctl -a -vvv -t open --context context:primary-signature "/Volumes/Voicebox 0.4.5/Voicebox.app"
codesign --verify --deep --strict --verbose=2 "/Volumes/Voicebox 0.4.5/Voicebox.app"
codesign -dv --verbose=4 "/Volumes/Voicebox 0.4.5/Voicebox.app/Contents/MacOS/voicebox-server"
hdiutil attach TalkBox_0.4.5_aarch64.dmg
xcrun stapler validate "/Volumes/TalkBox 0.4.5/TalkBox.app"
spctl -a -vvv -t open --context context:primary-signature "/Volumes/TalkBox 0.4.5/TalkBox.app"
codesign --verify --deep --strict --verbose=2 "/Volumes/TalkBox 0.4.5/TalkBox.app"
codesign -dv --verbose=4 "/Volumes/TalkBox 0.4.5/TalkBox.app/Contents/MacOS/talkbox-server"
```
The last command is the tell — look for `flags=0x10000(runtime)` and a `Timestamp=` line. If either is missing, the sidecar is the failure.
@@ -71,7 +71,7 @@ Pull the 0.4.5 DMG on a fresh Sequoia environment or a VM snapshot with no trust
### Phase 2 — Sign sidecars explicitly in the release workflow
Between tauri-action's build step and the DMG-notarization step already in `release.yml`, add a step that re-signs every `externalBin` present under `Voicebox.app/Contents/MacOS/` with:
Between tauri-action's build step and the DMG-notarization step already in `release.yml`, add a step that re-signs every `externalBin` present under `TalkBox.app/Contents/MacOS/` with:
- `--options=runtime` (hardened runtime)
- `--timestamp` (secure timestamp)
@@ -80,11 +80,11 @@ Between tauri-action's build step and the DMG-notarization step already in `rele
Re-sign the outer `.app` afterward so its seal covers the updated nested signatures.
Covers `voicebox-server` on 0.4.x and both sidecars from 0.5.0 forward.
Covers `talkbox-server` on 0.4.x and both sidecars from 0.5.0 forward.
### Phase 3 — Re-notarize and staple the `.app`
After sidecars are re-signed the outer bundle's notarization ticket is stale. Submit the `.app` (zipped) to `notarytool`, wait, then `xcrun stapler staple Voicebox.app`. This puts the ticket directly on the `.app` so the `spctl -t open` audit passes without any online ticket lookup.
After sidecars are re-signed the outer bundle's notarization ticket is stale. Submit the `.app` (zipped) to `notarytool`, wait, then `xcrun stapler staple TalkBox.app`. This puts the ticket directly on the `.app` so the `spctl -t open` audit passes without any online ticket lookup.
Then rebuild the DMG from the stapled `.app` and keep the existing DMG-level notarize/staple step — it still helps Finder drag-install.
@@ -94,7 +94,7 @@ Before upload, run the same four diagnostic commands against the built artifact
### Phase 5 — Re-request Homebrew CI
Once a tagged release passes Phase 4 locally, push a cask update to #260314. Expect `test voicebox (macos-15, arm)` and `test voicebox (macos-15-intel, intel)` to go green.
Once a tagged release passes Phase 4 locally, push a cask update to #260314. Expect `test talkbox (macos-15, arm)` and `test talkbox (macos-15-intel, intel)` to go green.
## Open questions
+57 -57
View File
@@ -1,4 +1,4 @@
# MCP Server — Voicebox Speed Run
# MCP Server — TalkBox Speed Run
**Status:** v1 shipped — HTTP transport, all 4 tools, per-client bindings, `POST /speak`, stdio shim (binary built, bundled into Tauri sidecar), Settings UI, speak-pill via SSE with Rust-side `dictate:show` handler so agent-initiated speech surfaces the pill on screen. `cargo check` clean, `tsc` clean, full Inspector round-trip verified.
**Last reviewed:** 2026-04-23
@@ -8,21 +8,21 @@
### Shipped (backend)
- **`fastmcp` + `sse-starlette`** pinned in `backend/requirements.txt`.
- **`backend/mcp_server/`** package with `server.py`, `tools.py`, `context.py`, `resolve.py`, `events.py`, `README.md`. Named `mcp_server` (not `mcp`) to sidestep a shadowing conflict with the installed `mcp` PyPI package that FastMCP imports internally.
- **Streamable HTTP mount at `/mcp`** via FastMCP's `http_app(transport='http')`. Sub-app lifespan composed with Voicebox's own startup/shutdown through an `@asynccontextmanager lifespan=` in `backend/app.py` (migrated away from the deprecated `@app.on_event` handlers).
- **Streamable HTTP mount at `/mcp`** via FastMCP's `http_app(transport='http')`. Sub-app lifespan composed with TalkBox's own startup/shutdown through an `@asynccontextmanager lifespan=` in `backend/app.py` (migrated away from the deprecated `@app.on_event` handlers).
- **Four MCP tools**, dot-named to match the landing and ecosystem convention:
- `voicebox.speak(text, profile?, engine?, personality?, language?)`
- `voicebox.transcribe(audio_base64?, audio_path?, language?, model?)`
- `voicebox.list_captures(limit, offset)`
- `voicebox.list_profiles()`
- **`ClientIdMiddleware`** pulls `X-Voicebox-Client-Id` into a `ContextVar` on every `/mcp*` request; auto-stamps `MCPClientBinding.last_seen_at`, auto-creating the row if the client is new.
- `talkbox.speak(text, profile?, engine?, personality?, language?)`
- `talkbox.transcribe(audio_base64?, audio_path?, language?, model?)`
- `talkbox.list_captures(limit, offset)`
- `talkbox.list_profiles()`
- **`ClientIdMiddleware`** pulls `X-TalkBox-Client-Id` into a `ContextVar` on every `/mcp*` request; auto-stamps `MCPClientBinding.last_seen_at`, auto-creating the row if the client is new.
- **Profile resolution precedence** `explicit → per-client binding → capture_settings.default_playback_voice_id → error`. `services/profiles.get_profile_orm_by_name_or_id()` lets agents pass a voice by name ("Morgan") instead of UUID.
- **`MCPClientBinding` table** (new) via `Base.metadata.create_all` — no migration needed.
- **Bindings REST:** `GET|PUT /mcp/bindings`, `DELETE /mcp/bindings/{client_id}`.
- **`POST /speak`** REST wrapper for non-MCP callers (shell / ACP / A2A). Same `resolve_profile` precedence, same code path as the MCP tool.
- **Stdio shim** at `backend/mcp_shim/__main__.py` — ~200 lines of `httpx` proxy; reads env (`VOICEBOX_PORT`, `VOICEBOX_HOST`, `VOICEBOX_CLIENT_ID`), waits for `/health`, then streams JSON-RPC ↔ SSE. Rolled our own after the `mcp` SDK's session-management helpers mis-shook-hands. Smoke-tested: `initialize`, `tools/list`, and `tools/call` all round-trip cleanly.
- **Stdio shim** at `backend/mcp_shim/__main__.py` — ~200 lines of `httpx` proxy; reads env (`TALKBOX_PORT`, `TALKBOX_HOST`, `TALKBOX_CLIENT_ID`), waits for `/health`, then streams JSON-RPC ↔ SSE. Rolled our own after the `mcp` SDK's session-management helpers mis-shook-hands. Smoke-tested: `initialize`, `tools/list`, and `tools/call` all round-trip cleanly.
- **Pill SSE:** `GET /events/speak` (`sse-starlette`) emits `speak-start` from the MCP tool and `POST /speak`, `speak-end` from `services/generation.run_generation`'s finally block.
- **PyInstaller:**
- `backend/build_binary.py` `--shim` flag builds a minimal `voicebox-mcp` binary (torch/transformers/mlx/etc. explicitly excluded, target <20 MB).
- `backend/build_binary.py` `--shim` flag builds a minimal `talkbox-mcp` binary (torch/transformers/mlx/etc. explicitly excluded, target <20 MB).
- The main server spec picks up `fastmcp`, `mcp`, `sse_starlette`, and `backend.mcp_server.*` via `--collect-all` / `--hidden-import`.
- **`backend/mcp_server/README.md`** quickstart (Inspector, `.mcp.json` snippets, tool reference).
@@ -39,12 +39,12 @@
- Router + `ServerTab` tab bar wired to `/settings/mcp`.
### Shipped (native shell)
- **`tauri.conf.json`** — `voicebox-mcp` added to `externalBin` (alongside `voicebox-server`).
- **`tauri.conf.json`** — `talkbox-mcp` added to `externalBin` (alongside `talkbox-server`).
- **`dictate:show` listener** in `tauri/src-tauri/src/main.rs` — invokes a new `show_dictate_window(app_handle)` helper that mirrors the hotkey-monitor's position+show logic (undo click-through, reposition to top-center of the current monitor, show). Agent-initiated speech now pops the pill visible on screen.
### Validated end-to-end (this session, via curl)
- `/mcp/` init → `tools/list``tools/call voicebox.speak` → actual audio plays (Jarvis, 1.68 s).
- `POST /speak` with `X-Voicebox-Client-Id: claude-code` resolves to the bound Jarvis profile without passing `profile`.
- `/mcp/` init → `tools/list``tools/call talkbox.speak` → actual audio plays (Jarvis, 1.68 s).
- `POST /speak` with `X-TalkBox-Client-Id: claude-code` resolves to the bound Jarvis profile without passing `profile`.
- `/events/speak` emits `ready`, `speak-start`, `speak-end` in order, generation_id threads through both.
- Stdio shim: `echo {…} | python -m backend.mcp_shim` returns valid JSON-RPC for all 4 methods.
- `last_seen_at` auto-stamps on first call; binding row auto-creates.
@@ -52,8 +52,8 @@
- `cargo check` on the Tauri crate: clean.
### Outstanding (must-do before release)
- **CI build for shim on Windows/Linux** — `python backend/build_binary.py --shim` is wired up and built cleanly for `aarch64-apple-darwin` (18 MB, installed at `tauri/src-tauri/binaries/voicebox-mcp-aarch64-apple-darwin`, Tauri `cargo check` green). The Windows and Linux triples (`x86_64-pc-windows-msvc`, `x86_64-unknown-linux-gnu`) need the same build in their respective CI runners and artifacts dropped alongside the macOS binary.
- **Windows/Linux paths in the stdio snippet** — the Settings page hardcodes the macOS path (`/Applications/Voicebox.app/Contents/MacOS/voicebox-mcp`). Needs a per-OS switch (`%LOCALAPPDATA%\Programs\Voicebox\voicebox-mcp.exe`, Linux bundled-path), ideally with the Tauri shell resolving its own app path at runtime and injecting it into the snippet.
- **CI build for shim on Windows/Linux** — `python backend/build_binary.py --shim` is wired up and built cleanly for `aarch64-apple-darwin` (18 MB, installed at `tauri/src-tauri/binaries/talkbox-mcp-aarch64-apple-darwin`, Tauri `cargo check` green). The Windows and Linux triples (`x86_64-pc-windows-msvc`, `x86_64-unknown-linux-gnu`) need the same build in their respective CI runners and artifacts dropped alongside the macOS binary.
- **Windows/Linux paths in the stdio snippet** — the Settings page hardcodes the macOS path (`/Applications/TalkBox.app/Contents/MacOS/talkbox-mcp`). Needs a per-OS switch (`%LOCALAPPDATA%\Programs\TalkBox\talkbox-mcp.exe`, Linux bundled-path), ideally with the Tauri shell resolving its own app path at runtime and injecting it into the snippet.
### Nice-to-have (follow-up passes)
- **One-click install buttons** — write/merge into `~/.claude/settings.json`, `~/.cursor/mcp.json`, etc. via a Tauri command. Copy-paste works today; this is pure ergonomics.
@@ -63,9 +63,9 @@
## Context
Voicebox already ships the I/O surface (Captures, Generate, personality-driven `/profiles/{id}/speak`), but local AI agents can't reach any of it. This plan adds a Model Context Protocol server so Claude Code / Cursor / Cline can call `voicebox.speak`, `voicebox.transcribe`, `voicebox.list_captures`, and `voicebox.list_profiles` — turning Voicebox into the local voice layer for every agent on the user's machine (Phase 5 of `docs/plans/VOICE_IO.md`).
TalkBox already ships the I/O surface (Captures, Generate, personality-driven `/profiles/{id}/speak`), but local AI agents can't reach any of it. This plan adds a Model Context Protocol server so Claude Code / Cursor / Cline can call `talkbox.speak`, `talkbox.transcribe`, `talkbox.list_captures`, and `talkbox.list_profiles` — turning TalkBox into the local voice layer for every agent on the user's machine (Phase 5 of `docs/plans/VOICE_IO.md`).
The shortest path to "Claude Code speaks in a cloned voice": mount **FastMCP** inside the existing FastAPI/uvicorn process at `/mcp` (Streamable HTTP), and users install it as a URL (`{"url": "http://127.0.0.1:17493/mcp"}`) — the ecosystem-idiomatic shape for a long-running local service. Per-client voice binding via a new `mcp_client_bindings` table + Settings UI, resolved from an `X-Voicebox-Client-Id` header. A **stdio shim binary** `voicebox-mcp` is bundled as a fallback sidecar for clients that can't speak HTTP MCP. A public `POST /speak` REST wrapper covers non-MCP callers (shell scripts, ACP, A2A). A `speaking` pill state gives agent-initiated audio visibility — trust-critical, non-negotiable.
The shortest path to "Claude Code speaks in a cloned voice": mount **FastMCP** inside the existing FastAPI/uvicorn process at `/mcp` (Streamable HTTP), and users install it as a URL (`{"url": "http://127.0.0.1:17494/mcp"}`) — the ecosystem-idiomatic shape for a long-running local service. Per-client voice binding via a new `mcp_client_bindings` table + Settings UI, resolved from an `X-TalkBox-Client-Id` header. A **stdio shim binary** `talkbox-mcp` is bundled as a fallback sidecar for clients that can't speak HTTP MCP. A public `POST /speak` REST wrapper covers non-MCP callers (shell scripts, ACP, A2A). A `speaking` pill state gives agent-initiated audio visibility — trust-critical, non-negotiable.
## Architecture
@@ -75,22 +75,22 @@ Claude Code / Cursor / Windsurf / VS Code MCP
├─ HTTP (primary) ────────────────────┐
│ {"url": ".../mcp"} │
│ │
└─ stdio (fallback) ───────────────▶ [voicebox-mcp shim binary]
{"command": "/abs/path/voicebox-mcp"} (absolute path;
└─ stdio (fallback) ───────────────▶ [talkbox-mcp shim binary]
{"command": "/abs/path/talkbox-mcp"} (absolute path;
│ Settings page
│ copies it for you)
uvicorn + FastAPI (port 17493)
uvicorn + FastAPI (port 17494)
├─ /mcp (FastMCP, Streamable HTTP)
└─ /speak (REST wrapper for non-MCP callers)
└─ tools call existing services
```
- **Transport:** Streamable HTTP as primary (Nov-2025 spec, post-SSE). Claude Code, Cursor, Windsurf, and the VS Code MCP extensions all support HTTP — it's the idiomatic shape for a long-running local service, which Voicebox already is.
- **Stdio fallback:** `voicebox-mcp` binary bundled inside the app for clients that can't speak HTTP MCP. The Settings page renders the exact snippet with the detected absolute path — user copies, pastes, done. No PATH manipulation, no custom CLI wrapper.
- **Identity:** HTTP clients set `X-Voicebox-Client-Id` header in their MCP config's `headers` block. Stdio clients set `VOICEBOX_CLIENT_ID` env var, which the shim forwards as the same HTTP header. Server reads it into a `ContextVar`.
- **Transport:** Streamable HTTP as primary (Nov-2025 spec, post-SSE). Claude Code, Cursor, Windsurf, and the VS Code MCP extensions all support HTTP — it's the idiomatic shape for a long-running local service, which TalkBox already is.
- **Stdio fallback:** `talkbox-mcp` binary bundled inside the app for clients that can't speak HTTP MCP. The Settings page renders the exact snippet with the detected absolute path — user copies, pastes, done. No PATH manipulation, no custom CLI wrapper.
- **Identity:** HTTP clients set `X-TalkBox-Client-Id` header in their MCP config's `headers` block. Stdio clients set `TALKBOX_CLIENT_ID` env var, which the shim forwards as the same HTTP header. Server reads it into a `ContextVar`.
- **Profile resolution precedence:** explicit tool arg → per-client `MCPClientBinding.profile_id``capture_settings.default_playback_voice_id` → error.
- **Port:** `17493`, matching `tauri/src-tauri/src/main.rs:63` (`SERVER_PORT` constant). Shim default with `VOICEBOX_PORT` env override.
- **Port:** `17494`, matching `tauri/src-tauri/src/main.rs:63` (`SERVER_PORT` constant). Shim default with `TALKBOX_PORT` env override.
- **Non-MCP access:** `POST /speak` is a thin REST wrapper around the same tool path — one endpoint for shell scripts, ACP, A2A, and anything that isn't MCP-native.
## Library choice
@@ -132,7 +132,7 @@ Global default stays in `capture_settings.default_playback_voice_id` — no dupl
| `backend/mcp/events.py` | In-memory `asyncio.Queue` pub/sub for speak-start / speak-end |
| `backend/mcp/README.md` | MCP Inspector quickstart + `.mcp.json` snippets |
| `backend/mcp_shim/__init__.py`, `__main__.py` | Stdio ↔ Streamable HTTP proxy (~150 lines) |
| `backend/voicebox-mcp.spec` | PyInstaller spec for the shim (strips torch/transformers from `hiddenimports`) |
| `backend/talkbox-mcp.spec` | PyInstaller spec for the shim (strips torch/transformers from `hiddenimports`) |
| `backend/routes/speak.py` | `POST /speak {text, profile?, engine?, personality?, language?}` — REST wrapper around `resolve_profile()` + `generate_speech()` for non-MCP agents |
### Backend — modified
@@ -150,8 +150,8 @@ Global default stays in `capture_settings.default_playback_voice_id` — no dupl
| `backend/routes/mcp_bindings.py` (new) | REST CRUD for bindings (list, upsert, delete). |
| `backend/routes/events.py` (new) | `GET /events/speak``EventSourceResponse` subscribed to the events queue. |
| `backend/requirements.txt` | `+ fastmcp` (or `mcp>=1.0`), `+ sse-starlette` |
| `backend/voicebox-server.spec` | `hiddenimports += ['mcp', 'mcp.server', 'fastmcp']` |
| `backend/build_binary.py` | Second PyInstaller invocation for `voicebox-mcp.spec`; copy to `tauri/src-tauri/binaries/` with target-triple suffix |
| `backend/talkbox-server.spec` | `hiddenimports += ['mcp', 'mcp.server', 'fastmcp']` |
| `backend/build_binary.py` | Second PyInstaller invocation for `talkbox-mcp.spec`; copy to `tauri/src-tauri/binaries/` with target-triple suffix |
### Frontend — new
@@ -175,19 +175,19 @@ Global default stays in `capture_settings.default_playback_voice_id` — no dupl
| File | Change |
|---|---|
| `tauri/src-tauri/tauri.conf.json` | `"externalBin": ["binaries/voicebox-server", "binaries/voicebox-mcp"]` |
| `tauri/src-tauri/binaries/voicebox-mcp-<triple>` | Build artifact from PyInstaller |
| `tauri/src-tauri/tauri.conf.json` | `"externalBin": ["binaries/talkbox-server", "binaries/talkbox-mcp"]` |
| `tauri/src-tauri/binaries/talkbox-mcp-<triple>` | Build artifact from PyInstaller |
## Tool signatures
All tools read `current_client_id.get()` (from middleware). Return JSON-serializable dicts.
Tools are registered with **dotted names** (`voicebox.speak`, etc.) to match the landing page and the industry convention (`filesystem.read_file`, `github.create_issue`). Python function names stay snake_case; the dot goes in the `name=` kwarg.
Tools are registered with **dotted names** (`talkbox.speak`, etc.) to match the landing page and the industry convention (`filesystem.read_file`, `github.create_issue`). Python function names stay snake_case; the dot goes in the `name=` kwarg.
```python
# backend/mcp/tools.py
@mcp.tool(name="voicebox.speak")
@mcp.tool(name="talkbox.speak")
async def speak(text: str,
profile: str | None = None, # name OR id
engine: str | None = None,
@@ -198,7 +198,7 @@ async def speak(text: str,
# route honors `personality=True` by running rewrite_as_profile on
# the input before running the normal TTS pipeline.
@mcp.tool(name="voicebox.transcribe")
@mcp.tool(name="talkbox.transcribe")
async def transcribe(audio_base64: str | None = None,
audio_path: str | None = None, # absolute local path
language: str | None = None,
@@ -206,11 +206,11 @@ async def transcribe(audio_base64: str | None = None,
"""Transcribe audio. Exactly one of audio_base64/audio_path. Returns {text, duration, language}."""
# validate path readable, size < 200 MB, then call services.transcribe.transcribe_bytes
@mcp.tool(name="voicebox.list_captures")
@mcp.tool(name="talkbox.list_captures")
async def list_captures(limit: int = 20, offset: int = 0) -> dict:
"""Recent captures with transcripts. Returns {captures: [...]}"""
@mcp.tool(name="voicebox.list_profiles")
@mcp.tool(name="talkbox.list_profiles")
async def list_profiles() -> dict:
"""Available voice profiles. Returns {profiles: [{id, name, voice_type, has_personality}]}"""
```
@@ -222,7 +222,7 @@ async def list_profiles() -> dict:
@router.post("/speak", response_model=GenerationResponse)
async def speak(data: SpeakRequest, request: Request, db: Session = Depends(get_db)):
"""Same behavior as the MCP tool — for shell scripts, ACP, A2A, or anything non-MCP."""
client_id = request.headers.get("X-Voicebox-Client-Id")
client_id = request.headers.get("X-TalkBox-Client-Id")
profile = resolve_profile(data.profile, client_id, db)
if profile is None: raise HTTPException(400, "No voice profile resolved.")
req = GenerationRequest(profile_id=profile.id, text=data.text,
@@ -244,14 +244,14 @@ mount_into(application)
`mount_into` installs `ClientIdMiddleware` and calls `app.mount("/mcp", mcp.streamable_http_app())`.
**Lifespan migration is load-bearing** — FastMCP's session manager requires the `lifespan=` kwarg, not `@app.on_event`. Wrap the existing startup/shutdown bodies in an `@asynccontextmanager` using `contextlib.AsyncExitStack` so both Voicebox's init and FastMCP's session manager run. Verify dev + packaged build after the migration.
**Lifespan migration is load-bearing** — FastMCP's session manager requires the `lifespan=` kwarg, not `@app.on_event`. Wrap the existing startup/shutdown bodies in an `@asynccontextmanager` using `contextlib.AsyncExitStack` so both TalkBox's init and FastMCP's session manager run. Verify dev + packaged build after the migration.
## Stdio shim (`backend/mcp_shim/__main__.py`)
1. Port: `int(os.environ.get("VOICEBOX_PORT", "17493"))`.
2. Client id: `os.environ.get("VOICEBOX_CLIENT_ID", "unknown")`.
1. Port: `int(os.environ.get("TALKBOX_PORT", "17494"))`.
2. Client id: `os.environ.get("TALKBOX_CLIENT_ID", "unknown")`.
3. Health probe `GET /health` with 30 s tolerance (torch imports slowly). On failure, emit JSON-RPC error on stdout, exit 1.
4. Connect Streamable HTTP MCP client to `http://127.0.0.1:{port}/mcp` with `X-Voicebox-Client-Id: {client_id}` header.
4. Connect Streamable HTTP MCP client to `http://127.0.0.1:{port}/mcp` with `X-TalkBox-Client-Id: {client_id}` header.
5. Proxy JSON-RPC bidirectionally — stdin → HTTP, SSE → stdout. Use `mcp` SDK's built-in stdio↔HTTP bridge if available; otherwise ~40 lines of asyncio.
6. Stdout = JSON-RPC only. All logs to stderr.
@@ -273,23 +273,23 @@ PyInstaller spec keeps only `mcp`, `httpx`, `anyio`, `click` — target binary <
HTTP form (primary):
```json
{"mcpServers": {"voicebox": {
"url": "http://127.0.0.1:17493/mcp",
"headers": {"X-Voicebox-Client-Id": "claude-code"}
{"mcpServers": {"talkbox": {
"url": "http://127.0.0.1:17494/mcp",
"headers": {"X-TalkBox-Client-Id": "claude-code"}
}}}
```
Stdio form (fallback, absolute path auto-filled from detected app location):
```json
{"mcpServers": {"voicebox": {
"command": "/Applications/Voicebox.app/Contents/MacOS/voicebox-mcp",
"env": {"VOICEBOX_CLIENT_ID": "claude-code"}
{"mcpServers": {"talkbox": {
"command": "/Applications/TalkBox.app/Contents/MacOS/talkbox-mcp",
"env": {"TALKBOX_CLIENT_ID": "claude-code"}
}}}
```
Plus the Claude-Code-specific one-liner:
```
claude mcp add voicebox --transport http --url http://127.0.0.1:17493/mcp --header "X-Voicebox-Client-Id: claude-code"
claude mcp add talkbox --transport http --url http://127.0.0.1:17494/mcp --header "X-TalkBox-Client-Id: claude-code"
```
- **One-click install buttons** for known clients (v1: Claude Code via `claude mcp add` invocation, and a config-file writer for Cursor/Windsurf whose config locations are known). Each has a matching "Remove" button. Hide buttons for clients not detected on disk.
- **Connection status** — small indicator next to each binding showing the last time that `client_id` actually called the server (rolling timestamp recorded by middleware), so users can tell their install worked.
@@ -297,26 +297,26 @@ PyInstaller spec keeps only `mcp`, `httpx`, `anyio`, `click` — target binary <
## Ordered task list (shortest path first)
1. `fastmcp` + `sse-starlette` → `backend/requirements.txt`; install.
2. Add `backend/mcp/{server,tools,context,resolve}.py` with the 4 tools registered as `voicebox.speak` etc. (no middleware yet — global default profile only).
2. Add `backend/mcp/{server,tools,context,resolve}.py` with the 4 tools registered as `talkbox.speak` etc. (no middleware yet — global default profile only).
3. Migrate `app.py` to `lifespan=`; mount FastMCP at `/mcp`.
4. **Milestone:** `npx @modelcontextprotocol/inspector http://127.0.0.1:17493/mcp` — call `voicebox.speak`, hear audio.
4. **Milestone:** `npx @modelcontextprotocol/inspector http://127.0.0.1:17494/mcp` — call `talkbox.speak`, hear audio.
5. Add `get_profile_by_name_or_id`; wire the tool's `profile` arg.
6. `MCPClientBinding` model + migration; middleware; full `resolve_profile` precedence.
7. `backend/routes/speak.py` — `POST /speak` REST wrapper, reusing `resolve_profile` + `speak_in_character`.
8. `/mcp/bindings` REST + `MCPBindings.tsx` UI with HTTP and stdio copy-snippets, one-click install for detected clients, and connection-status indicators. **Users can install Voicebox as an MCP server after this step.**
9. `backend/mcp_shim/__main__.py` + PyInstaller spec + `build_binary.py` second pass; register `voicebox-mcp` as a Tauri sidecar. (Fallback path goes live.)
8. `/mcp/bindings` REST + `MCPBindings.tsx` UI with HTTP and stdio copy-snippets, one-click install for detected clients, and connection-status indicators. **Users can install TalkBox as an MCP server after this step.**
9. `backend/mcp_shim/__main__.py` + PyInstaller spec + `build_binary.py` second pass; register `talkbox-mcp` as a Tauri sidecar. (Fallback path goes live.)
10. Events queue + `/events/speak` SSE + `DictateWindow` `speaking` pill state.
11. `backend/mcp/README.md` quickstart.
Claude Code can call `voicebox.speak` after step 4 (direct HTTP, manual config). Step 8 makes that a one-click experience. Step 9 adds the stdio fallback for clients that don't speak HTTP MCP.
Claude Code can call `talkbox.speak` after step 4 (direct HTTP, manual config). Step 8 makes that a one-click experience. Step 9 adds the stdio fallback for clients that don't speak HTTP MCP.
## Verification
- **Step 4 smoke:** `npx @modelcontextprotocol/inspector http://127.0.0.1:17493/mcp`. Call `voicebox.list_profiles`, then `voicebox.speak(text="hello from mcp")`. Audio plays; generation appears in History with `source="personality_speak"` (or new `source="mcp"` if we add one).
- **REST wrapper:** `curl -X POST http://127.0.0.1:17493/speak -d '{"text":"hi","profile":"Morgan"}'` — same behavior, same pill surface.
- **Per-client:** open two Inspector sessions with different `X-Voicebox-Client-Id` headers, bind each to a different profile in Settings, verify distinct voices without `profile` arg.
- **Claude Code end-to-end (HTTP):** `claude mcp add voicebox --transport http --url http://127.0.0.1:17493/mcp --header "X-Voicebox-Client-Id: claude-code"`, then ask Claude Code to speak. Pill shows `speaking: <profile>`, audio plays, capture appears in history.
- **Stdio fallback:** manually paste the stdio snippet from Settings into a client's config, verify same behavior. `VOICEBOX_CLIENT_ID=claude-code python -m backend.mcp_shim` while backend is up; pipe a tools/list JSON-RPC in, verify response over stdout.
- **Step 4 smoke:** `npx @modelcontextprotocol/inspector http://127.0.0.1:17494/mcp`. Call `talkbox.list_profiles`, then `talkbox.speak(text="hello from mcp")`. Audio plays; generation appears in History with `source="personality_speak"` (or new `source="mcp"` if we add one).
- **REST wrapper:** `curl -X POST http://127.0.0.1:17494/speak -d '{"text":"hi","profile":"Morgan"}'` — same behavior, same pill surface.
- **Per-client:** open two Inspector sessions with different `X-TalkBox-Client-Id` headers, bind each to a different profile in Settings, verify distinct voices without `profile` arg.
- **Claude Code end-to-end (HTTP):** `claude mcp add talkbox --transport http --url http://127.0.0.1:17494/mcp --header "X-TalkBox-Client-Id: claude-code"`, then ask Claude Code to speak. Pill shows `speaking: <profile>`, audio plays, capture appears in history.
- **Stdio fallback:** manually paste the stdio snippet from Settings into a client's config, verify same behavior. `TALKBOX_CLIENT_ID=claude-code python -m backend.mcp_shim` while backend is up; pipe a tools/list JSON-RPC in, verify response over stdout.
- **Transcribe:** point at `/tmp/test.wav`; diff against `POST /transcribe` response.
- **Failure modes:** kill backend mid-speak — shim must surface a JSON-RPC error, not deadlock. When backend isn't running, HTTP clients should get a clear connection-refused surfaced by the client.
@@ -326,8 +326,8 @@ Claude Code can call `voicebox.speak` after step 4 (direct HTTP, manual config).
- **Lifespan migration** touches critical path (DB init, task queue, watchdog). Dev + packaged build both need a smoke after.
- **Shim binary size** — if `mcp` pulls in enough dep weight that PyInstaller output is awkward, fall back to a Rust shim (Tauri shell is already Rust; JSON-RPC framing is trivial).
- **Source attribution** — consider `source="mcp"` on the `Generation` model, or a dedicated `originator_client` column, if the Captures tab should filter MCP-originated generations.
- **`audio_path` in `voicebox_transcribe`** — local-only today, but if the server ever binds beyond 127.0.0.1 we need to restrict reads to `data_dir` + user-whitelist.
- **Auth** — none for now (127.0.0.1 only). If we bind outside, bearer token via `~/.voicebox/secret` + plumb through shim.
- **`audio_path` in `talkbox_transcribe`** — local-only today, but if the server ever binds beyond 127.0.0.1 we need to restrict reads to `data_dir` + user-whitelist.
- **Auth** — none for now (127.0.0.1 only). If we bind outside, bearer token via `~/.talkbox/secret` + plumb through shim.
- **HTTP MCP client support** — the plan leads with direct HTTP. Claude Code, Cursor, Windsurf, and VS Code MCP extensions all support it as of 2026, but if we discover an important client is stdio-only we still have the shim fallback ready.
- **`.mcpb` desktop extension for Claude Desktop** (v2 polish) — Claude Desktop supports a double-clickable extension bundle format. Worth revisiting after v1 ships for an even cleaner install; skipped for now since Claude Desktop isn't the primary user (Claude Code + IDE users are).
+7 -7
View File
@@ -2,11 +2,11 @@
**Status:** Planned for v0.2.0
**Issue:** [#10 OpenAI API compatibility](https://github.com/jamiepine/voicebox/issues/10)
**Issue:** [#10 OpenAI API compatibility](https://github.com/jamiepine/talkbox/issues/10)
## Overview
This feature exposes OpenAI-compatible endpoints from Voicebox, allowing any tool, library, or application that speaks the OpenAI Audio API to use Voicebox as a drop-in local replacement.
This feature exposes OpenAI-compatible endpoints from TalkBox, allowing any tool, library, or application that speaks the OpenAI Audio API to use TalkBox as a drop-in local replacement.
```mermaid
flowchart LR
@@ -16,7 +16,7 @@ flowchart LR
Apps[Third-party Apps]
end
subgraph voicebox [Voicebox Server]
subgraph talkbox [TalkBox Server]
OpenAI["/v1/audio/* endpoints"]
TTS[TTSModel]
Whisper[WhisperModel]
@@ -33,8 +33,8 @@ flowchart LR
## Use Cases
- **OpenAI SDK users**: `openai.audio.speech.create()` works with Voicebox
- **LLM frameworks**: LangChain, AutoGen, etc. can use Voicebox for TTS
- **OpenAI SDK users**: `openai.audio.speech.create()` works with TalkBox
- **LLM frameworks**: LangChain, AutoGen, etc. can use TalkBox for TTS
- **Shell scripts**: `curl` commands copy-pasted from OpenAI docs work
- **Existing integrations**: Any tool expecting OpenAI's API works without code changes
@@ -60,7 +60,7 @@ OpenAI spec: https://platform.openai.com/docs/api-reference/audio/createSpeech
**Voice Mapping Strategy:**
- `voice` parameter maps to Voicebox profile names (case-insensitive)
- `voice` parameter maps to TalkBox profile names (case-insensitive)
- If no match, use a configurable default profile
- Support special syntax: `voice: "profile:uuid"` for explicit profile ID
@@ -132,7 +132,7 @@ Add helper in [backend/profiles.py](backend/profiles.py):
```python
async def resolve_voice_for_openai(voice: str, db: Session) -> Optional[VoiceProfile]:
"""
Resolve OpenAI voice parameter to a Voicebox profile.
Resolve OpenAI voice parameter to a TalkBox profile.
Priority:
1. Exact profile name match (case-insensitive)
+30 -30
View File
@@ -64,7 +64,7 @@ Paste half (macOS):
`NSRunningApplication.activateWithOptions:` for re-activation.
- `accessibility.rs``AXIsProcessTrusted` gate.
- `paste_final_text` command — activate → 120 ms settle → save clip →
write text → ⌘V → 400 ms → restore. Skips when focus was in Voicebox
write text → ⌘V → 400 ms → restore. Skips when focus was in TalkBox
itself.
- Focus rides the `dictate:start` event payload; `DictateWindow` holds the
snapshot in a ref and consume-once-nulls on paste so a late-arriving
@@ -75,13 +75,13 @@ Paste half (macOS):
Outstanding: Windows `SendInput` / UIAutomation / `SetForegroundWindow`
equivalents, Linux `uinput` / AT-SPI equivalents (and the Wayland story),
first-run Accessibility prompt UI with deep-link to System Settings,
direct-injection path for focus-was-inside-Voicebox (step 6 — dictating
direct-injection path for focus-was-inside-TalkBox (step 6 — dictating
into our own Generate tab currently falls back to the capture list).
### Not started
- **Phase 5 — Agent voice output + persona loop.** No `/speak` endpoint, no
`voicebox.speak` MCP tool, no per-agent voice binding, no persona metadata
`talkbox.speak` MCP tool, no per-agent voice binding, no persona metadata
on profiles.
- **Phase 6 — STT engine expansion.** Only Whisper (`mlx_backend.py`).
Parakeet v3, Qwen3-ASR, Kyutai — all unregistered.
@@ -120,7 +120,7 @@ Called out in recent sessions but not yet in a phase:
## Overview
Voicebox ships the output half of a voice I/O loop: clone a voice, generate
TalkBox ships the output half of a voice I/O loop: clone a voice, generate
speech, apply effects, compose multi-voice projects. The input half — speech to
text, dictation, routing — exists today as a single Whisper model wired into the
Recording & Transcription panel. This doc proposes making voice *input* a
@@ -128,18 +128,18 @@ first-class pillar: more STT engines, a dictation shell (global hotkey, audio
capture, paste, streaming), a local LLM backend, and a user-configurable
pipeline from captured audio to whatever the user wants to do with it.
Positioning is the key move. **Voicebox becomes the local voice I/O layer for
Positioning is the key move. **TalkBox becomes the local voice I/O layer for
humans and AI agents** — a local alternative to cloud dictation tools, with the
differentiator that we also do TTS and voice cloning. The same app that
captures your voice can generate a response in any voice profile you've
cloned. "Anything voice is Voicebox."
cloned. "Anything voice is TalkBox."
### Positioning shift
Before this plan, Voicebox was **"the open-source AI voice cloning studio."**
Before this plan, TalkBox was **"the open-source AI voice cloning studio."**
Cloning was the headline capability.
After this plan, Voicebox is **"the open-source AI voice studio."** Cloning is
After this plan, TalkBox is **"the open-source AI voice studio."** Cloning is
one capability in a broader category that now spans input (STT, dictation),
intelligence (local LLM, refinement, persona), output (TTS, cloning, effects,
Stories), and routing. The word "cloning" drops out of the top-line descriptor
@@ -147,7 +147,7 @@ because it's become a feature rather than the thesis.
### Competitive frame
Voicebox ends up covering the territory of two separately-funded, separately
TalkBox ends up covering the territory of two separately-funded, separately
branded cloud incumbents that operate on opposite sides of the same voice I/O
loop:
@@ -155,7 +155,7 @@ loop:
- **WisprFlow** (~$70M raised): voice dictation for agents and power users —
the "users talk" side
Both are cloud-only. Voicebox becomes the only local alternative to either,
Both are cloud-only. TalkBox becomes the only local alternative to either,
running in one app, with a single model directory and LLM shared between input
and output. That bridging — dictation → LLM → TTS with a cloned voice in the
middle — is the thing no single incumbent can match, because neither has the
@@ -167,10 +167,10 @@ These are not engineering tasks but should ride the Phase 4 ship so marketing
and positioning stay in sync with the product.
- **README.md** — drop "cloning" from the top-line descriptor. Add a section
that explicitly frames Voicebox as "the open-source local alternative to
that explicitly frames TalkBox as "the open-source local alternative to
WisprFlow and ElevenLabs." Competitive framing belongs in the README and on
the landing page — not in-app (reads as defensive).
- **voicebox.sh landing page** — same positioning shift.
- **talkbox.sh landing page** — same positioning shift.
- **GitHub About / repo topics** — swap "voice-cloning" or similar tags for
broader "voice-io," "local-tts," "local-stt," etc.
- **Release notes** — the Phase 4 launch note is the "we're now voice I/O" moment.
@@ -187,11 +187,11 @@ and positioning stay in sync with the product.
- The **persona loop** — speak to an agent, have it reply in a cloned voice —
is a feature only we can ship. Nobody with a dictation product has TTS; nobody
with a TTS product has good dictation. The full duplex is ours.
- Agent harnesses already pipe Voicebox TTS into their stacks. Giving those
users STT from the same app closes the loop and makes Voicebox the default
- Agent harnesses already pipe TalkBox TTS into their stacks. Giving those
users STT from the same app closes the loop and makes TalkBox the default
voice I/O layer for the agentic dev-tool crowd.
- **Typing a 2,000-character TTS script is user-hostile.** The most immediate
internal win is dictating directly into Voicebox's own generation form —
internal win is dictating directly into TalkBox's own generation form —
speak the script, generate the voice. This dogfoods the whole STT pipeline
without touching a single OS-level API.
- **Voice-to-voice models are landing.** Moshi (Kyutai), GLM-4-Voice, Qwen2.5
@@ -202,7 +202,7 @@ and positioning stay in sync with the product.
## Non-goals
- Cloud fallback or "bring your own API key" STT/LLM. Local is the product.
- A separate tray-only dictation app. We extend Voicebox, not fork it.
- A separate tray-only dictation app. We extend TalkBox, not fork it.
- Replacing the Stories editor with a notes layout. Long-form capture is a
preset on top of the pipeline, not a new product surface.
- Real-time translation UI. It can exist as a transform later, but it's not in
@@ -262,7 +262,7 @@ API call (WS / HTTP) ──┘ MCP server sink
`Source → Transform → Sink` is internal, dataflow-style vocabulary (same shape
as Unix pipes, Apache Beam, Kafka) — not user-facing. The UI surface will use
Voicebox-native language (see open questions).
TalkBox-native language (see open questions).
Concrete preset examples this shape enables:
@@ -310,7 +310,7 @@ branching behavior, not four separate sinks.
| Target | Delivery strategy |
|---|---|
| Focused text field inside Voicebox | Direct React state update via event. No clipboard involved. |
| Focused text field inside TalkBox | Direct React state update via event. No clipboard involved. |
| Focused text field in another app | Accessibility-verified paste: save clipboard, write transcript, simulate paste, restore clipboard. |
| No text focus detected | Clipboard only, toast notification ("Transcript copied — no text field focused"). |
| Platform-specific special cases (terminal apps, specific editors) | Per-app overrides where the generic path misbehaves. |
@@ -355,7 +355,7 @@ Generate · Stories · Captures · Voices · Effects · Models · Settings.
### Parallel explainer on the Generate tab
The Captures settings page gets a "What's different" aside that introduces
Voicebox's dictation story. The Generate tab deserves a parallel — first-time
TalkBox's dictation story. The Generate tab deserves a parallel — first-time
users need to be told what voice generation is *for* in a post-Voice-I/O
world, not just handed a text field.
@@ -394,13 +394,13 @@ Dictation is one half of the loop — user speaks, agent listens. The other half
first-class primitive rather than being buried as a TTS loopback sink or a
consumer read-aloud button.
The shape is a single new capability: any agent can call Voicebox to speak
The shape is a single new capability: any agent can call TalkBox to speak
arbitrary text in a user-configured voice. The same pill that surfaces during
dictation surfaces during agent speech, so the user always sees what's coming
out of their machine.
```
MCP tool: voicebox.speak({ text, profile?, style? })
MCP tool: talkbox.speak({ text, profile?, style? })
REST: POST /speak { text, profile_id?, style? }
```
@@ -422,7 +422,7 @@ audio through system output, and surface the pill in a `speaking` state.
- **Mute + rate limits.** One-toggle mute for all agent speech. Per-source
rate limits prevent a runaway agent from monologuing.
This primitive is what makes "Voicebox as voice layer for every agent on your
This primitive is what makes "TalkBox as voice layer for every agent on your
machine" a concrete shipping capability rather than marketing language. MCP,
ACP, and A2A integrations all slot into it — none of those agent protocols
need to know anything about TTS models, GPU placement, or voice profiles.
@@ -517,7 +517,7 @@ tracking these models.
1. **Tab name.** Leaning **Captures** — neutral, extensible across dictation,
long-form recordings, and uploaded audio without repainting the tab later.
"Dictations" is narrower (office-productivity coded, doesn't fit meeting
recordings). "Notes" is the wrong mental model — nobody opens Voicebox to
recordings). "Notes" is the wrong mental model — nobody opens TalkBox to
write notes. "Transcriptions" is flat.
2. **Refinement vocabulary.** The LLM-post-STT step needs a user-facing name.
"Refine," "polish," "rewrite," "smart edit" are candidates. "Refinement" in
@@ -532,7 +532,7 @@ tracking these models.
5. **Long-form capture product surface.** Pure preset, or dedicated entry point
in the new tab? Leaning preset, but long-form is the feature that most
justifies its own landing page.
6. **Hotkey primitive naming.** Hold-vs-tap needs Voicebox-native phrasing in
6. **Hotkey primitive naming.** Hold-vs-tap needs TalkBox-native phrasing in
UI copy. Settings can still use industry-standard terms.
## Ordered phases
@@ -565,9 +565,9 @@ No new runtime. No `llama.cpp`, no `ollama`, no fragmented model cache.
### Phase 3 — In-app voice input
A universal mic button on every Voicebox text input. Hold, speak, release —
A universal mic button on every TalkBox text input. Hold, speak, release —
text lands in the focused field via direct React state update. No OS APIs
involved; Voicebox owns the input.
involved; TalkBox owns the input.
Marquee use cases:
@@ -593,7 +593,7 @@ of the persona loop and it lands here for free — no LLM involved, no new
backend endpoints, just a Captures-tab button that sends the transcript text
to the existing `/generate` endpoint with a user-selected voice profile and
plays the result. Category-defining differentiator from the v1 prototype
onward: Superwhisper and WisprFlow cannot do this because they have no TTS. Voicebox can, with one day of frontend wiring.
onward: Superwhisper and WisprFlow cannot do this because they have no TTS. TalkBox can, with one day of frontend wiring.
Keep it aggressively minimal on day one. A capture list, a detail view, a
model picker, a Play-as-voice dropdown. Refinement prompt editing, correction
@@ -602,11 +602,11 @@ Tier-2 work when someone actually asks for them.
### Phase 5 — Agent voice output + persona loop
Two features that together make "Voicebox as the voice layer for every agent
Two features that together make "TalkBox as the voice layer for every agent
on your machine" a shipping reality:
1. **`speak()` primitive.** New `POST /speak` endpoint and `voicebox.speak`
MCP tool. Any agent calls Voicebox to speak arbitrary text in a
1. **`speak()` primitive.** New `POST /speak` endpoint and `talkbox.speak`
MCP tool. Any agent calls TalkBox to speak arbitrary text in a
user-configured voice; the pill surfaces in a `speaking` state. Settings
UI for default voice, per-agent voice binding (Claude Code → Morgan,
Cursor → Scarlett), and a global mute.