changelog

This commit is contained in:
Jamie Pine
2026-04-25 11:17:07 -07:00
parent a6a5717eb2
commit 0dbdec4531
2 changed files with 38 additions and 38 deletions
+33 -38
View File
@@ -7,47 +7,42 @@
## [0.5.0] - 2026-04-22
**The Capture release.** Voicebox stops being just a voice-cloning studio and becomes a full AI voice studio. The loop closes in both directions: your voice goes into your computer through a global hotkey, and any agent's voice comes out of your computer through a voice you own.
When enabled, hold a key anywhere on your machine, speak, release — the transcript lands in the focused text field in whatever app you were using. Flip the primitive around and any MCP-aware agent — Claude Code, Cursor, Cline, Spacebot — speaks back through the same on-screen pill in one of your cloned voices. A local LLM sits between the two, so transcripts come out clean and voice profiles can carry a personality that reshapes what the agent actually says before it gets spoken.
**The Capture release.** Voicebox stops being just a voice-cloning studio and becomes a full AI voice studio. Hold a key anywhere on your machine, speak, release — the transcript lands in the focused text field. Flip the primitive around and any MCP-aware agent — Claude Code, Cursor, Spacebot — speaks back through an on-screen pill in one of your cloned voices. A local LLM sits between the two, so transcripts come out clean and voice profiles can carry a personality that reshapes what the agent says before it gets spoken.
### Dictation — speak anywhere, paste anywhere
- **Global hotkey capture.** Hold a customizable chord anywhere on your machine (defaults: right-Cmd + right-Option on macOS, right-Ctrl + right-Shift on Windows), speak, release. A floating on-screen pill surfaces over your current app and walks through recording → transcribing → refining → done with a live elapsed timer during the clip. Your dictation lands as a clean transcript.
- **Hotkey engine.** Chord detection runs on `keytap`, a small cross-platform library with first-class handling of left/right modifier variants on macOS. Resolves long-standing issues where left-Cmd / right-Option sometimes registered interchangeably on macOS 14+ and let the chord fire on the wrong hardware key.
- **Push-to-talk and toggle modes, each with its own chord.** The default toggle chord adds Space to the push-to-talk chord. Holding PTT and tapping Space mid-hold upgrades a hold into a hands-free session without a gap in the recording — the session rolls forward, you stop when you want.
- **Auto-paste into the focused app.** Once transcription finishes, Voicebox synthesizes a platform-native paste into whatever text field had focus when you started the chord — not wherever focus drifted while you were talking. Your clipboard is saved before and restored after, so nothing you had copied goes missing.
- **macOS paste hardening.** Synthetic paste uses the keyboard-layout-aware V keycode so the physical V key fires regardless of Dvorak / AZERTY / non-QWERTY layouts. On macOS 14+, a cooperative `NSRunningApplication.activate` handshake ensures the target app is frontmost before keystrokes inject, so paste lands in the focused field instead of Voicebox. Clipboard restore only runs when Voicebox actually overwrote the clipboard, and a paste-failure fallback retries the injection after a brief delay.
- **Chord picker UI.** Customize either chord from Settings → Captures by holding the keys you want. Left/right modifier badges show whether a key is the left or right variant, so you can pick the exact hardware signature you want to capture.
- **Defaults picked to stay out of your way.** macOS defaults deliberately avoid left-hand Cmd+Option chords so Cmd+Option+I (devtools), Cmd+Option+Esc (force quit), and Cmd+Option+Space (Spotlight) all remain yours. Windows defaults route around AltGr collisions on German/French/Spanish layouts where Ctrl+Alt synthesizes AltGr.
- **Accessibility permission is scoped.** The macOS permission prompt lives inline next to the auto-paste toggle in Settings → Captures, not as a global banner on every page. If permission isn't granted, dictation still runs and transcripts still land in the Captures tab — only synthetic paste is disabled.
- **Global hotkey capture.** Hold a customizable chord anywhere on your machine (defaults: right-Cmd + right-Option on macOS, right-Ctrl + right-Shift on Windows), speak, release. A floating on-screen pill walks through recording → transcribing → refining → done with a live elapsed timer. The transcript lands as clean text.
- **Push-to-talk and toggle modes, each with its own chord.** The default toggle chord adds Space to the push-to-talk chord. Holding PTT and tapping Space mid-hold upgrades a hold into a hands-free session without a gap in the recording.
- **Auto-paste into the focused app.** Once transcription finishes, Voicebox synthesizes a paste into whatever text field had focus when you started the chord — not wherever focus drifted while you were talking. Works across Dvorak / AZERTY layouts. Your clipboard is saved before and restored after.
- **Chord picker UI.** Customize either chord from Settings → Captures by holding the keys you want. Left/right modifier badges show whether a key is the left or right variant.
- **Defaults stay out of your way.** macOS defaults avoid left-hand Cmd+Option chords so the system shortcuts they collide with stay yours. Windows defaults route around AltGr collisions on German / French / Spanish layouts.
- **Accessibility permission is scoped.** If macOS Accessibility isn't granted, dictation still runs and transcripts still land in the Captures tab — only synthetic paste is disabled. The permission prompt lives inline next to the auto-paste toggle, not as a global banner.
### Personality — voice profiles that speak for themselves
Voice profiles now carry an optional **personality** — a free-form description of who this voice is, up to 2000 characters. When set, two new controls appear next to the generate button, each powered by a bundled Qwen3 LLM running entirely locally:
Voice profiles now carry an optional **personality** — a free-form description of who this voice is, up to 2000 characters. When set, two new controls appear next to the generate button, each powered by a new Qwen3 LLM running entirely locally:
- **Compose** — the shuffle button drops a fresh in-character line into the textarea. Click again for variety, edit before speaking.
- **Speak in character** — the wand toggle runs your input through the personality LLM before TTS, preserving every idea but delivering it in the character's voice.
Temperatures are tuned per mode (compose hot for variety, rewrite cold for fidelity) and the character framing enforces "speech only" output — no narration, no action tags, no meta-commentary. The same LLM doubles as the refinement model, so there's one local LLM in the app, not two.
The same LLM doubles as the refinement model, so there's one local LLM in the app, not two.
**API surface.** `POST /generate`, `POST /speak`, and the MCP `voicebox.speak` tool accept `personality: bool` to run input through the personality LLM before TTS. `POST /profiles/{id}/compose` powers the shuffle button. MCP client bindings carry a `default_personality: bool` that applies when `personality` isn't passed explicitly.
**API surface.** `POST /generate`, `POST /speak`, and the MCP `voicebox.speak` tool accept `personality: bool`. `POST /profiles/{id}/compose` powers the shuffle button. MCP client bindings carry a `default_personality: bool` that applies when `personality` isn't passed explicitly.
### Agents — any MCP-aware agent gets a voice
Voicebox ships a built-in **Model Context Protocol** server at `http://127.0.0.1:17493/mcp` so Claude Code, Cursor, Windsurf, Cline, VS Code MCP extensions — any MCP-aware agent — can call into your local Voicebox install. Four tools ship with dotted names:
- **`voicebox.speak`** — speak text in any voice profile, with optional `personality: true` to run through the profile's personality LLM first
- **`voicebox.transcribe`** — Whisper transcription of a base64 blob or an absolute local path
- **`voicebox.transcribe`** — Whisper transcription of a base64 blob or an absolute local path. Path mode is restricted to loopback callers so a Voicebox bound on `0.0.0.0` doesn't double as an unauthenticated arbitrary-local-file read primitive.
- **`voicebox.list_captures`** — recent captures with their transcripts
- **`voicebox.list_profiles`** — available voice profiles (cloned + preset)
- **Streamable HTTP as primary transport.** FastMCP mounted inside the existing FastAPI process (Nov-2025 MCP spec, post-SSE). Cursor / Windsurf / VS Code / Claude Code all support it out of the box — drop a `mcpServers` block with the URL and an `X-Voicebox-Client-Id` header.
- **Stdio shim for clients that don't speak HTTP MCP.** A `voicebox-mcp` binary ships inside the app bundle as a Tauri sidecar — ~18 MB PyInstaller build with torch / transformers / mlx explicitly excluded. The Settings page renders the install snippet with the right absolute path pre-filled.
- **Per-client voice binding.** Pin Claude Code to Morgan, Cursor to Scarlett, Cline to its own voice — the `X-Voicebox-Client-Id` header resolves to a bound voice whenever `speak` is called without an explicit `profile`. Managed in **Settings → MCP**, with an auto-stamped `last_seen_at` timestamp on each row so you can tell the install actually took. New clients register their binding row on first call.
- **Streamable HTTP as primary transport.** Cursor / Windsurf / VS Code / Claude Code all support it out of the box — drop a `mcpServers` block with the URL and an `X-Voicebox-Client-Id` header.
- **Stdio shim for clients that don't speak HTTP MCP.** A `voicebox-mcp` binary ships inside the app bundle as a Tauri sidecar. The Settings page renders the install snippet with the right absolute path pre-filled.
- **Per-client voice binding.** Pin Claude Code to Morgan, Cursor to Scarlett, Cline to its own voice — the `X-Voicebox-Client-Id` header resolves to a bound voice whenever `speak` is called without an explicit `profile`. Managed in **Settings → MCP**.
- **Profile resolution precedence.** Explicit `profile` arg (name or id, case-insensitive) → per-client binding → global default from `capture_settings.default_playback_voice_id` → error with a pointer to Settings.
- **Speaking pill.** Agent-initiated speech surfaces the same on-screen pill as dictation, in a new `speaking` state with the profile name and an elapsed timer. Driven by SSE at `/events/speak`; a Tauri-side `dictate:show` handler repositions and reveals the pill over the current monitor even when the main Voicebox window is hidden. Silent background TTS is a trust hazard — the pill always shows what's coming out of your machine.
- **Pill resilience.** The Tauri-side `/events/speak` monitor retries with escalating backoff on disconnect and times out idle connections, so a server hiccup can't leave the pill stuck in `speaking` forever. Lifespan shutdown drains in-flight MCP requests before unloading models, so a `voicebox.speak` call racing with app exit returns cleanly.
- **Speaking pill.** Agent-initiated speech surfaces the same on-screen pill as dictation, in a `speaking` state with the profile name and an elapsed timer. Silent background TTS is a trust hazard — the pill always shows what's coming out of your machine.
- **`POST /speak` REST wrapper.** Same code path and voice resolution for shell scripts, ACP, A2A, GitHub Actions, or anything else that isn't MCP-native.
**Claude Code one-liner:**
@@ -56,15 +51,13 @@ Voicebox ships a built-in **Model Context Protocol** server at `http://127.0.0.1
claude mcp add voicebox --transport http --url http://127.0.0.1:17493/mcp --header "X-Voicebox-Client-Id: claude-code"
```
### Refinement hardening
### Refinement
Refinement shipped earlier; 0.5.0 closes the stubborn edge cases:
A clean transcript needs more than Whisper. Each capture flows through a small Qwen3 LLM that strips fillers, fixes punctuation, and optionally rewrites self-corrections — all on-device.
- **Deterministic loop-stripping before the LLM sees the transcript.** Whisper's "thanks for watching thanks for watching thanks for watching…" hallucination loops are collapsed at a six-identical-tokens threshold (case-insensitive) so a small refinement model can't echo them back. Coverage spans single-word runs, multi-word phrases, CJK character runs, and Japanese emphasis patterns; legitimate repetition ("no, no, no, no, no") doesn't cross the threshold.
- **Refinement flags snapshot per capture.** `smart_cleanup`, `self_correction`, and `preserve_technical` are stored on each capture, so refinement can be re-run later with different flags without losing the raw transcript.
- **Ten-transcript evaluation harness** (`backend/tests/test_refinement_samples.py`) scoring prompt leaks, answer leaks, loop echoes, filler removal, length-ratio outliers, substring preservation ("npm install", "handleSubmit"), and question-mark survival against every bundled refinement model size.
- **Refinement model picker** — Qwen3 0.6B (400 MB, very fast), 1.7B (1.1 GB, fast), 4B (2.5 GB, full quality). 0.6B is the default; 1.7B is the sweet spot for transcripts with code identifiers.
- **MLX bundling for Apple Silicon.** The PyInstaller build now collects the full `mlx_lm` tokenizer + config tree, so the MLX-backed Qwen3 refinement / personality LLM loads inside the shipped `.app` bundle without extra setup on Apple Silicon.
- **Loop-stripping before the LLM sees the transcript.** Whisper's "thanks for watching thanks for watching thanks for watching…" hallucination loops are collapsed at a six-identical-tokens threshold (case-insensitive) so a small refinement model can't echo them back. Coverage spans single-word runs, multi-word phrases, CJK character runs, and Japanese emphasis patterns; legitimate repetition ("no, no, no, no, no") doesn't cross the threshold.
- **Per-capture flag snapshot.** `smart_cleanup`, `self_correction`, and `preserve_technical` are stored on each capture, so refinement can be re-run later with different flags without losing the raw transcript.
- **Model picker** — Qwen3 0.6B (400 MB, very fast), 1.7B (1.1 GB, fast), 4B (2.5 GB, full quality). 0.6B is the default; 1.7B is the sweet spot for transcripts with code identifiers.
### Captures tab + settings
@@ -73,27 +66,29 @@ Settings → Captures is now the home for the whole dictation flow:
- **Dictation**: global shortcut toggle, push-to-talk chord picker, toggle chord picker, live pill preview, auto-paste into focused field (with inline accessibility prompt).
- **Transcription**: model picker (Whisper Base / Small / Medium / Large / Turbo), language lock.
- **Refinement**: auto-refine toggle, model picker, smart cleanup, remove self-corrections, preserve technical terms.
- **Playback**: default voice for the Captures tab's "Play as" action.
- **Playback**: default voice for the Captures tab's "Play as" action — picking a voice from the split-button persists the choice across tab switches and restarts.
- **Storage**: captures folder quick-open.
### Stories — timeline editor
The Stories tab graduates from a TTS sequencer into a real timeline editor. Same generation-row backing, but clips now compose with imported audio, per-clip levels, and a flexible track stack.
- **Import external audio.** Drag a music file onto the story content area or pick one from the new "Import audio" entry in the add-clip popover. Accepted formats: wav / mp3 / flac / ogg / m4a / aac / webm, capped at 200 MB. Imported clips show their filename instead of a profile name and skip the regenerate / version-picker controls — there's nothing to regenerate.
- **Per-clip volume.** A `Volume2` icon in the clip-edit toolbar opens a 0200% slider. Adjustments apply live and to exports. Split and duplicate carry the volume forward into the new clips.
- **Regenerate** from both the clip's chat-list dropdown and the track-editor toolbar. Re-runs the underlying generation through the same path the History tab uses, with completion tracked in the global pending set.
- **Add empty tracks above or below the timeline** via tiny `+` strips at the top of the topmost label cell and the bottom of the bottommost. Sticky in the label column so they follow horizontal scroll.
- **Zoom bar tracks the project.** Min scope is 10 seconds visible (zoomed in cap), max is the entire project (zoomed out cap), default lands on 60 s. Both the +/ buttons and the scrollbar edge-drag handles clamp to those dynamic bounds.
### Interface
- **Theme selector.** Light / dark / system in **Settings → General**, persisted across sessions. System mode listens for OS-level appearance changes and flips live without a restart.
- **Scrubbable waveform player on captures.** The capture detail card now embeds a WaveSurfer waveform with click-to-seek and a current / total timestamp pair, replacing the static duration label.
- **Capture pill light mode.** The on-screen pill gets a dedicated light palette so it stays legible against bright windows, and the inline waveform progress color reads against the pill background in both themes.
- **Stories polish.** The Stories sidebar matches the Captures sidebar (search, item layout, border treatment), the floating generate box moved into the right column with a top fade mask, and the track editor ships sticky track labels (aligned via flex rows) plus a custom horizontal scrollbar with dedicated left/right zoom handles.
- **Capture pill light mode.** The on-screen pill gets a dedicated light palette so it stays legible against bright windows.
- **Readiness checklist in the Captures settings sidebar.** The same six-gate checklist the Captures empty state uses mirrors into Settings → Captures so a red gate can't hide behind a green toggle. Hidden once every gate is green. macOS-only rows (Input Monitoring, Accessibility) hide entirely on Windows and Linux.
### Windows parity
- **Synthetic paste** via `SendInput` with correct scan codes, plus a `SetForegroundWindow` + `AttachThreadInput` handshake to defeat foreground-lock when pasting into a window that wasn't frontmost at chord-start.
- **Right-hand default chord** (Ctrl+Shift) to avoid AltGr collisions on layouts where Ctrl+Alt is the compose key.
- **Focus capture via UIAutomation** for control-class identity and `GetForegroundWindow` for window identity, snapshotted at chord-start so paste lands in the original field even if focus drifts during transcribe/refine.
- **UAC/UIPI caveat**: synthetic paste into an elevated window from a non-elevated Voicebox is blocked by Windows itself. Run Voicebox elevated if you regularly dictate into elevated apps.
### Landing page — /capture
New `/capture` route tells the Capture story end-to-end. Hero line: *"Just talk to your computer."* Three pillars — STT (Whisper, Whisper Turbo), refined transcripts, agents speaking in voices you own. Drop-in MCP config block for Claude Code / Cursor / Cline. Agent-integration section showing per-client voice binding.
Same dictation flow on Windows. Right-hand default chord (Ctrl+Shift) avoids AltGr collisions on layouts where Ctrl+Alt is the compose key. Focus is captured at chord-start so paste lands in the original field even if focus drifts during transcribe/refine.
## [0.4.5] - 2026-04-22
+5
View File
@@ -116,6 +116,11 @@ export function Footer() {
Issues
</Link>
</li>
<li>
<a href="/sponsors" className="hover:text-foreground transition-colors">
Corporate Sponsorship
</a>
</li>
</ul>
</div>