mirror of
https://github.com/jamiepine/voicebox.git
synced 2026-09-16 21:30:39 -07:00
personality: bool API, i18n across the app
- Collapse intent tri-state (respond/rewrite/compose) to `personality: bool` on /generate, /speak, and voicebox.speak. Drop respond entirely; keep compose as a standalone button via /profiles/{id}/compose. Remove /rewrite, /respond, and /speak profile endpoints.
- FloatingGenerateBox: Wand2 persona toggle + Dices compose button appear when the selected profile has a personality. ProfileCard badges Wand2 alongside the effects Sparkles.
- MCP bindings: default_intent column → default_personality: bool. Migration drops the legacy column.
- i18n: en / ja / zh-CN / zh-TW translation files filled out and wired through the capture, server, and profile UI.
```ts
voicebox.speak({
text: "Deploy complete.",
profile: "Morgan",
personality: true, // rewrite through the profile's personality LLM
});
```
This commit is contained in:
@@ -13,7 +13,7 @@ you own. Everything runs on your hardware.
|
||||
|
||||
- **Dictation** — hold a chord anywhere on your machine, speak, release; the transcript pastes into the focused field
|
||||
- **Captures tab** — paired audio + transcript archive, retranscribe / refine / play-as-voice
|
||||
- **Voice personalities** — per-profile compose / rewrite / respond, powered by a local LLM
|
||||
- **Voice personalities** — per-profile compose button + persona-rewrite toggle, powered by a local LLM
|
||||
- **Agents speak back** — any MCP-aware agent can call Voicebox to speak in one of your cloned voices
|
||||
- **7 TTS engines** — Qwen3-TTS, Qwen CustomVoice, LuxTTS, Chatterbox Multilingual, Chatterbox Turbo, HumeAI TADA, Kokoro
|
||||
- **Cloning and preset voices** — zero-shot cloning or 50+ curated preset voices
|
||||
@@ -42,5 +42,5 @@ you own. Everything runs on your hardware.
|
||||
- [Installation](/overview/installation) — download and install Voicebox
|
||||
- [Quick Start](/overview/quick-start) — get up and running in 5 minutes
|
||||
- [Dictation](/overview/dictation) — start talking to your computer
|
||||
- [Voice Personalities](/overview/voice-personalities) — compose, rewrite, respond in any profile
|
||||
- [Voice Personalities](/overview/voice-personalities) — compose and rewrite in any profile
|
||||
- [API Reference](/api-reference) — integrate voice synthesis into your apps
|
||||
|
||||
@@ -185,8 +185,8 @@ of your machine can talk to.
|
||||
The global hotkey flow that feeds most captures.
|
||||
</Card>
|
||||
<Card title="Voice Personalities" href="/overview/voice-personalities">
|
||||
Per-profile compose / rewrite / respond modes for captures you want to
|
||||
transform, not just transcribe.
|
||||
Per-profile compose button and persona rewrite toggle for captures you
|
||||
want to transform, not just transcribe.
|
||||
</Card>
|
||||
<Card title="Creating Voice Profiles" href="/overview/creating-voice-profiles">
|
||||
Promote a capture into a voice sample on a profile.
|
||||
|
||||
@@ -200,7 +200,7 @@ A few cases where Voicebox deliberately does *not* synthesize a paste:
|
||||
The paired audio + transcript archive every dictation lands in.
|
||||
</Card>
|
||||
<Card title="Voice Personalities" href="/overview/voice-personalities">
|
||||
The same local LLM doubles as per-profile compose / rewrite / respond.
|
||||
The same local LLM powers per-profile compose and persona rewrite.
|
||||
</Card>
|
||||
<Card title="Transcription" href="/developer/transcription">
|
||||
Developer-level details on Whisper, Whisper Turbo, and the STT backend.
|
||||
|
||||
@@ -14,7 +14,7 @@ accounts:
|
||||
- **Agents talk back** — any MCP-aware agent can call Voicebox to speak in
|
||||
one of your cloned voices
|
||||
- **Voices speak for themselves** — voice profiles can carry a personality
|
||||
that compose, rewrite, or respond to text before it's spoken
|
||||
that composes fresh lines or rewrites text before it's spoken
|
||||
|
||||
It's the free, local alternative to both ElevenLabs (voice cloning and TTS)
|
||||
and WisprFlow (voice dictation for agents and power users) — covering both
|
||||
@@ -32,7 +32,7 @@ shared between input and output.
|
||||
- **Preset voices** — 50+ curated voices via Kokoro and Qwen CustomVoice
|
||||
for when you don't want to clone (see [Preset Voices](/overview/preset-voices))
|
||||
- **Voice personalities** — optional free-form personality on any profile
|
||||
plus compose / rewrite / respond modes powered by a local LLM (see
|
||||
plus a compose button and persona-rewrite toggle powered by a local LLM (see
|
||||
[Voice Personalities](/overview/voice-personalities))
|
||||
- **Post-processing effects** — pitch shift, reverb, delay, chorus,
|
||||
compression, filters (Spotify's Pedalboard)
|
||||
@@ -66,7 +66,7 @@ between dictation, the Captures tab, and per-profile personality modes:
|
||||
| Layer | Models |
|
||||
|---|---|
|
||||
| **STT** | Whisper Base / Small / Medium / Large / Turbo (PyTorch or MLX) |
|
||||
| **LLM** | Qwen3 0.6B / 1.7B / 4B (refinement + per-profile compose / rewrite / respond) |
|
||||
| **LLM** | Qwen3 0.6B / 1.7B / 4B (refinement + per-profile compose / persona-rewrite) |
|
||||
|
||||
No cloud fallback, no bring-your-own-API-key. Local is the product.
|
||||
|
||||
|
||||
@@ -117,7 +117,7 @@ voicebox.speak({
|
||||
text: "Deploy complete.",
|
||||
profile?: "Morgan", // name or id; falls back to per-client binding, then default
|
||||
engine?: "qwen", // qwen | qwen_custom_voice | luxtts | chatterbox | chatterbox_turbo | tada | kokoro
|
||||
intent?: "respond", // respond | rewrite | compose — only if the profile has a personality
|
||||
personality?: true, // rewrite via the profile's personality LLM before TTS; default comes from the per-client binding
|
||||
language?: "en",
|
||||
})
|
||||
```
|
||||
@@ -134,10 +134,9 @@ Returns:
|
||||
}
|
||||
```
|
||||
|
||||
- **Plain TTS** — omit `intent`. Text is spoken as-is.
|
||||
- **Persona mode** — pass `intent` and the profile must have a personality set.
|
||||
The LLM transforms the text (respond to it, rewrite it in character, or
|
||||
compose a fresh utterance) before TTS. See [Voice Personalities](/overview/voice-personalities).
|
||||
- **Plain TTS** — `personality: false` (or omitted + binding default is false). Text is spoken as-is.
|
||||
- **Persona mode** — `personality: true` and the profile must have a personality prompt set.
|
||||
The LLM rewrites the text in character before TTS. See [Voice Personalities](/overview/voice-personalities).
|
||||
|
||||
### `voicebox.transcribe`
|
||||
|
||||
@@ -196,7 +195,7 @@ carries:
|
||||
| `label` | Display name in the Settings UI (e.g. "Claude Code"). |
|
||||
| `profile_id` | The voice this client uses when `profile` isn't passed. |
|
||||
| `default_engine` | Override the TTS engine for this client. |
|
||||
| `default_intent` | Default persona mode (`respond` / `rewrite` / `compose`). |
|
||||
| `default_personality` | When true, `voicebox.speak` routes through the profile's personality LLM (rewrite) by default. |
|
||||
| `last_seen_at` | Last time the server saw a request from this client. |
|
||||
|
||||
`last_seen_at` is stamped automatically by middleware on every `/mcp/*`
|
||||
@@ -228,7 +227,7 @@ curl -X POST http://127.0.0.1:17493/speak \
|
||||
```
|
||||
|
||||
Body fields match the MCP tool: `text`, optional `profile`, `engine`,
|
||||
`intent`, `language`. Returns a `GenerationResponse` — the same shape as
|
||||
`personality`, `language`. Returns a `GenerationResponse` — the same shape as
|
||||
`POST /generate`.
|
||||
|
||||
## Debugging
|
||||
@@ -286,7 +285,7 @@ For the full developer-facing tour of the code layout, see
|
||||
|
||||
<Cards>
|
||||
<Card title="Voice Personalities" href="/overview/voice-personalities">
|
||||
Persona mode (`intent=respond/rewrite/compose`) for agents that should
|
||||
Persona mode (`personality: true`) for agents that should
|
||||
transform text in-character before speaking.
|
||||
</Card>
|
||||
<Card title="Dictation" href="/overview/dictation">
|
||||
|
||||
@@ -1,18 +1,19 @@
|
||||
---
|
||||
title: "Voice Personalities"
|
||||
description: "Attach a personality to a voice profile and use Compose, Rewrite, or Respond to generate in-character speech — all powered by a local LLM."
|
||||
description: "Attach a personality to a voice profile, compose fresh in-character lines, and rewrite input text in their voice — all powered by a local LLM."
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
A **personality** is an optional free-form description attached to a voice
|
||||
profile — who this voice is, how they speak, what they care about. Set one
|
||||
and three new actions appear on the profile, each powered by a bundled
|
||||
Qwen3 LLM running entirely locally:
|
||||
and two new controls appear next to the generate button, both powered by a
|
||||
bundled Qwen3 LLM running entirely locally:
|
||||
|
||||
- **Compose** — generate a fresh utterance in this character's voice
|
||||
- **Rewrite** — restate your text in their voice while preserving every idea
|
||||
- **Respond** — treat your text as a prompt and produce the character's reply
|
||||
- **Compose** — drop a fresh in-character line into the textarea. Click
|
||||
again for a different take.
|
||||
- **Speak in character** — a toggle that rewrites your input text in the
|
||||
character's voice before TTS, preserving every idea.
|
||||
|
||||
The LLM produces the text. The voice profile speaks it. No cloud round-trip,
|
||||
no external API — the whole loop runs on your hardware.
|
||||
@@ -41,15 +42,16 @@ Good descriptions tend to include:
|
||||
You can set a personality on any voice profile type — cloned or preset. The
|
||||
three modes work identically regardless of engine.
|
||||
|
||||
## The three modes
|
||||
## The two actions
|
||||
|
||||
Each mode is tuned for a specific job and the LLM temperature is adjusted
|
||||
Each action is tuned for a specific job and the LLM temperature is adjusted
|
||||
to match.
|
||||
|
||||
### Compose
|
||||
|
||||
Generate a fresh utterance in the character's voice, with no seed text.
|
||||
Click again to get a different take.
|
||||
Click the shuffle button to drop a line straight into the generate
|
||||
textarea; click again for a different take.
|
||||
|
||||
- **When to use:** prototyping, sampling a character's voice, brainstorming
|
||||
a line without typing one first
|
||||
@@ -57,30 +59,20 @@ Click again to get a different take.
|
||||
- **Typical output:** a short, punchy line that fits the character's
|
||||
register
|
||||
|
||||
### Rewrite
|
||||
### Speak in character (rewrite)
|
||||
|
||||
Take your input text and restate it in the character's voice while
|
||||
preserving every idea. High-fidelity mode — the content doesn't change, only
|
||||
the voice does.
|
||||
Flip the persona toggle and whatever you type (or dictate) gets rewritten in
|
||||
the character's voice before TTS — every idea preserved, only the phrasing
|
||||
changes. High-fidelity mode: the content doesn't change, only the voice does.
|
||||
|
||||
- **When to use:** turning a dictated memo into in-character speech; lifting
|
||||
a plain-English script into a specific voice without editing by hand
|
||||
- **Temperature:** cold — faithfulness wins
|
||||
- **Typical output:** same ideas, same order, different phrasing and cadence
|
||||
|
||||
### Respond
|
||||
|
||||
Treat your input as a prompt and produce the character's reply — as if
|
||||
you'd said it *to* them.
|
||||
|
||||
- **When to use:** spoken-input agents; Q&A with a specific voice;
|
||||
interactive character experiences
|
||||
- **Temperature:** balanced — creative but grounded
|
||||
- **Typical output:** a reply to your prompt, written in-character
|
||||
|
||||
## Speech-only framing
|
||||
|
||||
All three modes enforce **speech-only** output. The LLM is prompted to
|
||||
Both modes enforce **speech-only** output. The LLM is prompted to
|
||||
produce things a person would actually say out loud — no narration, no
|
||||
action tags (`*sighs*`, `[laughs]`), no meta-commentary, no markdown
|
||||
formatting, no stage directions.
|
||||
@@ -109,57 +101,58 @@ Pick a size in **Settings → Captures → Refinement → Refinement model** —
|
||||
personality modes reuse it. If you switch models, both refinement and
|
||||
personality output pick up the change on the next call.
|
||||
|
||||
## Using the modes
|
||||
## Using the controls
|
||||
|
||||
The three actions appear as buttons on the profile when a personality is
|
||||
set. For each:
|
||||
Both controls appear on the floating generate box when the selected profile
|
||||
has a personality set.
|
||||
|
||||
<Steps>
|
||||
<Step title="Type or dictate your input">
|
||||
Rewrite and Respond need input text. Compose doesn't.
|
||||
<Step title="Compose — shuffle a line">
|
||||
Click the shuffle button. The LLM runs and the result fills the generate
|
||||
textarea. Edit if you want, then hit generate.
|
||||
</Step>
|
||||
<Step title="Click the mode">
|
||||
The LLM runs, then the result fills the generate box.
|
||||
</Step>
|
||||
<Step title="Hit generate">
|
||||
The TTS engine speaks the LLM output in the profile's voice. The result
|
||||
lands in generation history as a normal generation.
|
||||
<Step title="Speak in character — toggle persona rewrite">
|
||||
Type (or dictate) what you want said. Flip the wand toggle on. Hit
|
||||
generate — Voicebox runs the text through the personality LLM first,
|
||||
then TTS speaks the rewritten version. Leave the toggle off for plain
|
||||
TTS.
|
||||
</Step>
|
||||
</Steps>
|
||||
|
||||
Each button also has an inline regenerate affordance — click again to
|
||||
resample. Compose will give you something totally different; Rewrite and
|
||||
Respond will give you a variation on the same content.
|
||||
Compose always gives you something different on re-click. The persona
|
||||
toggle, on the other hand, is a mode — it applies to every generate call
|
||||
until you flip it back off.
|
||||
|
||||
## Use cases
|
||||
|
||||
- **Agents that speak in a voice you own.** Combine Respond with the
|
||||
built-in [MCP Server](/overview/mcp-server) so Claude Code, Cursor,
|
||||
- **Agents that speak in a voice you own.** Combine the persona toggle with
|
||||
the built-in [MCP Server](/overview/mcp-server) so Claude Code, Cursor,
|
||||
Cline, or any MCP-aware agent can talk back through a profile with a
|
||||
personality. The agent calls `voicebox.speak({ text, profile, intent:
|
||||
"respond" })` and Voicebox produces in-character speech in your cloned
|
||||
voice.
|
||||
personality. The agent calls `voicebox.speak({ text, profile, personality:
|
||||
true })` and Voicebox rewrites the text in character before speaking.
|
||||
- **Interactive characters.** Games, narrative tools, accessibility
|
||||
experiences. A character with a personality description plus a cloned
|
||||
voice becomes a reusable prop.
|
||||
- **Accessibility.** People who can't speak in their original voice can
|
||||
keep a personality description of how they used to sound and use Rewrite
|
||||
to turn typed input into in-character speech.
|
||||
- **Creative drafting.** Write a plain outline, Rewrite line-by-line into
|
||||
the character's voice, drop the audio into a Story.
|
||||
keep a personality description of how they used to sound and use the
|
||||
rewrite toggle to turn typed input into in-character speech.
|
||||
- **Creative drafting.** Write a plain outline, flip the persona toggle,
|
||||
generate line-by-line into the character's voice, drop the audio into a
|
||||
Story.
|
||||
|
||||
## API surface
|
||||
|
||||
Personalities and the three modes are accessible via REST:
|
||||
Personalities are accessible via REST:
|
||||
|
||||
| Method | Endpoint | Body |
|
||||
|---|---|---|
|
||||
| `PUT` | `/profiles/{id}` | Include a `personality` field up to 2,000 chars to set it. |
|
||||
| `POST` | `/profiles/{id}/speak` | Runs the LLM + TTS in one shot. Body includes `text`, `intent` (`compose`, `rewrite`, or `respond`), optional `engine`, `language`. |
|
||||
| `POST` | `/profiles/{id}/compose` | No body. Returns a fresh in-character utterance as text. |
|
||||
| `POST` | `/generate` | Include `personality: true` to run input text through the personality LLM before TTS. Same for `POST /speak`. |
|
||||
|
||||
The `/profiles/{id}/speak` endpoint is the same primitive MCP's
|
||||
`voicebox.speak` tool calls when that ships. Scripts and agents can use it
|
||||
directly today.
|
||||
`POST /generate` with `personality: true` is the same primitive MCP's
|
||||
`voicebox.speak` tool uses when you pass `personality: true`. Scripts and
|
||||
agents can use it directly.
|
||||
|
||||
## Limits and gotchas
|
||||
|
||||
|
||||
Reference in New Issue
Block a user