personality: bool API, i18n across the app

- Collapse intent tri-state (respond/rewrite/compose) to `personality: bool` on /generate, /speak, and voicebox.speak. Drop respond entirely; keep compose as a standalone button via /profiles/{id}/compose. Remove /rewrite, /respond, and /speak profile endpoints.
- FloatingGenerateBox: Wand2 persona toggle + Dices compose button appear when the selected profile has a personality. ProfileCard badges Wand2 alongside the effects Sparkles.
- MCP bindings: default_intent column → default_personality: bool. Migration drops the legacy column.
- i18n: en / ja / zh-CN / zh-TW translation files filled out and wired through the capture, server, and profile UI.

```ts
voicebox.speak({
  text: "Deploy complete.",
  profile: "Morgan",
  personality: true, // rewrite through the profile's personality LLM
});
```
This commit is contained in:
Jamie Pine
2026-04-23 17:31:23 -07:00
parent 7c50e189cb
commit abf5dfda8c
41 changed files with 2146 additions and 1193 deletions
+2 -2
View File
@@ -13,7 +13,7 @@ you own. Everything runs on your hardware.
- **Dictation** — hold a chord anywhere on your machine, speak, release; the transcript pastes into the focused field
- **Captures tab** — paired audio + transcript archive, retranscribe / refine / play-as-voice
- **Voice personalities** — per-profile compose / rewrite / respond, powered by a local LLM
- **Voice personalities** — per-profile compose button + persona-rewrite toggle, powered by a local LLM
- **Agents speak back** — any MCP-aware agent can call Voicebox to speak in one of your cloned voices
- **7 TTS engines** — Qwen3-TTS, Qwen CustomVoice, LuxTTS, Chatterbox Multilingual, Chatterbox Turbo, HumeAI TADA, Kokoro
- **Cloning and preset voices** — zero-shot cloning or 50+ curated preset voices
@@ -42,5 +42,5 @@ you own. Everything runs on your hardware.
- [Installation](/overview/installation) — download and install Voicebox
- [Quick Start](/overview/quick-start) — get up and running in 5 minutes
- [Dictation](/overview/dictation) — start talking to your computer
- [Voice Personalities](/overview/voice-personalities) — compose, rewrite, respond in any profile
- [Voice Personalities](/overview/voice-personalities) — compose and rewrite in any profile
- [API Reference](/api-reference) — integrate voice synthesis into your apps
+2 -2
View File
@@ -185,8 +185,8 @@ of your machine can talk to.
The global hotkey flow that feeds most captures.
</Card>
<Card title="Voice Personalities" href="/overview/voice-personalities">
Per-profile compose / rewrite / respond modes for captures you want to
transform, not just transcribe.
Per-profile compose button and persona rewrite toggle for captures you
want to transform, not just transcribe.
</Card>
<Card title="Creating Voice Profiles" href="/overview/creating-voice-profiles">
Promote a capture into a voice sample on a profile.
+1 -1
View File
@@ -200,7 +200,7 @@ A few cases where Voicebox deliberately does *not* synthesize a paste:
The paired audio + transcript archive every dictation lands in.
</Card>
<Card title="Voice Personalities" href="/overview/voice-personalities">
The same local LLM doubles as per-profile compose / rewrite / respond.
The same local LLM powers per-profile compose and persona rewrite.
</Card>
<Card title="Transcription" href="/developer/transcription">
Developer-level details on Whisper, Whisper Turbo, and the STT backend.
+3 -3
View File
@@ -14,7 +14,7 @@ accounts:
- **Agents talk back** — any MCP-aware agent can call Voicebox to speak in
one of your cloned voices
- **Voices speak for themselves** — voice profiles can carry a personality
that compose, rewrite, or respond to text before it's spoken
that composes fresh lines or rewrites text before it's spoken
It's the free, local alternative to both ElevenLabs (voice cloning and TTS)
and WisprFlow (voice dictation for agents and power users) — covering both
@@ -32,7 +32,7 @@ shared between input and output.
- **Preset voices** — 50+ curated voices via Kokoro and Qwen CustomVoice
for when you don't want to clone (see [Preset Voices](/overview/preset-voices))
- **Voice personalities** — optional free-form personality on any profile
plus compose / rewrite / respond modes powered by a local LLM (see
plus a compose button and persona-rewrite toggle powered by a local LLM (see
[Voice Personalities](/overview/voice-personalities))
- **Post-processing effects** — pitch shift, reverb, delay, chorus,
compression, filters (Spotify's Pedalboard)
@@ -66,7 +66,7 @@ between dictation, the Captures tab, and per-profile personality modes:
| Layer | Models |
|---|---|
| **STT** | Whisper Base / Small / Medium / Large / Turbo (PyTorch or MLX) |
| **LLM** | Qwen3 0.6B / 1.7B / 4B (refinement + per-profile compose / rewrite / respond) |
| **LLM** | Qwen3 0.6B / 1.7B / 4B (refinement + per-profile compose / persona-rewrite) |
No cloud fallback, no bring-your-own-API-key. Local is the product.
+7 -8
View File
@@ -117,7 +117,7 @@ voicebox.speak({
text: "Deploy complete.",
profile?: "Morgan", // name or id; falls back to per-client binding, then default
engine?: "qwen", // qwen | qwen_custom_voice | luxtts | chatterbox | chatterbox_turbo | tada | kokoro
intent?: "respond", // respond | rewrite | compose — only if the profile has a personality
personality?: true, // rewrite via the profile's personality LLM before TTS; default comes from the per-client binding
language?: "en",
})
```
@@ -134,10 +134,9 @@ Returns:
}
```
- **Plain TTS** — omit `intent`. Text is spoken as-is.
- **Persona mode** — pass `intent` and the profile must have a personality set.
The LLM transforms the text (respond to it, rewrite it in character, or
compose a fresh utterance) before TTS. See [Voice Personalities](/overview/voice-personalities).
- **Plain TTS** — `personality: false` (or omitted + binding default is false). Text is spoken as-is.
- **Persona mode** — `personality: true` and the profile must have a personality prompt set.
The LLM rewrites the text in character before TTS. See [Voice Personalities](/overview/voice-personalities).
### `voicebox.transcribe`
@@ -196,7 +195,7 @@ carries:
| `label` | Display name in the Settings UI (e.g. "Claude Code"). |
| `profile_id` | The voice this client uses when `profile` isn't passed. |
| `default_engine` | Override the TTS engine for this client. |
| `default_intent` | Default persona mode (`respond` / `rewrite` / `compose`). |
| `default_personality` | When true, `voicebox.speak` routes through the profile's personality LLM (rewrite) by default. |
| `last_seen_at` | Last time the server saw a request from this client. |
`last_seen_at` is stamped automatically by middleware on every `/mcp/*`
@@ -228,7 +227,7 @@ curl -X POST http://127.0.0.1:17493/speak \
```
Body fields match the MCP tool: `text`, optional `profile`, `engine`,
`intent`, `language`. Returns a `GenerationResponse` — the same shape as
`personality`, `language`. Returns a `GenerationResponse` — the same shape as
`POST /generate`.
## Debugging
@@ -286,7 +285,7 @@ For the full developer-facing tour of the code layout, see
<Cards>
<Card title="Voice Personalities" href="/overview/voice-personalities">
Persona mode (`intent=respond/rewrite/compose`) for agents that should
Persona mode (`personality: true`) for agents that should
transform text in-character before speaking.
</Card>
<Card title="Dictation" href="/overview/dictation">
@@ -1,18 +1,19 @@
---
title: "Voice Personalities"
description: "Attach a personality to a voice profile and use Compose, Rewrite, or Respond to generate in-character speech — all powered by a local LLM."
description: "Attach a personality to a voice profile, compose fresh in-character lines, and rewrite input text in their voice — all powered by a local LLM."
---
## Overview
A **personality** is an optional free-form description attached to a voice
profile — who this voice is, how they speak, what they care about. Set one
and three new actions appear on the profile, each powered by a bundled
Qwen3 LLM running entirely locally:
and two new controls appear next to the generate button, both powered by a
bundled Qwen3 LLM running entirely locally:
- **Compose** — generate a fresh utterance in this character's voice
- **Rewrite** — restate your text in their voice while preserving every idea
- **Respond** — treat your text as a prompt and produce the character's reply
- **Compose** — drop a fresh in-character line into the textarea. Click
again for a different take.
- **Speak in character** — a toggle that rewrites your input text in the
character's voice before TTS, preserving every idea.
The LLM produces the text. The voice profile speaks it. No cloud round-trip,
no external API — the whole loop runs on your hardware.
@@ -41,15 +42,16 @@ Good descriptions tend to include:
You can set a personality on any voice profile type — cloned or preset. The
three modes work identically regardless of engine.
## The three modes
## The two actions
Each mode is tuned for a specific job and the LLM temperature is adjusted
Each action is tuned for a specific job and the LLM temperature is adjusted
to match.
### Compose
Generate a fresh utterance in the character's voice, with no seed text.
Click again to get a different take.
Click the shuffle button to drop a line straight into the generate
textarea; click again for a different take.
- **When to use:** prototyping, sampling a character's voice, brainstorming
a line without typing one first
@@ -57,30 +59,20 @@ Click again to get a different take.
- **Typical output:** a short, punchy line that fits the character's
register
### Rewrite
### Speak in character (rewrite)
Take your input text and restate it in the character's voice while
preserving every idea. High-fidelity mode — the content doesn't change, only
the voice does.
Flip the persona toggle and whatever you type (or dictate) gets rewritten in
the character's voice before TTS — every idea preserved, only the phrasing
changes. High-fidelity mode: the content doesn't change, only the voice does.
- **When to use:** turning a dictated memo into in-character speech; lifting
a plain-English script into a specific voice without editing by hand
- **Temperature:** cold — faithfulness wins
- **Typical output:** same ideas, same order, different phrasing and cadence
### Respond
Treat your input as a prompt and produce the character's reply — as if
you'd said it *to* them.
- **When to use:** spoken-input agents; Q&A with a specific voice;
interactive character experiences
- **Temperature:** balanced — creative but grounded
- **Typical output:** a reply to your prompt, written in-character
## Speech-only framing
All three modes enforce **speech-only** output. The LLM is prompted to
Both modes enforce **speech-only** output. The LLM is prompted to
produce things a person would actually say out loud — no narration, no
action tags (`*sighs*`, `[laughs]`), no meta-commentary, no markdown
formatting, no stage directions.
@@ -109,57 +101,58 @@ Pick a size in **Settings → Captures → Refinement → Refinement model** —
personality modes reuse it. If you switch models, both refinement and
personality output pick up the change on the next call.
## Using the modes
## Using the controls
The three actions appear as buttons on the profile when a personality is
set. For each:
Both controls appear on the floating generate box when the selected profile
has a personality set.
<Steps>
<Step title="Type or dictate your input">
Rewrite and Respond need input text. Compose doesn't.
<Step title="Compose — shuffle a line">
Click the shuffle button. The LLM runs and the result fills the generate
textarea. Edit if you want, then hit generate.
</Step>
<Step title="Click the mode">
The LLM runs, then the result fills the generate box.
</Step>
<Step title="Hit generate">
The TTS engine speaks the LLM output in the profile's voice. The result
lands in generation history as a normal generation.
<Step title="Speak in character — toggle persona rewrite">
Type (or dictate) what you want said. Flip the wand toggle on. Hit
generate — Voicebox runs the text through the personality LLM first,
then TTS speaks the rewritten version. Leave the toggle off for plain
TTS.
</Step>
</Steps>
Each button also has an inline regenerate affordance — click again to
resample. Compose will give you something totally different; Rewrite and
Respond will give you a variation on the same content.
Compose always gives you something different on re-click. The persona
toggle, on the other hand, is a mode — it applies to every generate call
until you flip it back off.
## Use cases
- **Agents that speak in a voice you own.** Combine Respond with the
built-in [MCP Server](/overview/mcp-server) so Claude Code, Cursor,
- **Agents that speak in a voice you own.** Combine the persona toggle with
the built-in [MCP Server](/overview/mcp-server) so Claude Code, Cursor,
Cline, or any MCP-aware agent can talk back through a profile with a
personality. The agent calls `voicebox.speak({ text, profile, intent:
"respond" })` and Voicebox produces in-character speech in your cloned
voice.
personality. The agent calls `voicebox.speak({ text, profile, personality:
true })` and Voicebox rewrites the text in character before speaking.
- **Interactive characters.** Games, narrative tools, accessibility
experiences. A character with a personality description plus a cloned
voice becomes a reusable prop.
- **Accessibility.** People who can't speak in their original voice can
keep a personality description of how they used to sound and use Rewrite
to turn typed input into in-character speech.
- **Creative drafting.** Write a plain outline, Rewrite line-by-line into
the character's voice, drop the audio into a Story.
keep a personality description of how they used to sound and use the
rewrite toggle to turn typed input into in-character speech.
- **Creative drafting.** Write a plain outline, flip the persona toggle,
generate line-by-line into the character's voice, drop the audio into a
Story.
## API surface
Personalities and the three modes are accessible via REST:
Personalities are accessible via REST:
| Method | Endpoint | Body |
|---|---|---|
| `PUT` | `/profiles/{id}` | Include a `personality` field up to 2,000 chars to set it. |
| `POST` | `/profiles/{id}/speak` | Runs the LLM + TTS in one shot. Body includes `text`, `intent` (`compose`, `rewrite`, or `respond`), optional `engine`, `language`. |
| `POST` | `/profiles/{id}/compose` | No body. Returns a fresh in-character utterance as text. |
| `POST` | `/generate` | Include `personality: true` to run input text through the personality LLM before TTS. Same for `POST /speak`. |
The `/profiles/{id}/speak` endpoint is the same primitive MCP's
`voicebox.speak` tool calls when that ships. Scripts and agents can use it
directly today.
`POST /generate` with `personality: true` is the same primitive MCP's
`voicebox.speak` tool uses when you pass `personality: true`. Scripts and
agents can use it directly.
## Limits and gotchas