progress

2026-09-16 05:10:42 -07:00 · 2026-04-23 04:08:51 -07:00
parent be73a33ee1
commit 7c50e189cb
7 changed files with 269 additions and 12 deletions
@@ -9,11 +9,9 @@

 ## [0.5.0] - 2026-04-22

-**The Capture release.** Voicebox stops being a voice-cloning studio and becomes an AI voice studio. The loop closes in both directions: your voice goes into your computer through a global hotkey, and any agent's voice comes out of your computer through a voice you own.
+**The Capture release.** Voicebox stops being just a voice-cloning studio and becomes a full AI voice studio. The loop closes in both directions: your voice goes into your computer through a global hotkey, and any agent's voice comes out of your computer through a voice you own.

-Hold a key anywhere on your machine, speak, release — the transcript lands in the focused text field in whatever app you were using. Flip the primitive around and any MCP-aware agent — Claude Code, Cursor, Cline, Spacebot — speaks back through the same on-screen pill in one of your cloned voices. A local LLM sits between the two, so transcripts come out clean and voice profiles can carry a personality that reshapes what the agent actually says before it gets spoken.
-
-Everything still runs on your hardware. No cloud, no accounts, no audio leaving the machine.
+When enabled, hold a key anywhere on your machine, speak, release — the transcript lands in the focused text field in whatever app you were using. Flip the primitive around and any MCP-aware agent — Claude Code, Cursor, Cline, Spacebot — speaks back through the same on-screen pill in one of your cloned voices. A local LLM sits between the two, so transcripts come out clean and voice profiles can carry a personality that reshapes what the agent actually says before it gets spoken.

 ### Dictation — speak anywhere, paste anywhere

@@ -36,9 +34,25 @@ Temperatures are tuned per mode (compose hot for variety, rewrite cold for fidel

 ### Agents — any MCP-aware agent gets a voice

-> **Note:** the MCP server implementation is not yet in this tag. The pieces it depends on — the personality/compose/rewrite/respond runtime, voice-binding-per-profile, and the on-screen pill as a generic surface — are all in place. The MCP route lands in a follow-up commit before the final 0.5.0 tag.
+Voicebox ships a built-in **Model Context Protocol** server at `http://127.0.0.1:17493/mcp` so Claude Code, Cursor, Windsurf, Cline, VS Code MCP extensions — any MCP-aware agent — can call into your local Voicebox install. Four tools ship with dotted names:

-Once the MCP server ships, any MCP-aware agent — Claude Code, Cursor, Cline, Spacebot — can call `voicebox.speak({ profile, text, intent })` and Voicebox will produce in-character speech in the bound voice. The on-screen pill surfaces whenever an agent is talking so you always see what's coming out of your machine. Pin Claude Code to Morgan, Cursor to Scarlett, Spacebot to its own voice — you can tell which agent is speaking without looking.
+- **`voicebox.speak`** — speak text in any voice profile, with optional `intent: compose | rewrite | respond` to run through the profile's personality LLM first
+- **`voicebox.transcribe`** — Whisper transcription of a base64 blob or an absolute local path
+- **`voicebox.list_captures`** — recent captures with their transcripts
+- **`voicebox.list_profiles`** — available voice profiles (cloned + preset)
+
+- **Streamable HTTP as primary transport.** FastMCP mounted inside the existing FastAPI process (Nov-2025 MCP spec, post-SSE). Cursor / Windsurf / VS Code / Claude Code all support it out of the box — drop a `mcpServers` block with the URL and an `X-Voicebox-Client-Id` header.
+- **Stdio shim for clients that don't speak HTTP MCP.** A `voicebox-mcp` binary ships inside the app bundle as a Tauri sidecar — ~18 MB PyInstaller build with torch / transformers / mlx explicitly excluded. The Settings page renders the install snippet with the right absolute path pre-filled.
+- **Per-client voice binding.** Pin Claude Code to Morgan, Cursor to Scarlett, Cline to its own voice — the `X-Voicebox-Client-Id` header resolves to a bound voice whenever `speak` is called without an explicit `profile`. Managed in **Settings → MCP**, with an auto-stamped `last_seen_at` timestamp on each row so you can tell the install actually took. New clients register their binding row on first call.
+- **Profile resolution precedence.** Explicit `profile` arg (name or id, case-insensitive) → per-client binding → global default from `capture_settings.default_playback_voice_id` → error with a pointer to Settings.
+- **Speaking pill.** Agent-initiated speech surfaces the same on-screen pill as dictation, in a new `speaking` state with the profile name and an elapsed timer. Driven by SSE at `/events/speak`; a Tauri-side `dictate:show` handler repositions and reveals the pill over the current monitor even when the main Voicebox window is hidden. Silent background TTS is a trust hazard — the pill always shows what's coming out of your machine.
+- **`POST /speak` REST wrapper.** Same code path and voice resolution for shell scripts, ACP, A2A, GitHub Actions, or anything else that isn't MCP-native.
+
+**Claude Code one-liner:**
+
+```
+claude mcp add voicebox --transport http --url http://127.0.0.1:17493/mcp --header "X-Voicebox-Client-Id: claude-code"
+```

 ### Refinement hardening

@@ -68,7 +82,7 @@ Settings → Captures is now the home for the whole dictation flow:

 ### Landing page — /capture

-New `/capture` route tells the Capture story end-to-end. Hero line: *"Just talk to your computer."* Three pillars — multi-engine STT (Whisper, Whisper Turbo, Parakeet v3, Qwen3-ASR), refined transcripts, agents speaking in voices you own. Drop-in MCP config block for Claude Code / Cursor / Cline. Agent-integration section showing per-client voice binding.
+New `/capture` route tells the Capture story end-to-end. Hero line: *"Just talk to your computer."* Three pillars — STT (Whisper, Whisper Turbo), refined transcripts, agents speaking in voices you own. Drop-in MCP config block for Claude Code / Cursor / Cline. Agent-integration section showing per-client voice binding.

 ## [0.4.5] - 2026-04-22

@@ -394,11 +394,11 @@ export function CapturesTab() {
        <div className="absolute top-0 left-0 right-0 h-20 bg-gradient-to-b from-background to-transparent z-10 pointer-events-none" />

        <div className="absolute top-0 left-0 right-0 z-20 pl-4 pr-4">
-          <div className="flex items-center gap-2 mb-5">
+          <div className="flex items-center mb-3">
            <h1 className="text-2xl px-4 font-bold">Captures</h1>
            <Badge
              variant="secondary"
-              className="h-5 px-1.5 text-[10px] font-medium text-accent bg-accent/10 border border-accent/20"
+              className="h-5 px-1.5 -ml-2 text-[10px] font-medium text-accent bg-accent/10 border border-accent/20"
            >
              Beta
            </Badge>
@@ -488,7 +488,7 @@ export function CapturesTab() {
        <div className="absolute top-0 left-0 right-0 z-20 px-8">
          <div className="flex items-center gap-3 py-4">
            <div className="flex items-center gap-1.5 text-xs text-muted-foreground">
-              <span className="w-1.5 h-1.5 rounded-full bg-emerald-500" />
+              <span className="w-1.5 h-1.5 rounded-full bg-accent" />
              <span>
                Whisper {sttModel.charAt(0).toUpperCase() + sttModel.slice(1)}
                <span className="mx-1.5 text-muted-foreground/40">·</span>
@@ -54,7 +54,8 @@ function ChecklistRow({ icon, title, description, ready, action }: RowProps) {

 function progressPercent(task: ActiveDownloadTask | undefined): number | null {
  if (!task) return null;
-  if (typeof task.progress === 'number') return Math.max(0, Math.min(100, task.progress));
+  if (typeof task.progress === 'number')
+    return Math.round(Math.max(0, Math.min(100, task.progress)));
  if (task.current && task.total) return Math.round((task.current / task.total) * 100);
  return null;
 }
@@ -54,6 +54,7 @@ def _profile_to_response(
        preset_voice_id=getattr(profile, "preset_voice_id", None),
        design_prompt=getattr(profile, "design_prompt", None),
        default_engine=getattr(profile, "default_engine", None),
+        personality=getattr(profile, "personality", None),
        generation_count=generation_count,
        sample_count=sample_count,
        created_at=profile.created_at,
@@ -181,6 +182,7 @@ async def create_profile(
        preset_voice_id=data.preset_voice_id,
        design_prompt=data.design_prompt,
        default_engine=default_engine,
+        personality=data.personality,
        created_at=datetime.utcnow(),
        updated_at=datetime.utcnow(),
    )
@@ -398,6 +400,7 @@ async def update_profile(
    profile.name = data.name
    profile.description = data.description
    profile.language = data.language
+    profile.personality = data.personality
    if data.default_engine is not None:
        profile.default_engine = data.default_engine or None  # empty string → NULL
    profile.updated_at = datetime.utcnow()
@@ -9,6 +9,7 @@ import {ControlUI} from "@/components/ControlUI";
 import {Features} from "@/components/Features";
 import {Footer} from "@/components/Footer";
 import {Navbar} from "@/components/Navbar";
+import {Personalities} from "@/components/Personalities";
 import {AppleIcon, LinuxIcon, WindowsIcon} from "@/components/PlatformIcons";
 import {SupportedModels} from "@/components/SupportedModels";
 import {TutorialsSection} from "@/components/TutorialsSection";
@@ -141,6 +142,9 @@ export default function Home() {
 			{/* ── Agent integration (speak primitive + MCP) ───────────── */}
 			<AgentIntegration />

+			{/* ── Personalities (Compose / Rewrite / Respond) ──────────── */}
+			<Personalities />
+
 			{/* ── API Section ──────────────────────────────────────────── */}
 			<ApiSection />

@@ -58,7 +58,7 @@ export function Navbar() {
            className="flex items-center gap-1.5 rounded-md px-3 py-1.5 text-sm font-medium text-muted-foreground transition-colors hover:text-foreground"
          >
            Capture
-            <span className="rounded-full bg-accent/15 px-1.5 py-0.5 text-[9px] font-semibold uppercase tracking-wider text-accent">
+            <span className="rounded-full bg-accent/15 px-1.5 text-[9px] font-semibold uppercase tracking-wider text-accent">
              New
            </span>
          </a>
@@ -0,0 +1,235 @@
+'use client';
+
+import { AnimatePresence, motion } from 'framer-motion';
+import { ArrowRight, MessageSquareReply, PenLine, Sparkles } from 'lucide-react';
+import { useEffect, useState } from 'react';
+
+// ─── Modes ──────────────────────────────────────────────────────────────────
+
+type Mode = {
+  id: 'compose' | 'rewrite' | 'respond';
+  label: string;
+  icon: typeof Sparkles;
+  inputLabel: string;
+  outputLabel: string;
+  input: string;
+  output: string;
+};
+
+const MODES: Mode[] = [
+  {
+    id: 'compose',
+    label: 'Compose',
+    icon: Sparkles,
+    inputLabel: 'Prompt',
+    outputLabel: "Marlowe, in character",
+    input: 'celebrate the deploy going green',
+    output:
+      "She came through clean. Not a single test casting a shadow. In this town, that's usually when you start worrying.",
+  },
+  {
+    id: 'rewrite',
+    label: 'Rewrite',
+    icon: PenLine,
+    inputLabel: 'Your text',
+    outputLabel: "Marlowe, in character",
+    input: 'the build is done and we shipped to production',
+    output:
+      "Build's wrapped, ship's left the dock. Another stack of code makes its way into prod, another row of green checks lining the wall.",
+  },
+  {
+    id: 'respond',
+    label: 'Respond',
+    icon: MessageSquareReply,
+    inputLabel: 'Question',
+    outputLabel: "Marlowe, in character",
+    input: 'should I refactor this before merging or do it after?',
+    output:
+      "Listen, kid. You can polish the brass on the door, or you can open it. Open the door — refactor in daylight.",
+  },
+];
+
+const PERSONA_DESCRIPTION =
+  "1940s noir detective. World-weary, cynical, every situation a metaphor for the city's underbelly. Talks like he's seen one stack trace too many.";
+
+// ─── Persona card ───────────────────────────────────────────────────────────
+
+function PersonaCard() {
+  return (
+    <div className="rounded-xl border border-app-line bg-app-darkBox p-5 shadow-[0_20px_60px_rgba(0,0,0,0.35)]">
+      <div className="flex items-center gap-3 mb-4">
+        <div
+          className="h-12 w-12 rounded-full shrink-0 ring-1 ring-white/10"
+          style={{ background: 'linear-gradient(135deg, #dc2626, #7f1d1d)' }}
+        />
+        <div className="min-w-0">
+          <div className="text-[15px] font-semibold text-foreground leading-tight">Marlowe</div>
+          <div className="text-[11px] text-muted-foreground/80 leading-tight mt-0.5">
+            Voice profile · cloned from a 12s sample
+          </div>
+        </div>
+      </div>
+
+      <div className="mb-1.5 text-[10px] font-mono uppercase tracking-[0.2em] text-ink-faint/70">
+        Personality
+      </div>
+      <p className="text-[13px] leading-relaxed text-ink-dull italic">&ldquo;{PERSONA_DESCRIPTION}&rdquo;</p>
+    </div>
+  );
+}
+
+// ─── Mode demo ──────────────────────────────────────────────────────────────
+
+function ModeDemo({ mode, cycleKey }: { mode: Mode; cycleKey: number }) {
+  return (
+    <div className="rounded-xl border border-app-line bg-app-darkerBox overflow-hidden flex flex-col flex-1">
+      {/* Mode tabs */}
+      <div className="flex items-center gap-1 p-1.5 border-b border-app-line bg-app-darkBox/40">
+        {MODES.map((m) => {
+          const Icon = m.icon;
+          const active = m.id === mode.id;
+          return (
+            <div
+              key={m.id}
+              className={`flex items-center gap-1.5 h-8 px-3 rounded-md text-[12px] font-medium transition-colors ${
+                active
+                  ? 'bg-white/[0.07] text-foreground border border-white/[0.08]'
+                  : 'text-muted-foreground/60'
+              }`}
+            >
+              <Icon className="h-3.5 w-3.5" />
+              {m.label}
+            </div>
+          );
+        })}
+      </div>
+
+      {/* Input → Output */}
+      <div className="p-5 flex-1 flex flex-col">
+        <AnimatePresence mode="wait">
+          <motion.div
+            key={`${cycleKey}-${mode.id}`}
+            initial={{ opacity: 0, y: 4 }}
+            animate={{ opacity: 1, y: 0 }}
+            exit={{ opacity: 0, y: -4 }}
+            transition={{ duration: 0.25 }}
+            className="flex flex-col gap-4 flex-1"
+          >
+            {/* Input */}
+            <div>
+              <div className="text-[10px] font-mono uppercase tracking-[0.2em] text-ink-faint/70 mb-1.5">
+                {mode.inputLabel}
+              </div>
+              <div className="text-[13px] leading-relaxed text-ink-dull/90 font-mono bg-black/20 rounded-md border border-app-line/60 px-3 py-2.5">
+                {mode.input}
+              </div>
+            </div>
+
+            {/* Arrow */}
+            <div className="flex items-center justify-center gap-2 text-[10px] font-mono uppercase tracking-[0.2em] text-ink-faint/50">
+              <span>In character</span>
+              <ArrowRight className="h-3 w-3" />
+            </div>
+
+            {/* Output */}
+            <div>
+              <div className="text-[10px] font-mono uppercase tracking-[0.2em] text-accent mb-1.5">
+                {mode.outputLabel}
+              </div>
+              <div className="text-[14px] leading-relaxed text-foreground bg-accent/[0.06] rounded-md border border-accent/20 px-3 py-2.5 italic">
+                &ldquo;{mode.output}&rdquo;
+              </div>
+            </div>
+          </motion.div>
+        </AnimatePresence>
+      </div>
+    </div>
+  );
+}
+
+// ─── Bullets ────────────────────────────────────────────────────────────────
+
+const BULLETS = [
+  {
+    icon: Sparkles,
+    title: 'Compose',
+    description:
+      'Generate a fresh utterance in the character’s voice from a short prompt. Useful for game dialogue, narration cues, or character barks.',
+  },
+  {
+    icon: PenLine,
+    title: 'Rewrite',
+    description:
+      'Restate your text in their voice while preserving every idea. Same content, their delivery — for scripts, dubs, and consistent character voice across long-form work.',
+  },
+  {
+    icon: MessageSquareReply,
+    title: 'Respond',
+    description:
+      'Treat your text as a prompt and produce the character’s reply. The persona half of the dictation → speak loop.',
+  },
+];
+
+// ─── Section ────────────────────────────────────────────────────────────────
+
+export function Personalities() {
+  const [idx, setIdx] = useState(0);
+
+  useEffect(() => {
+    const iv = window.setInterval(() => {
+      setIdx((i) => (i + 1) % MODES.length);
+    }, 4500);
+    return () => window.clearInterval(iv);
+  }, []);
+
+  const mode = MODES[idx];
+
+  return (
+    <section id="personalities" className="border-t border-border py-24">
+      <div className="mx-auto max-w-6xl px-6">
+        {/* Header */}
+        <div className="max-w-3xl mx-auto text-center mb-14">
+          <div className="text-[11px] font-semibold uppercase tracking-[0.22em] text-accent mb-4">
+            Personalities
+          </div>
+          <h2 className="text-4xl md:text-5xl font-semibold tracking-tight text-foreground mb-5">
+            Voices with a personality.
+          </h2>
+          <p className="text-muted-foreground text-base md:text-lg leading-relaxed">
+            Give any voice profile a free-form personality. Then{' '}
+            <b className="text-foreground/90">Compose</b>,{' '}
+            <b className="text-foreground/90">Rewrite</b>, or{' '}
+            <b className="text-foreground/90">Respond</b> — your cloned voice, in full character.
+          </p>
+        </div>
+
+        {/* Mockup: persona card (left) + mode demo (right) */}
+        <div className="grid md:grid-cols-[340px_1fr] gap-6 mb-12 items-stretch">
+          <PersonaCard />
+          <ModeDemo mode={mode} cycleKey={idx} />
+        </div>
+
+        {/* Bullets */}
+        <div className="grid md:grid-cols-3 gap-6">
+          {BULLETS.map((bullet) => {
+            const Icon = bullet.icon;
+            return (
+              <div
+                key={bullet.title}
+                className="rounded-xl border border-border bg-card/40 backdrop-blur-sm p-5"
+              >
+                <div className="flex items-center gap-2 mb-2">
+                  <Icon className="h-4 w-4 text-accent" />
+                  <h3 className="text-[14px] font-semibold text-foreground">{bullet.title}</h3>
+                </div>
+                <p className="text-[13px] leading-relaxed text-muted-foreground">
+                  {bullet.description}
+                </p>
+              </div>
+            );
+          })}
+        </div>
+      </div>
+    </section>
+  );
+}