voicebox/mobile/PLAN.md

# Voicebox Mobile — V1 plan

A companion app for the Voicebox desktop. iPhone-first, dictate-anywhere, Captures as the hero. Talks to a paired desktop over Tailscale or LAN with end-to-end encryption.

V1 is entirely local — no cloud, no account. The device key minted during pairing is the root of the user's lifetime encryption identity and gets reused by the cloud phases that follow; see [`docs/plans/CLOUD_ROADMAP.md`](../docs/plans/CLOUD_ROADMAP.md) for the post-mobile arc (backup & sync → private inference → marketplace).

---

## Repo layout

- New `mobile/` at repo root, sibling to `app/`, `backend/`, `tauri/`, `landing/`, `web/`
- Standalone Expo project — not a Bun workspace member (avoids React/Tauri/Bun version drag)
- Type-sharing via OpenAPI: generate `mobile/src/api/types.ts` from backend's `/openapi.json`, commit it, regenerate on demand
- Branch off `main` once 0.5.0 ships → `feat/mobile-app`

## Stack (verified 2026-04-25)

### The SDK 54 vs SDK 55 fork

Expo SDK 55 is the current `latest` (`[email protected]`, React Native 0.83, React 19.2, New Architecture only — Legacy Architecture was dropped in 55). It also ships Expo Router v7, Hermes v1 with bytecode diffing, and `expo-brownfield`. But: **NativeWind v5 is the only NativeWind line that targets SDK 55, and v5 is still pre-release** (`5.0.0-preview.3`, with explicit "not intended for production use" warning in the v5 docs). NativeWind v4 stable (`4.2.3`) is paired with SDK 54.

Two real options:

- **Option A — Ship-fast (recommended for V1):** Expo SDK 54 + NativeWind v4.2.3. Both stable, official pairing, well-documented. Loses SDK 55's bytecode-diff updates and Expo Router v7 sugar but everything works.
- **Option B — Cutting-edge:** Expo SDK 55.0.17 + NativeWind 5.0.0-preview.3. Latest everything, but the NativeWind v5 maintainer explicitly says it's for experimentation. More breakage during dev, brittle CI.

Recommendation: **Option A**. We're trying to ship a companion app, not stress-test pre-release styling layers. We can bump to SDK 55 + NativeWind v5 once both stabilize (Expo targets stable SDK 55 mid-2026).

### Pinned versions (assuming Option A)

| Package | Version | Notes |
| --- | --- | --- |
| `expo` | `~54` (latest 54.x) | New Architecture default since SDK 51 |
| `expo-router` | bundled with SDK | file-based, typed routes |
| `nativewind` | `4.2.3` | Tailwind class parity with `app/` |
| `expo-audio` | `~54` (bundled with SDK) | stable in SDK 54+, replaces `expo-av` recording |
| `@shopify/react-native-skia` | `2.6.2` | live waveform; WaveSurfer is DOM-only |
| `expo-camera` | `~54` (bundled) | QR scan (barcode scanning built in) |
| `expo-secure-store` | `~54` (bundled) | paired device key |
| `react-native-reanimated` | `4.3.0` | rewritten for new arch |
| `react-native-gesture-handler` | `2.31.1` | |
| `zustand` | `5.0.12` | mirrors desktop |
| `@tanstack/react-query` | `5.100.5` | mirrors desktop |

(Versions for `expo-*` packages are managed by `npx expo install`, which picks the patch that matches the SDK — don't pin them by hand.)

### Other stack decisions

- **TypeScript strict**
- Theme tokens copied straight from `app/`'s shadcn theme (`hsl(43 60% 50%)` for the gold accent, dark surfaces match)
- **EAS Build** + **Dev Client** from day one — Skia and SecureStore push us off Expo Go

## Pairing & transport — Tailscale-friendly

1. Desktop: new **Settings → Mobile** with "Pair device" → renders QR + 6-digit fallback
2. QR payload: `voicebox://pair?host=<url>&secret=<b64>&fp=<sha256>`
   - `host` = whatever address the user can reach: LAN IP (`192.168.x.x:17493`), Tailscale 100.x address, or MagicDNS name (`mac.tail-xxxx.ts.net:17493`) — Tailscale Just Works with zero extra code
   - `secret` = one-time pairing token; mobile exchanges it for a long-lived device key on first request
   - `fp` = self-signed cert fingerprint we mint at pair time, pinned on mobile
3. Auth: bearer token + XChaCha20-Poly1305 payload encryption with HKDF per-session keys — E2E layer above HTTP, survives any future cloud relay swap

## Backend additions (desktop, separate PR before mobile work)

- `POST /pair/init` — mint pairing token, return QR payload
- `POST /pair/complete` — exchange token for device-bound long-lived key, persist `paired_devices` row
- `GET/DELETE /devices` — Settings → Mobile lists & revokes paired devices, with `last_seen_at`
- Bearer middleware on `/generate`, `/transcribe`, `/profiles`, `/captures`, `/speak` (loopback callers stay unauthenticated as today; paired-device callers use the bearer)
- `/captures` upload accepts `m4a` (expo-audio's iOS default) in addition to existing formats

## Screens

### First-run

Pair flow: scan QR → confirm desktop name + fingerprint → store creds in SecureStore.

### Tab 1 — Generate

- Profile cards horizontal scroll (top)
- Recent generations list (middle) — tap to play, long-press for version picker / regenerate / share
- Floating generate box (bottom): text input + engine indicator + speak button

### Tab 2 — Voices

- List of profiles (cloned + presets), grouped by engine compatibility
- Tap to inspect: samples (Skia waveform thumbs), language, last used
- **No** profile creation in V1

### Tab 3 — Captures (the hero)

- **Big gold mic button** bottom-front-and-center — same accent as the sponsor CTA
- Tap-to-toggle in V1 (push-to-talk fights iOS gestures, defer)
- Live mic waveform via Skia, amplitude polled at 30 Hz, scrolling buffer
- Transcript text area above the waveform — empty during recording, populated on stop
- State pill at top: `recording → uploading → transcribing → refining → done` (mirrors desktop pill semantics)
- Captures list below: each row has a Skia mini-waveform, tap to expand, scrub, edit transcript inline, "Play as voice profile"
- **Audio preserved + downloadable** — visually obvious in the UI; this is the USP
- Schema/UI built so resumable capture can land later without re-architecting (no "one capture = one continuous take" assumptions baked in)

## Out of scope for V1 (deliberately)

Stories editor · Voice profile creation / sample recording · Effects editor · Personality LLM controls · Streaming transcription (lands with resumable capture in V2) · Settings beyond pairing · Android (pipeline supports it; QA focus iOS first)

## V2+ candidates

- **Resumable capture** — pause/resume that actually appends audio AND transcript (Apple Voice Notes drops the second-half transcript on resume; Voicebox shouldn't)
- **Document mode** — refinement LLM writes/edits a markdown document live as you speak, including dictated edits ("change the second bullet to…")
- **Streaming transcription** — partial transcripts during recording
- **Voice profile creation on mobile** — record samples directly
- **Android**
- **Cloud relay** — once the Voicebox platform exists, the same E2E layer rides over a relay so pairing isn't tied to Tailscale/LAN

## Build/dev workflow

```bash
cd mobile
bun install
bunx expo prebuild           # generate native projects (Skia + SecureStore need it)
bunx expo run:ios            # device on the same Tailnet
eas build --profile development   # distributable Dev Client for TestFlight
```

Bundle ID: `sh.voicebox.mobile` (need to register in App Store Connect).

## Order of attack

1. Backend: pairing endpoints + bearer middleware (desktop PR, can land before mobile)
2. Mobile: Expo scaffold + NativeWind + theme tokens + Pair screen
3. Mobile: Captures tab end-to-end — validates transport, E2E layer, audio upload, waveform, pill states all in one flow
4. Mobile: Generate tab
5. Mobile: Voices tab
6. EAS dev build + TestFlight internal track

## Open questions

1. **iOS only V1?** Default: yes.
2. **NativeWind or hand-rolled styles?** Default: NativeWind — class parity with `app/` is worth the small bundler tax.
3. **Crib patterns from the Spacedrive mobile app first**, or start clean from current Expo docs?
4. **Bundle ID + display name** — confirm `sh.voicebox.mobile` / "Voicebox" before EAS setup.
5. **Pair screen UX** — QR-only, or always offer the 6-digit code as an a11y/fallback?