From 41ee04489924d9aeb556077615ed8b75108af499 Mon Sep 17 00:00:00 2001 From: Labyricorn Date: Fri, 17 Jul 2026 17:04:36 -0700 Subject: [PATCH] Document normative provenance architecture --- IMPLEMENTATION_STATUS.md | 2 + README.md | 6 + docs/provenance/COMPLETION-CHECKLIST.md | 244 +++++++++ docs/provenance/README.md | 9 + .../STAGE-1-NORMATIVE-SPECIFICATION.md | 481 ++++++++++++++++++ docs/provenance/STATE-MACHINES.md | 254 +++++++++ 6 files changed, 996 insertions(+) create mode 100644 docs/provenance/COMPLETION-CHECKLIST.md create mode 100644 docs/provenance/README.md create mode 100644 docs/provenance/STAGE-1-NORMATIVE-SPECIFICATION.md create mode 100644 docs/provenance/STATE-MACHINES.md diff --git a/IMPLEMENTATION_STATUS.md b/IMPLEMENTATION_STATUS.md index e46acb7..3145193 100644 --- a/IMPLEMENTATION_STATUS.md +++ b/IMPLEMENTATION_STATUS.md @@ -26,6 +26,8 @@ These items cannot be honestly certified from this workspace alone and need proj ## Remaining hardening +- Replace the current MVP provenance persistence path with the approved [Thinkloom 1.0 Stage 1 normative provenance subsystem](docs/provenance/README.md). The specification is complete; formal schemas, fixtures, native implementation, and fault-injection conformance remain future stages. + - Run microphone/VAD/transcription integration tests once the approved model assets are supplied. - Run signed clean-install, upgrade, downgrade-warning, and uninstall tests on each release platform. - Run full 20,000-word interaction and fault-injection profiling on packaged release hardware. diff --git a/README.md b/README.md index 4ae4664..19d5584 100644 --- a/README.md +++ b/README.md @@ -84,6 +84,12 @@ Every language-model prompt is editable JSON. Thinkloom creates documented user See [PROMPTS.md](PROMPTS.md) for each file's effect, variables, editing workflow, validation behavior, and reset procedure. +## Provenance architecture + +The approved Thinkloom 1.0 provenance architecture is documented in the [Stage 1 normative provenance specification](docs/provenance/README.md). It defines the future single-writer subsystem, canonical records, segmented ledger, recovery protocol, retention policy, native verification, and release bindings. + +This specification is a forward implementation contract. The current 0.2.0 provenance implementation must not be represented as already conforming to it. + ## Provider setup Ollama defaults to `http://127.0.0.1:11434` and model `llama3.2`. OpenAI and compatible credentials are entered in Settings and saved through Windows Credential Manager, macOS Keychain, or Linux Secret Service. The first cloud operation in each project requires explicit approval. diff --git a/docs/provenance/COMPLETION-CHECKLIST.md b/docs/provenance/COMPLETION-CHECKLIST.md new file mode 100644 index 0000000..2620c7a --- /dev/null +++ b/docs/provenance/COMPLETION-CHECKLIST.md @@ -0,0 +1,244 @@ +# Thinkloom Stage 1 Completion Checklist + +Status: **Normative traceability and handoff gate** + +Stage 1 is complete only when every item below is represented without contradiction in the normative specification and state-machine companion. This checklist also defines the minimum Stage 2 and later implementation test handoff. + +## 1. Normative specification gate + +| Requirement | Normative location | Status | +|---|---|---| +| Correct tamper-evident claim and trust limitation | Specification §§3, 26 | Complete | +| Authority hierarchy | Specification §4 | Complete | +| Self-contained repository boundaries | Specification §5 | Complete | +| Stable IDs and contiguous event sequences | Specification §6 | Complete | +| Canonical repository paths | Specification §7.1 | Complete | +| UTC timestamp representation | Specification §7.2 | Complete | +| RFC 8785 plus Thinkloom canonicalization rules | Specification §8 | Complete | +| Exact self-hash exclusion requirement | Specification §8 | Complete | +| Immutable records and append-only revisions | Specification §9 | Complete | +| Text range and revision identity | Specification §10 | Complete | +| Meaningful edit transaction boundary | Specification §10 | Complete | +| Minimal/full retention semantics | Specification §11 | Complete | +| Sanitized export separation | Specification §11.3 | Complete | +| Prospective policy changes | Specification §11.4 | Complete | +| Encryption identity | Specification §11.5 | Complete | +| Protected-project portability and recovery | Specification §11.5 | Complete | +| Sensitive SQLite restriction | Specification §11.5 | Complete | +| Single native writer boundary | Specification §12 | Complete | +| Cross-store write-intent phases | Specification §12; State Machines §1 | Complete | +| Ledger authority over SQLite | Specification §§4, 12 | Complete | +| Idempotent `client_action_id` behavior | State Machines §1.2 | Complete | +| Segmented ledger and sealing rules | Specification §13; State Machines §5 | Complete | +| Provider I/O outside the writer lock | Specification §14; State Machines §3 | Complete | +| Stale-context handling | Specification §14; State Machines §3 | Complete | +| Native verification authority | Specification §15 | Complete | +| Verification findings/statuses and gates | Specification §15 | Complete | +| Historical index generator warning behavior | Specification §§15–16 | Complete | +| Deterministic derived-index rules | Specification §16 | Complete | +| Two-stage Git checkpoint without circularity | Specification §17; State Machines §6 | Complete | +| Frozen Git tree/index requirement | Specification §17; State Machines §6 | Complete | +| SQLite online backup | Specification §18 | Complete | +| Staged verified backup import | Specification §18; State Machines §8 | Complete | +| Non-self-referential release state machine | Specification §19; State Machines §7 | Complete | +| Bounded encrypted temporary output | Specification §20; State Machines §4 | Complete | +| Pre-durable-write secret filtering | Specification §21 | Complete | +| No audio retention | Specification §21 | Complete | +| Sanitization versus emergency purge | Specification §22; State Machines §10 | Complete | +| Explicit legacy preview-project policy | Specification §23 | Complete | +| Stage 2 formal schema inventory | Specification §24 | Complete | +| Migration deferred until after 1.0.0 | Specification §§23–24, 26 | Complete | + +## 2. Stage 2 schema and fixture handoff + +Stage 2 MUST NOT implement application behavior. It produces formal contracts and evidence that those contracts are deterministic. + +Required outputs: + +- JSON Schema Draft 2020-12 files listed in Specification §24. +- A schema catalog mapping schema IDs, versions, filenames, and compatible application versions. +- Valid fixture for every schema. +- Invalid fixtures covering each required property, enum, pattern, bound, path restriction, and additional-property policy. +- Canonical JSON vectors, including RFC 8785 and Thinkloom NFC preprocessing. +- Timestamp and repository-path vectors. +- Event hash and contiguous-sequence vectors. +- Cross-segment chain and sealed-manifest vectors. +- Self-digest identity vectors for templates, encrypted records, manifests, and other self-hashing objects. +- Minimal and full-private invocation fixtures. +- Sanitized-export omission fixtures. +- Encrypted-envelope, key-envelope, key-rotation, and recovery fixtures. +- Deterministic derived-index fixtures. +- Backup/release manifest and Merkle-root fixtures. +- Verification-report fixtures for every status and severity. + +Stage 2 must preserve the distinction between the provenance schema version and the Thinkloom application version. + +## 3. Durable-boundary fault-injection matrix + +The native implementation and test harness MUST support deterministic termination or injected failure after: + +- Write-intent creation +- First staged record write +- Each record flush/fsync +- Each authoritative atomic move +- Parent-directory durability step +- Ledger append before flush +- Ledger flush/fsync +- Chain-head temporary write +- Chain-head replacement +- SQLite operational domain update +- SQLite idempotency-result update +- Segment-manifest write +- Segment sealing move +- New active-segment creation +- Segment-opening event append +- Git source-tree capture +- Source checkpoint commit +- Checkpoint acknowledgment event +- Audit checkpoint commit +- Release source freeze +- Release manifest generation +- Release-file verification +- Release commit +- Release tag creation +- Backup snapshot completion +- Backup archive finalization +- Import extraction and each verification gate + +For each boundary, tests MUST prove one of: + +1. The operation is absent and safely retryable. +2. The operation is committed and idempotently discoverable. +3. Recovery deterministically completes it. +4. Recovery quarantines it without presenting false success. +5. Authoritative contradiction is reported and editing remains blocked. + +## 4. Concurrency and idempotency tests + +- Hundreds of concurrent frontend commands serialize without event loss or sequence gaps. +- Duplicate retries with the same canonical command return the original result. +- Reuse of `client_action_id` with a different command is rejected. +- Concurrent provider completions create distinct contiguous events. +- Provider calls do not hold the writer lock. +- Segment rotation cannot race an append. +- Checkpoint and release operations use frozen trees despite continuing edits. +- Clock rollback does not change event ordering. +- Stale OS lock artifacts do not block a project after process death. + +## 5. Recovery and integrity tests + +- Truncated active JSONL line +- Complete ledger event ahead of chain head +- Chain head ahead of ledger +- SQLite behind the ledger +- SQLite ahead of the ledger +- Missing or corrupt write-intent database +- Durable records without an event +- Event referencing a missing record +- Duplicate event sequence +- Event-sequence gap across a segment boundary +- Duplicate segment number +- Corrupted sealed segment or manifest +- Orphaned staging directory +- Stale or nondeterministic derived index +- Historical generator unavailable +- Unsupported authoritative schema +- Git unavailable or damaged while ledger remains valid +- Disk full and permission loss at every write phase +- Antivirus/synchronization lock during replacement + +## 6. Invocation and lineage tests + +- Successful local and cloud invocation +- Failed provider invocation +- Cancellation before and during streaming +- Partially streamed response +- Output spool limit exceeded +- Concurrent invocation spools and aggregate limit +- Crash with recoverable encrypted spool +- Spool key missing or corrupt +- Stale spool cleanup after the configured period +- No provider-resume claim for an unsupported provider +- Response completion after manuscript context changes +- Revalidation, rebase, explicit stale acceptance, and rejection paths +- Full, partial, and rejected disposition revisions +- Manual edit transactions after generated-text acceptance +- Transcript correction after downstream use +- Range verification across UTF-8, Unicode scalar, UTF-16, and editor coordinates + +## 7. Privacy, secret, and encryption tests + +- Minimal retention omits every prohibited raw field. +- Full private retention contains allowed records but no credentials. +- Minimal-to-full and full-to-minimal changes are prospective. +- Sanitized export does not mutate project storage. +- Sanitized export discloses every omission class. +- Secret detection occurs before filesystem, SQLite, spool, Git, log, and archive writes. +- Low-entropy redacted values are not exposed by guessable digests. +- No audio byte, path, filename, or content digest exists anywhere persistent. +- Protected records verify while locked at the ciphertext level. +- Authorized verification detects plaintext modification. +- Incorrect key and authenticated-metadata failures are detected. +- Wrapping-key rotation leaves record ciphertext and references unchanged. +- Recovery key re-entry and test unwrap are required before protection activation. +- Protected backup restores on a different device using only approved recovery material. +- Loss of both device and recovery access produces the documented unrecoverable state. +- Sensitive operational SQLite payloads are not left unprotected. +- Temporary cleanup is described and tested as cryptographic/logical deletion. + +## 8. Backup, import, and release tests + +- SQLite online backup during active editing +- SQLite snapshot integrity and manifest digest +- Backup file digest corruption +- Missing and extra manifest entries +- ZIP traversal, absolute/device path, symlink, case collision, duplicate entry, and decompression-bomb attempts +- Interrupted import at every staging/verification/activation phase +- No unverified file reaches an active destination +- Destination identity and conflict handling +- `VERIFIED_WITH_WARNINGS` security/non-security distinction +- `INCOMPLETE` import quarantine +- `FAILED` import block +- `UNSAFE` package rejection +- Release failure at every state transition +- Source commit/tree/chain-head consistency +- Release manifest self-reference exclusion +- Merkle ordering and path-normalization vectors +- Release commit and tag binding +- Missing optional binary produces warning without falsifying authoritative integrity + +## 9. Native verifier and UI tests + +- UI Verify History invokes the native verifier. +- Frontend cannot manufacture `VERIFIED` status. +- Finding severity maps correctly to overall status. +- Incremental verification never skips an altered authoritative record. +- Full verification is mandatory for release and import. +- Stored indexes are not trusted as verification inputs. +- A stale index is repairable without changing provenance. +- Locked encrypted evidence yields `INCOMPLETE`, not `FAILED`. +- Git-only damage yields warning when authoritative evidence remains valid. +- Missing authoritative records yield `FAILED`. +- Unsafe archive structure yields `UNSAFE`. + +## 10. Legacy 1.0 behavior tests + +- Known preview markers are detected without modifying the project. +- Normal opening and editing are refused. +- Show Project Folder remains available. +- Raw archival ZIP preserves the selected legacy tree without conversion. +- The archive is labeled as unverified and unconverted. +- No 1.0 provenance verification or evidence report is offered. +- Legacy Git history remains unchanged. +- No migration schema or implied migration success appears in 1.0. + +## 11. Stage 1 disposition + +Stage 1 is **complete** when: + +- The normative documents have passed editorial review. +- No older active project document is allowed to silently override their provenance mechanics. +- The Stage 2 schema work uses this checklist as its acceptance boundary. +- Any future architectural change is recorded as a versioned normative amendment rather than an informal implementation choice. + +Completion of Stage 1 does not claim implementation conformance and does not change the current application version. diff --git a/docs/provenance/README.md b/docs/provenance/README.md new file mode 100644 index 0000000..ba1fa8c --- /dev/null +++ b/docs/provenance/README.md @@ -0,0 +1,9 @@ +# Thinkloom provenance specification + +This directory contains the approved Stage 1 normative specification for the Thinkloom 1.0 provenance subsystem. + +- [Stage 1 Normative Specification](STAGE-1-NORMATIVE-SPECIFICATION.md) +- [State Machines and Recovery Protocols](STATE-MACHINES.md) +- [Stage 1 Completion Checklist](COMPLETION-CHECKLIST.md) + +These documents govern future schema and implementation work. They do not describe the current 0.2.0 implementation as already compliant. diff --git a/docs/provenance/STAGE-1-NORMATIVE-SPECIFICATION.md b/docs/provenance/STAGE-1-NORMATIVE-SPECIFICATION.md new file mode 100644 index 0000000..b1059f4 --- /dev/null +++ b/docs/provenance/STAGE-1-NORMATIVE-SPECIFICATION.md @@ -0,0 +1,481 @@ +# Thinkloom Stage 1 Normative Provenance Specification + +Status: **Approved architecture baseline for formal schema work** +Target: **Thinkloom 1.0.0** +Provenance schema family: **1.0** +Migration support: **Deferred until after Thinkloom 1.0.0** + +## 1. Purpose and precedence + +This specification defines the authority, persistence, integrity, privacy, recovery, verification, backup, and release contracts for Thinkloom provenance. + +It supersedes the provenance-specific transaction order, single-ledger layout, mutable-record assumptions, live-database snapshot method, and release-binding sequence in the earlier MVP architecture and implementation plans. It does not supersede their product requirements, native Tauri boundary, preview-first generation model, user-control requirements, accessibility requirements, or prohibition on retained audio. + +The current Thinkloom 0.2.0 implementation is not represented as conforming to this specification. Conformance begins only after the Stage 2 schemas, fixtures, native implementation, and required fault-injection tests are complete. + +## 2. Normative language + +The words **MUST**, **MUST NOT**, **REQUIRED**, **SHOULD**, **SHOULD NOT**, and **MAY** are normative. + +- **MUST/MUST NOT** identify a conformance requirement. +- **SHOULD/SHOULD NOT** identify a strong recommendation that requires a documented reason when not followed. +- **MAY** identifies permitted optional behavior. + +## 3. Correct product claim + +Thinkloom provenance is: + +> A local, transactionally coordinated, tamper-evident creative-process record with configurable retention, native verification, recoverable storage, and reproducible release manifests. + +Thinkloom MUST NOT claim that local provenance is tamper-proof, an independently trusted timestamp, conclusive legal proof, or a quantitative measure of human versus AI authorship. + +The strongest valid claim without an external anchor is: + +> The system can detect changes relative to a previously retained chain head, signed release, or external anchor. + +## 4. Authority hierarchy + +The following hierarchy is binding: + +1. **Immutable filesystem records and the provenance ledger** are authoritative evidence. +2. **Canonical publication files and manuscript revisions** are authoritative publication content when bound by ledger references. +3. **Release manifests** are authoritative bindings for a completed release. +4. **SQLite** stores operational state, UI state, write intents, idempotency indexes, and rebuildable query indexes. +5. **Git** stores meaningful milestone history and release state but is not the provenance authority. +6. **Derived indexes and generated reports** are disposable, reproducible caches or projections. + +SQLite MUST NOT be the only location of an evidentiary fact. A valid ledger MUST take precedence over contradictory SQLite state. Git failure MUST NOT invalidate an otherwise valid provenance ledger. + +## 5. Repository boundaries + +A conforming 1.0 project SHOULD organize authoritative and operational data under these boundaries: + +```text +publication-project/ +├── project.json +├── manuscript/ +├── ideas/ +├── records/ +│ ├── conversations/ +│ ├── invocations/ +│ ├── prompt-templates/ +│ ├── sources/ +│ └── transformations/ +├── provenance/ +│ ├── schema/ +│ ├── ledger/active/ +│ ├── ledger/sealed/ +│ ├── indexes/ +│ ├── integrity/ +│ └── report-config/ +├── releases/ +├── reports/ +├── assets/ +├── .app/ +│ ├── state.sqlite +│ ├── locks/ +│ ├── temp/ +│ └── recovery/ +└── .git/ +``` + +Operational `.app/` data, routine generated reports, temporary files, live database files, spools, and non-release exports MUST NOT be tracked in Git. Authoritative records, ledger segments, schemas, canonical publication content, release manifests, and retained release hashes MUST be tracked at meaningful checkpoints. + +Large PDFs, ZIP packages, and regenerable binaries SHOULD remain outside ordinary Git history. Their hashes and manifests MAY be tracked. + +## 6. Identifiers and event ordering + +Stable sortable identifiers SHOULD use ULIDs with type prefixes, including `event_`, `record_`, `intent_`, `turn_`, `session_`, `invocation_`, `revision_`, `fragment_`, `checkpoint_`, and `release_`. + +Identifiers MAY be allocated before an operation commits. Abandoned identifiers MUST NOT be reused. + +Committed `event_sequence` values MUST be contiguous across active and sealed ledger segments: + +- The first committed event sequence is defined by the project schema, normally `1`. +- A sequence is assigned only while holding the exclusive project writer lock immediately before ledger append. +- A write intent MUST NOT reserve an event sequence. +- The next sequence MUST be derived from the verified ledger head; SQLite MAY cache but MUST NOT authoritatively assign it. +- A missing or duplicated committed sequence is an authoritative integrity failure. + +## 7. Canonical paths and timestamps + +### 7.1 Repository paths + +Stored paths MUST: + +- Be relative to the project root. +- Use forward slashes. +- Contain no empty, `.` or `..` components. +- Contain no drive prefix, UNC prefix, URI scheme, NUL, or control character. +- Resolve within the project root after platform normalization. +- Be compared using a documented case policy appropriate to cross-platform verification. + +Schema work MUST define handling for Windows reserved names, trailing dots/spaces, Unicode-equivalent names, and case collisions. Unsafe or ambiguous paths MUST be rejected before a durable write or archive extraction. + +### 7.2 Timestamps + +Canonical timestamps MUST: + +- Use RFC 3339 UTC. +- Use exactly millisecond precision. +- End in `Z`. +- Use the form `2026-07-17T18:42:10.123Z`. + +Wall-clock time is descriptive, not independently trusted. Event sequence supplies authoritative local ordering. Clock rollback MUST NOT permit sequence rollback or event reordering. + +## 8. Canonical JSON and hashing + +All authoritative JSON MUST use one native canonicalization implementation shared by writing, hashing, verification, backup, release, and report generation. + +Before RFC 8785 JSON Canonicalization Scheme processing: + +- Text strings MUST be normalized to Unicode NFC. +- Text intended for provider submission MUST be normalized before submission so the retained message matches what was sent. +- Content MUST be encoded as UTF-8 without BOM. +- JSONL records MUST use LF delimiters. +- Undefined, NaN, and infinite values are prohibited. +- Integers MUST be used when floating-point representation is unnecessary. +- Schema-defined numeric bounds MUST be enforced before canonicalization. + +SHA-256 digest strings MUST use `sha256:` followed by 64 lowercase hexadecimal characters unless a schema explicitly separates algorithm and digest fields. + +Any record containing a hash of itself MUST define the exact canonical identity object and explicitly exclude the hash field. No implementation may infer hash exclusions informally. + +Formal Stage 2 fixtures MUST include RFC 8785 vectors plus Thinkloom vectors for NFC, timestamps, paths, numbers, JSONL boundaries, and cross-platform line endings. + +## 9. Record mutability + +The following records become immutable after creation: + +- Invocation request +- Raw provider response retained by policy +- Normalized provider response +- Invocation failure +- Raw transcript turn retained by policy +- Transcript correction and normalization revision +- Model configuration and capability snapshot +- Prompt-template ID/version +- Manuscript revision +- Idea revision +- Edit transaction +- Sealed ledger segment and segment manifest +- Release manifest +- Protected record envelope + +Mutable concepts MUST use immutable ordered revisions. `current.json` files and similar pointers MAY exist only as derived, non-authoritative conveniences. + +Every staged authoritative record MUST carry or be recoverably associated with `intent_id`, `client_action_id`, `project_id`, record ID, schema version, and record type. Recovery MUST be possible even when SQLite coordination data is unavailable or damaged. + +Corrections MUST NOT erase original transcript content when the selected retention policy preserves it. Dispositions MUST be revision records, not updates to a single authoritative file. + +## 10. Content and text identity + +Content references MUST identify a stable record or revision and its digest. A path alone is insufficient identity. + +Text-range references MUST include: + +- Document revision ID +- Coordinate system +- Start and end positions +- Exact preimage digest +- Stable fragment ID when available +- Source and destination revision IDs for transformations + +Supported coordinate systems MUST be explicit, such as `utf8_byte`, `unicode_scalar`, `utf16_code_unit`, or `editor_position`. Offsets are supporting metadata and MUST NOT be the sole long-term identity. + +Meaningful manual editing MUST be grouped into edit transactions rather than keystroke events. A transaction SHOULD close on focus loss, configured idle interval, section change, AI operation, checkpoint, phase change, document close, explicit save, or milestone. + +## 11. Retention, export, and encryption policies + +These are independent settings: + +```text +retention_mode: minimal | full_private +encryption_mode: none | protected +default_export_profile: full | sanitized +``` + +### 11.1 Minimal retention + +Minimal provenance is the REQUIRED default for Thinkloom 1.0. It retains final user-approved input, operation purpose, prompt-template identity/hash, input references/hashes, provider/model identity, accepted generated text, disposition metadata, manuscript lineage, checkpoints, and releases. + +It MUST NOT retain raw speech hypotheses, complete provider-facing prompts, complete supplied context, unaccepted raw model responses, or provider transport metadata. + +### 11.2 Full private retention + +Full private provenance MAY retain raw/corrected/normalized transcripts, actual provider-facing messages, supplied context, raw and normalized responses, parameters, provider metadata, rejected output, and correction history. Credentials and prohibited secrets remain excluded. + +### 11.3 Sanitized exports + +Sanitization is an export profile, not a storage mode. It MUST NOT mutate the project. A sanitized package MUST identify its source chain head, disclose omissions, and include the applied rule set or its hash. + +### 11.4 Policy changes + +Policy changes apply prospectively and MUST create provenance events. Minimal-to-full cannot reconstruct discarded content. Full-to-minimal does not remove previously retained content. Removal requires the separately confirmed emergency-purge process. + +### 11.5 Protected mode + +Protected mode MUST NOT ship until portability, recovery, key-loss behavior, and encrypted-backup restoration pass required tests. + +Protected records use stable random record IDs independent of ciphertext. Unprivileged verification checks record identity, authenticated envelope metadata, and ciphertext hashes. Authorized verification additionally decrypts and validates the canonical plaintext identity hash. + +The plaintext identity digest MUST hash exactly the canonical object containing `schema_version`, `record_id`, `record_type`, and `payload`, excluding the digest and all encryption, path, nonce, and envelope fields. + +The required key hierarchy is: + +```text +Project data-encryption key +├── device envelope protected by OS-vault material +└── portable recovery envelope protected by a high-entropy recovery key + or Argon2id-derived recovery key +``` + +Protected mode MUST require recovery-key verification and a successful test unwrap before activation. Portable backups MUST contain encrypted records, an encrypted SQLite snapshot, recovery envelope, and key metadata, but no plaintext project key or device-vault key. + +Operational SQLite MUST NOT become an unprotected duplicate of sensitive protected records. It MUST store references and non-sensitive indexes or use a separately specified protection mechanism for sensitive operational payloads. + +## 12. Single-writer and cross-store operation journal + +Every provenance mutation MUST pass through one native Rust provenance service. The frontend MUST NOT directly modify ledger segments, chain heads, immutable records, transcript revisions, invocation evidence, dispositions, verification results, or release manifests. + +The native service MUST use an exclusive per-project OS-managed writer lock. Persistent lock files alone are insufficient. Provider calls, transcription, Git operations, report generation, and other long-running external work MUST NOT hold this lock. + +Every mutation receives a stable `client_action_id`. Retrying the same action MUST return the original committed result rather than create another event. + +SQLite MUST maintain a `write_intents` operation journal with these phases: + +```text +PREPARED +RECORDS_DURABLE +LEDGER_APPENDED +CHAIN_HEAD_ADVANCED +SQLITE_APPLIED +COMPLETE +QUARANTINED +FAILED +``` + +The ledger is authoritative after a complete event has been durably appended. SQLite idempotency state is a rebuildable index and MUST be reconstructable from ledger events containing `client_action_id`. + +The binding write order and recovery transitions are defined in [STATE-MACHINES.md](STATE-MACHINES.md). + +## 13. Ledger segments + +The ledger MUST contain one active JSONL segment and zero or more sealed segments. A project writer lock serializes appends and rotation. + +A sealed segment manifest MUST bind: + +- Schema version +- Segment number +- Previous segment file digest +- First and final event digests +- First and final event sequences +- Event count +- Byte length +- Segment file digest +- Seal timestamp + +Sealed segments MUST never be altered automatically. A truncated final line MAY be removed from the active segment only when it was never referenced by the chain head. A complete valid event ahead of the chain head is recoverable by advancing the head after verification. + +Rotation MUST stop appends under the writer lock, flush and verify the active segment, create and flush its manifest, move both to sealed storage, create the next active segment, append a sequence-contiguous opening event linked to the prior final event, and advance the head. + +Default rotation triggers SHOULD include 10,000 events, 10 MiB, release finalization, and schema-boundary changes. Exact thresholds are configuration, not integrity semantics. + +## 14. Model invocation lifecycle + +A model invocation is a multi-operation state machine. Before contacting a provider, Thinkloom MUST record the immutable request and `MODEL_INVOCATION_REQUESTED` event under the writer lock, including context record/revision references, model configuration snapshot, prompt-template reference, and the manuscript/idea/conversation heads used. + +Provider I/O occurs outside the lock. Incomplete output MAY exist only in bounded application-managed temporary storage and MUST NOT become canonical evidence. + +After provider completion, cancellation, or failure, Thinkloom reacquires the lock, records an immutable response or failure, appends the corresponding event, and creates any staged preview. User acceptance or rejection is a separate provenance operation. + +If source revisions change while the model runs, the preview MAY be shown, but insertion MUST require revalidation, rebase, or explicit stale-context confirmation. Thinkloom MUST NOT silently insert generation based on stale context. + +## 15. Verification + +Native verification is authoritative. The UI MUST render native results and MUST NOT independently declare history valid. + +The verifier MUST check, as applicable: + +- Schema compatibility and canonical JSON +- Contiguous event sequences +- Event digests and previous-event links +- Cross-segment linkage and sealed manifests +- Referenced authoritative record existence and digests +- Active chain head +- Release manifests and release-file bindings +- Source Git commit and tag bindings +- Deterministic derived-index reproducibility +- Protected record envelopes, and plaintext when keys are available + +Finding severities are `INFO`, `WARNING`, `ERROR`, and `CRITICAL`. Overall statuses are: + +- `VERIFIED`: all authoritative evidence validates. +- `VERIFIED_WITH_WARNINGS`: authoritative evidence validates; a derived or optional component needs attention. +- `INCOMPLETE`: no contradiction was found, but authoritative verification could not finish. +- `FAILED`: authoritative evidence is inconsistent. +- `UNSAFE`: an import or external package violates security policy. + +An unavailable historical index generator is a warning, not an incomplete result. A stale index is repairable and MUST NOT invalidate authoritative evidence. + +Incremental verification MAY use a cached verified chain head. The cache is non-authoritative. Full verification is REQUIRED for release completion, backup import, explicit deep verification, and integrity recovery. + +Release and import gates MUST follow the approved status matrix: `INCOMPLETE`, `FAILED`, and `UNSAFE` block release; unverified imports never enter an active project destination. + +## 16. Derived indexes + +Derived indexes MUST be reproducible from authoritative records. Deterministic index content MUST use stable sorting, locale-independent comparison, canonical inputs, fixed schema/configuration, no random identifiers, and no current timestamp. + +An outer manifest MUST identify the source chain head, event count, generator name/version, configuration digest, and deterministic content digest. Volatile generation metadata MUST remain outside the hashed deterministic content. + +Verification rebuilds an index in temporary storage and compares canonical content digests. The stored index is never authoritative. + +Physical packing of immutable records is deferred. Content-addressed identity MAY be included in Stage 2 schemas, but authoritative paths MUST NOT be rewritten by an unimplemented packing process. + +## 17. Git checkpoints + +Git checkpointing occurs after critical provenance writes and outside the provenance writer lock. A checkpoint uses two stages to avoid a self-referential commit: + +1. Source commit `C1` binds the frozen publication tree and provenance head `H0`. +2. A `GIT_CHECKPOINT_CREATED` event acknowledges `C1`, producing head `H1`. +3. Audit commit `C2` contains the acknowledgment event and checkpoint manifest but is not referenced by that event. + +Git commits MUST be created from a frozen captured tree or isolated temporary Git index. They MUST NOT depend on a live working tree remaining unchanged after the provenance lock is released. + +A checkpoint is not user-visible as complete until the acknowledgment event exists. Restore uses publication content from `C1`; an application checkpoint ref MAY point to `C2`. Git failure produces a warning and recoverable checkpoint state, not fabricated completion. + +## 18. Backup, import, and SQLite snapshots + +Thinkloom MUST use SQLite's online backup API for a consistent snapshot. Copying live database, WAL, or SHM files is prohibited. + +A snapshot manifest MUST include SQLite version, database schema version, completion timestamp, file digest, project ID, and provenance chain head. + +Backup import MUST occur in application-controlled staging outside the destination. Before activation it MUST enforce archive path/count/size rules, verify the backup manifest and every listed digest, perform full native provenance verification, verify SQLite integrity, verify expected Git structure, confirm project identity, and detect destination conflicts. + +Only a verified project may be atomically moved into its destination. `UNSAFE` packages are rejected; `INCOMPLETE` packages remain quarantined; no unverified archive file may be placed into an active project directory. + +## 19. Release finalization + +Release finalization follows the state machine in [STATE-MACHINES.md](STATE-MACHINES.md). It MUST distinguish a frozen source commit from the later release commit. + +The release manifest binds the source commit, source chain head, source manuscript digest, release schema/application versions, sanitization state, and a formally defined release-files Merkle root. It MUST NOT contain the release commit digest. + +The release commit contains the generated package and manifest. The release tag points to that commit. Self-referential manifests and hash files MUST be excluded from their own flat digest lists or handled by the formally specified Merkle construction. + +Routine large release binaries SHOULD remain outside Git; tracked manifests and digests bind them. Release status transitions MUST be durable and recoverable. + +## 20. Temporary model output + +Temporary streaming output MUST use an application-private, non-Git, non-backup directory with restrictive permissions and encrypted per-invocation spool files. + +Defaults: + +```text +Maximum spool per invocation: 8 MiB +Hard configurable ceiling: 64 MiB +Maximum stream duration: 30 minutes +Maximum concurrent spools/project: 4 +Maximum aggregate spool/project: 128 MiB +Stale recovery period: 1 hour +``` + +Limits MUST be enforced before accepting additional bytes. On limit violation Thinkloom stops intake, attempts cancellation, records a bounded failure summary, and does not promote partial output unless the user explicitly preserves it under the retention policy. + +Spool keys MUST be wrapped by temporary-storage key material and retained only in protected operational state. Completion or cancellation removes the spool, wrapped temporary key, and directory. Cleanup MUST be described as cryptographic and logical deletion, not guaranteed physical overwrite. + +Temporary recovery MUST distinguish local recovery of bounded partial data from resumption of a provider stream; Thinkloom MUST NOT claim a provider request is resumable unless the provider contract supports it. + +## 21. Secret filtering and audio + +Credentials, authorization headers, cookies, signed credentials, and detected secrets MUST be filtered before the first durable filesystem, SQLite, Git, log, spool, or archive write. Redaction after Git persistence is not sufficient. + +Redaction actions MUST be disclosed rather than pretending omitted content never existed. A low-entropy secret MUST NOT be exposed through an unhashed or ordinary plaintext digest that permits trivial guessing. + +Audio retention remains prohibited. Project storage, provenance, Git, logs, reports, spools, and backups MUST contain no audio bytes, audio paths, or audio content digests. Permitted transcript retention depends on provenance policy. + +## 22. Purge semantics + +Sanitized export does not alter project history. Emergency purge is a separate destructive operation requiring strong confirmation. + +A purge MAY rewrite affected records, ledger hashes, and Git history only through a defined purge state machine. It MUST create a new chain root, record the superseded chain head when safe, disclose that integrity history was reconstituted, and warn that earlier copies and exports cannot be revoked. + +Ordinary editing MUST NOT invoke purge behavior. + +## 23. Legacy preview projects + +Thinkloom 1.0 MUST detect known preview/experimental project markers and refuse normal opening or editing. It MUST preserve the original project untouched, explain that migration is deferred, offer Show Project Folder, and permit a byte-preserving raw archival ZIP labeled: + +```text +Legacy project preservation archive +Not verified or converted by Thinkloom 1.0.0 +``` + +Thinkloom 1.0 MUST NOT import legacy records into schema 1.0, regenerate or verify their provenance under 1.0 rules, create a 1.0 evidence report, modify their Git history, or present them as migrated. Formal legacy reading, conversion, and migration begin after 1.0.0. + +## 24. Formal schema inventory for Stage 2 + +Stage 2 MUST produce JSON Schema Draft 2020-12 documents for: + +```text +project-manifest.schema.json +provenance-policy.schema.json +write-intent.schema.json +provenance-event.schema.json +chain-head.schema.json +ledger-segment-manifest.schema.json +record-envelope.schema.json +content-reference.schema.json +prompt-template.schema.json +prompt-template-reference.schema.json +model-configuration-snapshot.schema.json +model-capability-snapshot.schema.json +encrypted-key-envelope.schema.json +project-key-manifest.schema.json +recovery-key-envelope.schema.json +conversation-session.schema.json +transcript-turn.schema.json +transcript-correction.schema.json +transcript-normalization.schema.json +invocation-request.schema.json +invocation-response.schema.json +invocation-failure.schema.json +invocation-state.schema.json +invocation-stream-state.schema.json +invocation-stream-summary.schema.json +disposition-revision.schema.json +idea.schema.json +idea-revision.schema.json +manuscript-revision.schema.json +edit-transaction.schema.json +text-fragment-reference.schema.json +derived-index-manifest.schema.json +verification-report.schema.json +backup-manifest.schema.json +release-manifest.schema.json +release-state.schema.json +sanitized-export-manifest.schema.json +purge-manifest.schema.json +``` + +Prompt-template and other self-digesting schemas MUST define exact digest identity objects. Migration schemas are deferred until after Thinkloom 1.0.0. + +## 25. Required implementation characteristics + +The Stage 3 native implementation MUST provide deterministic fault injection at every durable boundary and test at least concurrency, duplicate retries, partial writes, truncated JSONL, ledger/head disagreement, record corruption, stale context, SQLite online backup during editing, unsafe archives, release failure transitions, secret filtering, protected-key loss, and native-verifier/frontend consistency. + +The complete required test groups are enumerated in [COMPLETION-CHECKLIST.md](COMPLETION-CHECKLIST.md). + +## 26. Explicit deferrals + +The following are not required for the Stage 1 specification or initial unanchored provenance implementation: + +- Legacy-project migration or conversion before/at 1.0.0 +- External trusted timestamping +- Remote transparency logging +- Public hash anchoring +- Hardware-backed signing keys +- Multi-user authority or distributed conflict resolution +- Physical immutable-record packing +- Human-versus-AI contribution percentages + +Locally signed releases, external anchors, and hardware-backed keys remain compatible future trust enhancements. diff --git a/docs/provenance/STATE-MACHINES.md b/docs/provenance/STATE-MACHINES.md new file mode 100644 index 0000000..b70f3e7 --- /dev/null +++ b/docs/provenance/STATE-MACHINES.md @@ -0,0 +1,254 @@ +# Thinkloom Provenance State Machines and Recovery Protocols + +Status: **Normative Stage 1 companion** + +This document defines the required state transitions supporting the [Stage 1 Normative Specification](STAGE-1-NORMATIVE-SPECIFICATION.md). Diagrams are explanatory; the accompanying transition rules are authoritative. + +## 1. Cross-store provenance write + +```mermaid +stateDiagram-v2 + [*] --> PREPARED: create SQLite write intent + PREPARED --> RECORDS_DURABLE: stage, fsync, and move immutable records + RECORDS_DURABLE --> LEDGER_APPENDED: assign sequence; append and fsync event + LEDGER_APPENDED --> CHAIN_HEAD_ADVANCED: atomically replace chain head + CHAIN_HEAD_ADVANCED --> SQLITE_APPLIED: apply operational domain state + SQLITE_APPLIED --> COMPLETE: persist idempotent result + PREPARED --> FAILED: validation or staging failure + RECORDS_DURABLE --> QUARANTINED: operation cannot be completed safely + FAILED --> [*] + QUARANTINED --> [*] + COMPLETE --> [*] +``` + +### 1.1 Binding write procedure + +1. Acquire the exclusive OS-managed project writer lock. +2. Resolve `client_action_id`. If a committed ledger event already contains it, rebuild or return the original result. +3. Validate project identity, policy, schema, paths, references, and command payload. +4. Reserve record, event, intent, and revision IDs. Do not reserve `event_sequence`. +5. Create the `PREPARED` SQLite write intent using durable SQLite settings. +6. Write authoritative immutable records to a transaction staging directory located on the same filesystem as the project. +7. Canonicalize, hash, flush, and fsync each staged record. +8. Atomically move records to final locations and flush affected parent directories where the platform permits. +9. Mark the intent `RECORDS_DURABLE`. +10. Derive the next contiguous event sequence from the verified ledger head. +11. Construct the canonical event, including `client_action_id`, references, record digests, and the previous event digest. +12. Append the complete event plus LF to the active ledger segment and fsync it. +13. Mark `LEDGER_APPENDED`. +14. Atomically replace `chain-head.json`, flush its directory where supported, and mark `CHAIN_HEAD_ADVANCED`. +15. Apply operational SQLite domain state and rebuildable query/idempotency indexes. +16. Mark `SQLITE_APPLIED`, store the typed result, and mark `COMPLETE` in one SQLite transaction where practical. +17. Queue derived-index and Git work outside the critical write. +18. Release the project writer lock. + +Failures updating a non-authoritative SQLite phase after a durable ledger append MUST NOT invalidate the event. Recovery discovers the event through `client_action_id` and replays it into SQLite. + +### 1.2 Idempotency + +Every authoritative event MUST contain `client_action_id`. SQLite MAY accelerate lookups, but retry handling MUST remain possible by scanning or indexing the ledger. + +The same `client_action_id` with a materially different canonical command digest is an integrity or client-contract error; it MUST NOT be treated as an ordinary retry. + +## 2. Startup recovery + +Recovery begins before an editable project becomes active. It obtains the project writer lock and classifies the project: + +```text +CLEAN +RECOVERABLE_AUTOMATICALLY +REQUIRES_USER_CONFIRMATION +INTEGRITY_FAILURE +``` + +```mermaid +flowchart TD + A[Acquire writer lock] --> B[Inspect staging and write intents] + B --> C[Parse active and sealed ledger] + C --> D[Verify records, sequences, hashes, and chain head] + D --> E{Authoritative contradiction?} + E -- Yes --> F[INTEGRITY_FAILURE] + E -- No --> G{Repair needed?} + G -- No --> H[CLEAN] + G -- Safe and deterministic --> I[RECOVERABLE_AUTOMATICALLY] + G -- Ambiguous or destructive --> J[REQUIRES_USER_CONFIRMATION] + I --> K[Repair; append recovery event when ledger usable] + J --> L[Quarantine or await decision] +``` + +### 2.1 Recovery matrix + +| Observed durable state | Classification | Required action | +|---|---|---| +| SQLite intent only; no durable record | Automatic | Remove or retry the abandoned intent. | +| Complete staged files not moved | Automatic | Resume validated same-filesystem move or quarantine. | +| Final immutable records; no ledger event | Automatic or confirmation | Complete only when intent/record metadata determines the exact event; otherwise quarantine. | +| Complete ledger event; chain head behind | Automatic | Verify the event and referenced records, then advance the chain head. | +| Partial final line not referenced by chain head | Automatic | Remove only the incomplete suffix and record recovery after the ledger is usable. | +| Ledger and chain head valid; SQLite behind | Automatic | Replay events into SQLite and rebuild indexes. | +| SQLite ahead of ledger | Automatic | Roll back/rebuild SQLite from authoritative records. | +| Chain head ahead of readable ledger | Failure or confirmation | Do not invent an event; inspect staging and quarantine until resolved. | +| Event references a missing or modified authoritative record | Integrity failure | Block ordinary editing and release. | +| Sealed segment modified or truncated | Integrity failure | Never alter it automatically. | +| Duplicate `client_action_id` with identical command/result | Automatic | Collapse operational duplicates to the committed event. | +| Duplicate `client_action_id` with conflicting command | Integrity failure | Block the conflicting retry. | +| Stale derived index | Automatic warning | Rebuild without changing provenance. | +| Unacknowledged Git source commit | Automatic or warning | Finish checkpoint acknowledgment if the captured tree verifies; otherwise leave hidden. | +| Partial release state | State dependent | Resume or roll back using the release state machine. | + +Orphans MUST be moved to `.app/recovery/orphans/` rather than deleted when their role is uncertain. Recovery actions that change authoritative state MUST append a recovery event once a usable ledger is available. + +## 3. Model invocation lifecycle + +```mermaid +stateDiagram-v2 + [*] --> REQUEST_PREPARED + REQUEST_PREPARED --> REQUEST_RECORDED: immutable request + requested event + REQUEST_RECORDED --> MODEL_RUNNING: release writer lock + MODEL_RUNNING --> RESPONSE_RECEIVED: provider completes + MODEL_RUNNING --> FAILED: provider error or output limit + MODEL_RUNNING --> CANCELLED: user/provider cancellation + MODEL_RUNNING --> ABANDONED: process loss; no resumable provider state + RESPONSE_RECEIVED --> RESPONSE_RECORDED: reacquire lock; immutable response + event + RESPONSE_RECORDED --> STAGED_PREVIEW + STAGED_PREVIEW --> ACCEPTED + STAGED_PREVIEW --> PARTIALLY_ACCEPTED + STAGED_PREVIEW --> REJECTED + STAGED_PREVIEW --> STALE_CONTEXT_CONFIRMATION: source revisions changed + STALE_CONTEXT_CONFIRMATION --> ACCEPTED: revalidated/rebased/confirmed + FAILED --> [*] + CANCELLED --> [*] + ABANDONED --> [*] + ACCEPTED --> [*] + PARTIALLY_ACCEPTED --> [*] + REJECTED --> [*] +``` + +### 3.1 Before provider I/O + +Under the writer lock, Thinkloom MUST resolve and hash context; snapshot effective model configuration and prompt-template identity; create the immutable invocation request permitted by retention policy; record manuscript, idea, and conversation revision heads; append `MODEL_INVOCATION_REQUESTED`; then release the lock. + +### 3.2 Provider I/O + +The provider call occurs outside the writer lock. Output streams only to bounded encrypted temporary storage. The app MAY process other commands and model completions concurrently, subject to configured spool limits. + +### 3.3 Completion and disposition + +Thinkloom reacquires the lock to create an immutable response or failure, append the completion/failure event, and create a staged preview. Acceptance, partial acceptance, rejection, and later user edits are separate provenance commands. + +When a source revision changed during provider I/O, insertion MUST NOT proceed silently. The disposition records whether the result was revalidated, rebased, explicitly accepted as stale, or rejected. + +## 4. Temporary invocation spool + +Operational stream states are: + +```text +CREATED → CONNECTING → STREAMING → COMPLETED + ├→ FAILED + ├→ CANCELLED + ├→ ABANDONED + └→ RECOVERING +``` + +Incomplete spool content is not evidence. On completion, the canonical response/failure must become durable before spool-key deletion. On startup, a spool may be recovered for bounded local inspection or explicit preservation, but Thinkloom MUST NOT describe the original provider request as resumed unless the provider supports it. + +## 5. Ledger segment rotation + +```mermaid +flowchart LR + A[Hold writer lock] --> B[Flush and verify active segment] + B --> C[Create and fsync sealed manifest] + C --> D[Move segment and manifest to sealed] + D --> E[Create next active segment] + E --> F[Append contiguous segment-opening event] + F --> G[Advance chain head] + G --> H[Release writer lock] +``` + +The opening event links to the final event of the prior segment. Segment file numbering and event sequencing are independent: segment numbers are contiguous files; event sequences are contiguous events. A failure before the new opening event is recoverable from the sealed manifest and prior head. Sealed bytes MUST NOT be modified during recovery. + +## 6. Git checkpoint lifecycle + +```mermaid +sequenceDiagram + participant P as Provenance writer + participant S as Frozen snapshot/index + participant G as Git + P->>P: Flush meaningful edits; capture H0 + P->>S: Materialize immutable source tree + P-->>G: Release provenance lock + G->>G: Create source commit C1 from captured tree + G-->>P: Return C1 and tree hash + P->>P: Verify C1; append checkpoint event H1 + P->>S: Capture acknowledgment tree + P-->>G: Release provenance lock + G->>G: Create audit commit C2 from captured tree +``` + +The source and acknowledgment commits MUST be created from frozen captured trees or isolated Git indexes, not the changing live worktree. `C1` is the restore content. `C2` contains the acknowledgment and may be the visible checkpoint ref. If acknowledgment fails, `C1` remains hidden and recoverable. + +## 7. Release finalization + +```mermaid +stateDiagram-v2 + [*] --> WORKING + WORKING --> FREEZING_SOURCE + FREEZING_SOURCE --> SOURCE_FROZEN: create source commit and bind source head + SOURCE_FROZEN --> GENERATING_RELEASE + GENERATING_RELEASE --> RELEASE_STAGED + RELEASE_STAGED --> RELEASE_VERIFIED: full native verification + RELEASE_VERIFIED --> RELEASE_COMMITTED + RELEASE_COMMITTED --> RELEASE_TAGGED + RELEASE_TAGGED --> COMPLETE + FREEZING_SOURCE --> FAILED + GENERATING_RELEASE --> FAILED + RELEASE_STAGED --> FAILED + RELEASE_VERIFIED --> FAILED + RELEASE_COMMITTED --> RECOVERABLE: tag not yet created + RECOVERABLE --> RELEASE_TAGGED +``` + +Each state transition MUST be durably recoverable and idempotent. The source commit binds the frozen canonical project and source chain head. The manifest binds the source commit but not the later release commit. The release commit contains the verified release package and manifest; the tag points to it. + +A release is not `COMPLETE` until the tag and manifest bindings verify. Failure MUST NOT leave a partial directory or archive that appears complete. + +## 8. Backup import + +```mermaid +flowchart TD + A[Copy archive to controlled staging] --> B[Validate archive structure and limits] + B --> C{Unsafe?} + C -- Yes --> D[Reject as UNSAFE] + C -- No --> E[Extract into new staging directory] + E --> F[Verify manifest and every file digest] + F --> G[Verify provenance, SQLite, Git, identity] + G --> H{Verification status} + H -- VERIFIED --> I[Check destination conflicts] + H -- VERIFIED_WITH_WARNINGS --> J[Permit only non-security warnings with disclosure] + H -- INCOMPLETE --> K[Quarantine] + H -- FAILED --> L[Block] + I --> M[Atomically activate verified project] + J --> M +``` + +Extraction MUST NOT target an active project directory. Activation MUST remain on the same filesystem or use a verified copy-and-swap protocol with equivalent crash recovery. + +## 9. Retention and encryption policy changes + +Policy changes are append-only provenance operations: + +```text +minimal → full_private: future full records only +full_private → minimal: future minimization only; existing full records remain +none → protected: allowed only after recovery-key verification and protected rewrite protocol +protected → none: explicit confirmed decrypt/rewrite operation +full export ↔ sanitized export: export preference only; no project mutation +``` + +Content already discarded under minimal retention cannot be recreated. Content already committed under full retention remains until emergency purge. + +## 10. Emergency purge + +Emergency purge is not ordinary recovery. It requires a separate confirmed state machine that freezes the source, identifies every affected record/reference/index/Git object/export, creates a purge manifest, rewrites retained evidence and Git history, establishes a new chain root, verifies the result, and records the superseded root when safe. + +The UI MUST disclose that earlier copies, backups, releases, and exports cannot be revoked.