Files
XZBT/docs/XZBT_0-1_Verification_Gates.md
T
LabyricornandClaude Opus 5 f60b3b6442 docs: record the phase 6 scenario evidence and status
Add the Phase 6 evidence record, update the implementation status,
verification gates, gap-closure decisions, implementation plan and README
for the scenario director, and state plainly that Phase 6 is not accepted
until its two-hour real-duration soak is observed.

Co-Authored-By: Claude Opus 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01M7dgfQ12mpM4JjSMv3inLA
2026-09-06 22:54:40 +00:00

16 KiB
Raw Blame History

XZBT 0.1 Verification Gates

Status: Phase 0 shared-contract preparation complete before Phase 1. GC1 direct-file feasibility and GC2GC5 shared contracts are verified; subsystem integration gates remain in their assigned later phases. Related resources: PRD, decisions, format specification

A separate Phase 0 probe has been created and its JavaScript/JSON syntax checked. The agent's direct-file navigation was blocked by browser tool policy; see the attempt record. The user's manual run reports successful import, saves, and context unlock, but worklet loading failed. Version 2 separates the tests and records better diagnostics; its two stubbed regression tests pass, not its browser gate. Mark a check complete only with linked evidence; a partial result does not pass a compound acceptance check.

The second user run confirmed cache restoration between page loads, a heard tone, and audio suspend/resume, but Blob worklet loading failed. The version 3 user run verified the embedded-data-URL fix. The subsequent manual restart test, supplied after the explicit full-exit procedure, restored edited activity and volume with zero new saves. The directory fallback observation confirmed graceful error handling and state preservation when directory access is cancelled/denied; GC1 is fully passed. GC2 schema/semantic validation and its 12-case matrix are verified in the GC2 evidence. GC3GC5 shared resolution, clock/PRNG, and ownership/failure contracts and deterministic traces are verified in the combined contract evidence.

Phase 0 and later gates

The implementation plan is complete and Phase 1 may begin. GC1 and the shared portions of GC2GC5 are satisfied. Early GC6 measurements still precede expensive subsystem optimization decisions. Full production conformance and real-duration soak results remain later requirements.

Subsystem-specific format details can be completed in dependency order as planned tasks. Do not require the finished engine or eight-hour soak before creating the plan.

Gap Decision Evidence status Gate placement
GC1 Recorded Passed; all 10 direct-file feasibility checks verified with linked empirical evidence Phase 0, before deployment architecture commitment
GC2 Recorded; shared contracts complete Shared document, types, ValueSpec, ConditionSpec, schema, validator, and 12-case fixture suite verified; subsystem contracts in dependency order Shared contracts in Phase 0; subsystem contracts before their implementation
GC3 Recorded; shared transition rules complete Deterministic resolver traces pass Shared resolver contract complete; repeat with Phase 2 implementation
GC4 Recorded; shared clock/audio mapping/PRNG complete Deterministic clock, unlock-decision, and stream tests pass Clock contract complete; real synchronization tests in audio integration
GC5 Recorded; shared lifecycle rules complete Deterministic ownership/failure traces pass Ownership contract complete; repeat with real actions/audio/scenarios
GC6 Recorded; thresholds provisional Measurement protocol and schedule defined; reference setup/benchmark results pending Early combined-load prototype; development and release acceptance
GC7 Recorded; identity/update/build boundary complete Milestone map complete; implementation checks pending Planning complete; then library/build integration

1. GC1: Direct-file feasibility

  • Record OS, browser version/profile, launch path, and storage mode. The preceding environment records and regular-Chrome confirmation establish Windows, Chrome 152, direct-file launch, and IndexedDB in a regular non-Incognito profile.
  • Open the artifact directly from disk with network unavailable. The follow-up capture and user confirmation verify restoration, tone, and worklet with browser-scoped Offline mode enabled. The changed mobile user agent on Win32 does not establish physical Android support.
  • Import two exhibits with the ordinary file picker and activate each. The user confirmed ordinary import and selection of both fixtures worked.
  • Save exhibit definitions, per-exhibit parameters, selection, and master volume to IndexedDB. See the version 1 saves and subsequent restored snapshots, including edited values in the restart record.
  • Close the browser fully, reopen the same HTML, and restore the cached library and settings without selecting source files again. Passed as user-performed evidence following the explicit full-exit procedure; see the restart record for provenance and limits.
  • Verify user-initiated audio unlock and resumed playback. Native tone heard in versions 2/3; AudioContext suspend/resume states observed in version 2. This is not full audio-engine conformance.
  • If needed by the audio design, load engine-owned AudioWorklet code from the single-file artifact. Verified with the data-URL module in the version 3 user run.
  • Test renamed and moved HTML files; record observed storage behavior and the support boundary. The user confirmed restoration from both copies; see file-location evidence and limits.
  • Test unavailable/failed storage; session playback works and the UI explains lack of persistence. Injected-failure evidence confirms settings changes, import, and heard native playback remain operational with zero successful writes; the user subsequently confirmed the visible session-only warning. Passed using application fault injection, not real browser permission denial.
  • Test directory import/remembered handles where available and ordinary-picker fallback where unavailable or permission is denied. Directory import completed in the injected-failure run and ordinary import/activation passed separately. Directory cancellation/denial fallback is verified with preserved session state; see directory fallback evidence. Remembered handles are optional and not implemented in this probe; verification is required if implemented in the runtime.

Evidence: prototype artifact, reproducible steps, environment record, observed results. If ordinary-mode persistence fails, reopen decision 1 before dependent architecture commitment.

2. GC2: Format contract completeness

  • Complete shared document, type, reference, ValueSpec, ConditionSpec, time, and ownership contracts. Formalized in Format Specification Rev 0.2.
  • Fill the format-specification contract register in subsystem dependency order. Document/schema and Values/conditions completed.
  • Implement an internal structural schema plus separate semantic validation. Schema authored in schema/xzbt-0.1.schema.json, validator implemented in tools/validate-exhibit.mjs.
  • Provide complete valid and invalid fixtures with expected diagnostics. See test/fixtures/gc2/ and GC2 evidence.
  • Reject unsupported versions, unknown behavior fields, invalid reference types, recursive components, illegal graph cycles, and excessive resources. Verified with 12/12 passing automated test matrix in test/gc2-runner.test.mjs.
  • Produce two contrasting complete audiovisual exhibits early; expand toward all PRD challenge cases. Minimal and full feature test fixtures created; full audiovisual reference exhibits belong to Phase 1/Phase 9.
  • Reconcile prior conversation proposals into the local documents; resolve conflicts explicitly. Reconciled into local format specification Revision 0.2.

Evidence: versioned specification/schema/fixtures and validation results in 2026-09-05-gc2-format-contracts.md. Naming a node or behavior does not close its contract.

3. GC3: Resolution semantics

  • Record exact target capabilities, override defaults, smoothing, and interrupted transition rules.
  • Define expected traces for a bound bus gain receiving a direct override.
  • Define expected traces for a parameter override feeding a binding.
  • Verify stored user edits during masking and release toward the updated lower value.
  • Verify priority ties and competing duration/scenario overrides.
  • Verify automation continues while masked and permitted modulation applies after an override.
  • Verify numeric clamps and unsupported target/stage diagnostics.
  • Reject conflicting bindings and dependency cycles without introducing undocumented delays.

Evidence: GC3GC5 shared contract evidence, executable resolution model, normative JSON traces, and deterministic resolver tests. Production resolver conformance repeats in Phase 2.

4. GC4: Time and reproducibility

  • Define logical tick ordering, maximum work per turn, long-stall behavior, audio lookahead, and audio unlock alignment.
  • Define PRNG algorithm, seed normalization, and stream derivation for the runtime version.
  • Compare simulation traces at different render frequencies using the same logical inputs.
  • Simulate hide/restore and explicit pause/resume; no missed wall-clock work is replayed. Repeat with the Phase 1 runtime in a browser.
  • Verify in the clock model that visibility restoration does not undo an explicit pause.
  • Verify audio suspend/resume in the GC1 browser probe and verify pre-unlock one-shot suppression in the deterministic scheduling model. Repeat combined behavior during audio integration.
  • Verify manual SAMPLE does not perturb cadence/scenario random streams.
  • Use deterministic signal fixtures for audio-reactive decision tests.
  • Distinguish accelerated logical tests from real-time audio behavior and soak tests.

Evidence: GC3GC5 shared contract evidence, clock/PRNG model and known vectors, logical traces, deterministic signal fixture, plus the earlier GC1 browser audio observations. This closes the Phase 0 contract gate, not later real-time synchronization or soak gates. No cross-device pixel or waveform equality claim is required.

5. GC5: Ownership and failure

  • Define hook ordering, allowed hook actions, failure propagation, cleanup deadlines, deferred expiry/order, and dispatch-budget behavior.
  • Start resources through nested events and confirm inherited scenario ownership.
  • Verify duration overrides terminate with their owner and bounded releases eventually dispose.
  • Verify persistent state mutations survive a later critical failure.
  • Inject failures into startup, ordinary actions, and termination hooks; cleanup still completes.
  • Verify condition triggers require a false condition before rearming.
  • Verify at most one pending deferred request per definition, expiry, and eligibility recheck.
  • Exercise event/scenario feedback and show bounded dispatch with diagnostics.
  • Repeat completion/cancellation/failure cycles; counters return to baseline after releases finish.

Evidence: GC3GC5 shared contract evidence, lifecycle oracle, resource-count assertions, and failure-injection tests. Repeat with subsystem-owned resources during Phases 36.

6. GC6: Performance and audio acceptance

  • Record CPU/GPU/RAM, OS, browser version, 1920 x 1080 viewport, device-pixel ratio, sample rate, and relevant power settings.
  • Fix benchmark exhibit/version/seed and counts for particles, links, trails, effect passes, audio nodes, and concurrent voices.
  • Prototype combined visual/audio load early and record supported limits.
  • Define warm-up duration, measurement window, frame interval sampling, and retained-memory sampling method before collecting acceptance data. See the implementation plan; no benchmark result is claimed yet.
  • Check the provisional p95 frame interval threshold of less than 33.3 ms and report progress toward the 60 FPS target.
  • Record active resource baselines and verify expected return after repeated lifecycle operations.
  • Specify and test digital output peak ceiling, numerical tolerance, finite samples, release behavior, and unavoidable master routing.
  • Stress overlapping recipes and listen for clicks, clipping, and objectionable release artifacts.
  • Run a two-hour development soak and an eight-hour release soak; record retained-memory trends, resource counts, frame intervals, audio glitches, and scheduler growth.

Evidence: fixed workload, environment, measurement method, results, and investigated anomalies. Any threshold revision must be recorded before rerunning acceptance; do not silently change the benchmark to produce a pass.

7. GC7: Scope, updates, and build

  • Map all PRD completion criteria and challenge cases to milestones and evidence.
  • Integrate two contrasting exhibits early; make Phase 9 the completion/audit phase for all five reference exhibits.
  • Define import equality and parameter compatibility precisely.
  • Verify identical imports are no-ops and changed same-ID content requires an explicit replacement choice.
  • Verify invalid refreshes leave the last valid definition and configuration intact.
  • Verify compatible parameter preservation, numeric bound changes with notice, removed/new parameters, and incompatible-type/enum resets.
  • Validate and prepare a candidate without observable playback before stopping the working exhibit.
  • Inject activation failure; dispose of the candidate and attempt a fresh restart of the previous exhibit with saved settings.
  • Build separate source modules into one self-contained HTML artifact reproducibly.
  • Test the delivered artifact directly and offline; no external runtime scripts, modules, assets, fonts, or CDN requests are required.

Evidence: implementation-plan coverage, library failure tests, build instructions, and artifact verification.

Evidence record template

Phase 6 checkpoint — September 6, 2026

The production scenario director now re-runs the GC5 ownership, failure/hook, condition, deferred-order/expiry, dispatch-budget and repeated-cleanup cases with real runtime voice pools, visual instances, overrides and scheduler records. The native audio API is doubled in automated tests; the production graph/lifecycle code executes. Forty-minute Exhibit E traces are reproducible using accelerated fixed ticks. Evidence is in the Phase 6 record.

The self-contained acceptance page and versioned development workload are ready. No two-hour real-time soak, retained-memory plateau, audio listening result, or Phase 4h hardware benchmark is claimed by this checkpoint. Those GC6 checkboxes stay open.

For each result, record gate/check ID, date, specification revision, implementation revision or artifact hash, environment, fixture/seed, procedure, expected result, actual result, pass/fail/not-applicable status, linked logs or measurements, and unresolved limitations. A not-applicable result requires a reason. Do not prefill results from intended behavior.