Engine architecture

This is a conceptual map of the open Baryon engine: the pipeline, the one seam that matters most, and the contracts that stay stable across versions. For the governing equations and the modal physics, read the engine whitepaper; for the implementation, read the source on GitHub.

The pipeline

consecutive mono/stereo PCM
  -> signal preparation and DC removal
  -> signed modal pressure/velocity state
  -> full quadratic moments and response spectra
  -> AudioFeatureFrame / versioned model transport
  -> derived GPU field and spectral caches
  -> persistent surface observer
  -> material, light integration, postprocessing and output

The source/session owns generations and resets. Accepted audio samples advance the response; display frames, camera movement, cached reads and transport acknowledgements do not. Spectral analysis supports color and diagnostics rather than choosing which input frequencies may drive the modeled cavity.

The packages

packages/engine owns capture/preparation, modal response, moment and spectral evidence, model contracts, controls, derived GPU fields and rendering.
packages/app-shell owns shared React orchestration, control lifecycle, scene mounting and diagnostics.
The web and desktop applications integrate those shared owners into their hosts. Desktop additionally owns stage/operator transport and native output boundaries.

State and observation

The apparatus is a finite rectangular water-cavity model with a 6.25 m default side length. A spherical render hull clips the view. It does not change the cavity’s boundary conditions. Families with the same natural frequency and losses share shell state; signed source overlaps are applied once during spatial reconstruction. Each channel drives signed dimensionless pressure and velocity-basis coefficients P and R. For a shell with angular frequency omega and quality factor Q:

dP/dt = -d P - omega R + g x(t)
dR/dt = omega P
d = g = omega / Q

Here x(t) is the prepared waveform held over each sample interval. State propagation uses a precomputed affine matrix exponential. Raw second moments include every shell pair, retaining interference and finite-time beats. Stereo channels are independent responses whose quadratic measurements are averaged, not coherently downmixed. Reconstructed pressure and velocity mean squares give the normalized observation:

U = (w_p P2 - w_v V2) / N2       (N2 > 0)

N2 is the spatial mean of total pressure-plus-velocity energy; exact zero response has zero support. A fixed spatial aperture filters the field before geometry extraction. The persistent observer follows its zero-level surface. Raw energy remains available for brightness. These are normalized values from the acoustic model, not calibrated room pressure, particle forces or measured acoustic intensity.

Model boundary and lifecycle

AudioFeatureFrame carries the audio-owned model to visualization. Its versioned payload includes topology, signed response state, complete moments, spectral evidence, and source/sample identities. GPU projections are derived caches, not another solver. OSC Structure Export uses the complete v5 model for external receivers. Pause holds acoustic time. Seek, stop, source replacement and incompatible configuration changes invalidate the affected histories. Natural end supplies audio-clocked silence so preparation tails and free decay can continue. A display reconnect replays a matching cached model without restarting capture. Ordered observation delivery preserves the history the visual observer needs.

The contracts that stay stable

Control schema

The source of truth for control keys, defaults, applicability, and whether a control is live or debug-only. Changing a label is cheap. Changing a key that is persisted or synced is a compatibility change, because that key is effectively part of a stored format.

Persistence

Only controls marked live are saved into presets and auto-save; debug-only controls are excluded. Loading starts from schema defaults, drops unknown keys, falls back to defaults for missing ones, and normalizes a few legacy fields forward. Adding a new live control with a correct default is usually safe; renaming or removing a persisted key needs explicit migration.

Visualization method

raymarch is the only supported method. Legacy values (fullscreen-volume, cymatics-2d) collapse to raymarch at ingress. The field is kept as a constant for forward compatibility; there is no user-facing visualizer selector.

Performance profile

Three values: auto (adaptive step-budget behavior on), custom (adapts toward an explicit frame budget), and max-quality (step-budget adaptation off). Persisted control/config ingress still migrates the legacy none alias to max-quality; live render profiles stay canonical. The user-facing wording is “Performance Profile,” but some compatibility fields still use qualityPreset or renderQualityPreset — treat those as naming vocabulary that still matters, not as free renames.

Input semantics

Device classification and browser capture processing belong to acquisition. The retained Voice/Vocal and Ambient intent settings do not turn the broadband response into separate single-pitch and multi-source solvers. Consecutive prepared samples drive the same model. Echo cancellation, noise suppression and automatic gain can change the captured signal; use them deliberately, especially for direct music feeds.

Spectral Light color

The response spectrum uses a 2048-sample Blackman window with a 512-sample hop. All positive bins contribute, including Nyquist with its endpoint weighting; DC has no pitch hue. Reconstructed pressure and velocity spectra contribute additive support and first/second circular pitch moments. This includes cross-mode structure within each bin. Derived GPU spectral fields feed the persistent color observer. An octave-periodic artistic mapping selects chromaticity; it is not a conversion from sound frequency to optical wavelength. Spectral Chroma changes vividness at constant luminance. Color cannot change acoustic energy, geometry or extinction.

Optical integration

The camera accumulates emission and extinction along each ray using the constant-segment radiative-transfer equation. Its small-optical-depth evaluation uses a stable series, so faint contributions are not lost to Float32 subtraction. Ray-step count controls sampling accuracy, not the underlying acoustic state. Bloom and output composition act after this integration.

Verification boundaries

Numerical references, native GPU comparisons, lifecycle regressions, music-response checks and sustained runs measure different contracts. A passing software timing test does not establish physical audio-to-display latency. That requires synchronized audio and display measurement. Cross-platform native output and live external receivers also require their own acceptance checks.

Changing the engine safely

Before moving a boundary, ask whether the change is semantic or an implementation detail, whether the value crosses persistence or host boundaries, and whether tests already assert its shape. Prefer normalization layers and dual-read compatibility over broad renames at genuine persistence or host boundaries. Inside the semantic core, use one canonical name and translate legacy values only at the boundary that still requires them.

Foundations

Modal expansion describes the established use of driven resonant modes. Volume transmittance describes attenuation along a camera ray. These support the underlying methods; Baryon’s source coupling, observation surface, palette and material remain declared model and presentation choices.