sonora/README.md

Sonora — sound in space

A standalone local audio visualizer inspired by Lucio Arese’s sound study. Three interactive 3D plots show texture, tones and modulation. This is an interpretation, not the artist's original code or exact analysis algorithm.

The default Constellation view follows the visual direction of Seeing Birdsong and Visualizing Bird Songs at 4:42: coloured square markers, delicate connections, floating numbers and a slowly orbiting camera on a large black stage. Texture, Frequency and Modulation retain the original scientific plots.

Constellation uses an original nonlinear feature projection of spectral peaks, amplitude, entropy, spread and timestamp. Its visual coordinates are not physical units or the artist's original embedding. Each square represents a detected spectral peak. Labels show measured peak amplitude and frequency; Bright connections follow frequency-matched tones over time, falling back to the nearest point in the preceding audio frame when pitch changes quickly; softer connections indicate geometric proximity. These visual links are not a proven acoustic relationship. The top-right amplitude is waveform RMS. Rendering is bounded to 2,400 points, one sound-trail link and up to two neighbour links per point, and 24 labels. Automatic orbit defaults off when the system requests reduced motion.

Run

cd sonora
npm install
npm run dev -- --port 5173

Open the displayed localhost address. Choose Open a song, drop an audio file anywhere, or press play for a real blackbird field recording. Drag each plot to orbit; scroll/pinch to zoom. Space toggles playback when a control isn't focused. Use the fullscreen button for an immersive view.

Colour by → Position in space paints a fixed three-dimensional colour field over the plots. Horizontal position is the strongest influence, while height and depth shift hue and shade. Camera movement does not change the colour at a given plot position; travelling points change colour as they move. Brightness still responds to loudness. Spectrum is the default palette. Glow controls the softness and size of the light around points; Loudness restores amplitude-based colouring. These controls do not affect audio or measurements.

MP3, WAV, M4A, FLAC, OGG, AAC and MP4 audio are supported where your browser supports their codecs. Streaming-service URLs and DRM files are not supported. Files remain local and are streamed through a media element; no backend or file upload is used. The optional DM Sans font is fetched from Google Fonts, with a system fallback.

Screen-width ribbons make the connections visible on high-density screens. Points, connections and labels fade progressively to zero over six seconds by default (adjustable with Trail length), including the final notes after playback ends. Pausing holds the view. Points and their connections retain stable identities as history rolls forward. Incoming positions are smoothed, and point, line and label fades run at display refresh rate. Labels stay attached to their original points. The camera settles around a stable centre and widens only as needed; Reset camera fits the current shape again. Connections use a cached spatial index, and hidden plots skip geometry updates. The live FPS readout measures actual rendering cadence; performance depends on your device and display.

The default birdsong is Common Blackbird song (Turdus merula), recorded by Diana Tudor and used unmodified under CC BY 4.0. It streams from Wikimedia Commons and requires an internet connection. The player requests anonymous CORS access so Web Audio can analyze the recording.

Measurements

4096-point Web Audio FFT, sampled at up to 30 frames per second. Power-weighted spectral centroid and standard deviation, normalized Shannon entropy, geometric/arithmetic power ratio (flatness), waveform RMS and up to 12 spectral peaks. Modulation shows per-frame changes in centroid and RMS. Volume is applied after analysis. Sensitivity changes display scaling only. Trails are bounded to 30 seconds and reset on seeks and file changes. The initial preview is illustrative; live plots come from actual playback.

Validate and build

npm test
npm run build
npm run preview

The production output is in dist/. Requires WebGL2 and Web Audio. The original BudgyAI app and backend are independent of this folder.