Sonora: local 3D audio visualizer (initial import)

Three.js + Web Audio FFT visualizer with Constellation, Texture, Frequency
and Modulation views. Vite build, node --test suite.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JABXBRLs9T6nqHKUVwoTHJ
This commit is contained in:
marcuspaico
2026-09-08 15:57:51 +10:00
commit ab748bfb9d
18 changed files with 2282 additions and 0 deletions

2
.gitignore vendored Normal file
View File

@@ -0,0 +1,2 @@
node_modules/
dist/

20
DESIGN.md Normal file
View File

@@ -0,0 +1,20 @@
# Sonora audio space
Standalone browser instrument inspired by Lucio Arese's “A Nightingale's Song Translated Into Data”: black plotting space, fine grids, luminous points, connected histories and three coordinated views. Built separately from BudgyAI.
## Design and scope
Large spread–entropy plot with two smaller tone and modulation plots. Each is real 3D with orbit/zoom. Quiet graphite controls, warm white text, spectral violet, coral and yellow reserved for data. System sans for controls; monospaced numerals for scientific readouts. Desktop has a narrow control rail, mobile stacks controls and plots.
Files are selected or dropped locally. HTML media playback feeds Web Audio FFT analysis, then Three.js buffers. No file leaves the browser. Browser codec support determines playable formats. Generated demo is original synthesis, not the reference recording. Initial illustrative preview is explicitly labeled.
Features: play/pause, seek, volume, file replacement, trail length, sensitivity, palette, auto rotation, reset camera, fullscreen, three selectable main plots. RMS amplitude, power-weighted spectral centroid/spread, normalized Shannon entropy and spectral flatness are computed from measured samples. Modulation view uses frame-to-frame centroid and amplitude changes, not an exact reconstruction of the artist's unpublished algorithm.
## Implementation plan
1. Write numerical analysis tests for silence, single tones, uniform noise and invalid values; observe failure, implement and pass.
2. Build a bounded 3D history renderer with labeled axes, point glow, trails, responsive sizing and orbit controls.
3. Build local playback and upload lifecycle; clean object URLs on replacement, display codec errors, clear history on seeks and replacements, sample only when playing.
4. Compose responsive controls and demo audio. Verify uploads, playback, seeking, replacements, controls and mobile layout in the browser. Run tests and production build.
No backend changes, deployment or changes to the existing app are required.

39
README.md Normal file
View File

@@ -0,0 +1,39 @@
# Sonora — sound in space
A standalone local audio visualizer inspired by [Lucio Arese’s sound study](https://www.youtube.com/shorts/jtNGnkscgds). Three interactive 3D plots show texture, tones and modulation. This is an interpretation, not the artist's original code or exact analysis algorithm.
The default **Constellation** view follows the visual direction of [Seeing Birdsong](https://www.lucioarese.net/seeing-birdsong/) and [Visualizing Bird Songs at 4:42](https://www.youtube.com/watch?v=7lEiYqCV25s&t=282s): coloured square markers, delicate connections, floating numbers and a slowly orbiting camera on a large black stage. **Texture**, **Frequency** and **Modulation** retain the original scientific plots.
Constellation uses an original nonlinear feature projection of spectral peaks, amplitude, entropy, spread and timestamp. Its visual coordinates are not physical units or the artist's original embedding. Each square represents a detected spectral peak. Labels show measured peak amplitude and frequency; Bright connections follow frequency-matched tones over time, falling back to the nearest point in the preceding audio frame when pitch changes quickly; softer connections indicate geometric proximity. These visual links are not a proven acoustic relationship. The top-right amplitude is waveform RMS. Rendering is bounded to 2,400 points, one sound-trail link and up to two neighbour links per point, and 24 labels. Automatic orbit defaults off when the system requests reduced motion.
## Run
```sh
cd sonora
npm install
npm run dev -- --port 5173
```
Open the displayed localhost address. Choose **Open a song**, drop an audio file anywhere, or press play for a real blackbird field recording. Drag each plot to orbit; scroll/pinch to zoom. Space toggles playback when a control isn't focused. Use the fullscreen button for an immersive view.
**Colour by → Position in space** paints a fixed three-dimensional colour field over the plots. Horizontal position is the strongest influence, while height and depth shift hue and shade. Camera movement does not change the colour at a given plot position; travelling points change colour as they move. Brightness still responds to loudness. Spectrum is the default palette. **Glow** controls the softness and size of the light around points; **Loudness** restores amplitude-based colouring. These controls do not affect audio or measurements.
MP3, WAV, M4A, FLAC, OGG, AAC and MP4 audio are supported where your browser supports their codecs. Streaming-service URLs and DRM files are not supported. Files remain local and are streamed through a media element; no backend or file upload is used. The optional DM Sans font is fetched from Google Fonts, with a system fallback.
Screen-width ribbons make the connections visible on high-density screens. Points, connections and labels fade progressively to zero over six seconds by default (adjustable with Trail length), including the final notes after playback ends. Pausing holds the view. Points and their connections retain stable identities as history rolls forward. Incoming positions are smoothed, and point, line and label fades run at display refresh rate. Labels stay attached to their original points. The camera settles around a stable centre and widens only as needed; Reset camera fits the current shape again. Connections use a cached spatial index, and hidden plots skip geometry updates. The live FPS readout measures actual rendering cadence; performance depends on your device and display.
The default birdsong is [Common Blackbird song (Turdus merula)](https://commons.wikimedia.org/wiki/File:Common_Blackbird_song_(Turdus_merula).ogg), recorded by Diana Tudor and used unmodified under [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/). It streams from Wikimedia Commons and requires an internet connection. The player requests anonymous CORS access so Web Audio can analyze the recording.
## Measurements
4096-point Web Audio FFT, sampled at up to 30 frames per second. Power-weighted spectral centroid and standard deviation, normalized Shannon entropy, geometric/arithmetic power ratio (flatness), waveform RMS and up to 12 spectral peaks. Modulation shows per-frame changes in centroid and RMS. Volume is applied after analysis. Sensitivity changes display scaling only. Trails are bounded to 30 seconds and reset on seeks and file changes. The initial preview is illustrative; live plots come from actual playback.
## Validate and build
```sh
npm test
npm run build
npm run preview
```
The production output is in `dist/`. Requires WebGL2 and Web Audio. The original BudgyAI app and backend are independent of this folder.

53
index.html Normal file
View File

@@ -0,0 +1,53 @@
<!doctype html>
<html lang="en">
<head><meta charset="UTF-8"><meta name="viewport" content="width=device-width, initial-scale=1.0"><meta name="theme-color" content="#070809"><meta name="description" content="Explore your music as a living 3D data sculpture. Local audio visualization with interactive spectral analysis."><title>Sonora — Sound in space</title></head>
<body>
<div id="app">
<header class="topbar">
<a class="brand" href="./" aria-label="Sonora home"><svg viewBox="0 0 36 36" aria-hidden="true"><path d="M3 18h4l3-10 5 22 6-26 5 22 3-8h4"/></svg><span>sonora<span class="brand-dot">.</span></span></a>
<span class="tagline">Sound, given dimension.</span>
<div class="top-actions"><span class="local"><span></span> Runs locally</span><button id="help" class="quiet" aria-label="About the visualization">?</button></div>
</header>
<main>
<aside class="sidebar">
<div class="intro"><p class="eyebrow">Audio observatory</p><h1>Listen. <br>Look closer.</h1><p>Every sound leaves a shape.<br>Explore the one inside yours.</p></div>
<button id="upload" class="upload"><span class="upload-icon">↥</span><strong>Open a song</strong><span>or drop a file anywhere</span><small>MP3, WAV, M4A, FLAC, MP4</small></button>
<input id="file" type="file" accept="audio/*,video/mp4,.mp3,.wav,.m4a,.flac,.ogg,.aac,.mp4,.webm" hidden>
<button id="demo" class="demo-button"><span>▷</span> Play birdsong <span class="duration">1:10</span></button>
<div class="settings">
<div class="section-title">Shape the view<button id="reset-settings" class="text-button">Reset</button></div>
<label class="range-label" for="sensitivity">Sensitivity <output id="sensitivity-value">1.0×</output></label><input id="sensitivity" type="range" min="0.5" max="3" step="0.1" value="1">
<label class="range-label" for="trail">Trail length <output id="trail-value">6 s</output></label><input id="trail" type="range" min="3" max="30" step="1" value="6">
<div class="range-label palette-label">Color palette <span id="palette-name">Ember</span></div>
<div class="palettes" role="group" aria-label="Color palette"><button data-palette="ember" class="swatch ember selected" aria-label="Ember palette" aria-pressed="true"></button><button data-palette="spectrum" class="swatch spectrum" aria-label="Spectrum palette" aria-pressed="false"></button><button data-palette="glacier" class="swatch glacier" aria-label="Glacier palette" aria-pressed="false"></button><button data-palette="orchid" class="swatch orchid" aria-label="Orchid palette" aria-pressed="false"></button></div>
<div class="appearance"><label class="mapping-label" for="color-mode">Colour by <select id="color-mode"><option value="position">Position in space</option><option value="frequency">Frequency</option><option value="amplitude">Loudness</option></select></label><p id="color-description">Colour follows position. Brightness follows sound.</p><label class="range-label" for="glow">Glow <output id="glow-value">1.35×</output></label><input id="glow" type="range" min="0.5" max="2.5" step="0.05" value="1.35"></div>
<label class="toggle-row" for="rotate">Orbit automatically <input id="rotate" type="checkbox" role="switch"><span class="switch"></span></label>
<label class="toggle-row" for="connections">Connect the dots <input id="connections" type="checkbox" role="switch" checked><span class="switch"></span></label>
<label class="toggle-row" for="labels">Floating measurements <input id="labels" type="checkbox" role="switch" checked><span class="switch"></span></label>
</div>
<p class="privacy">Your music stays on your device.<br>No uploads. Just sound and space.</p>
</aside>
<section class="workspace constellation-view" aria-label="Audio visualization">
<div class="workspace-heading"><div><span class="status-dot"></span><span id="status">Illustrative preview</span></div><div class="performance-info"><span id="sample-info">Waiting for sound</span><span id="fps" title="Measured rendering frames per second">— fps</span></div></div>
<div class="plots">
<article class="plot main-plot" id="main-plot">
<div class="plot-heading"><div><h2 id="main-title">Acoustic constellation</h2><p id="main-description">A living architecture of sound</p></div><div class="plot-actions"><button id="reset-camera" aria-label="Reset camera" title="Reset camera">↺</button><button id="fullscreen" aria-label="Toggle fullscreen" title="Fullscreen">⛶</button></div></div>
<div id="main-canvas" class="canvas-host" aria-label="Interactive 3D audio plot. Drag to rotate and scroll to zoom."></div>
<div class="scene-hud"><span>Amplitude <b id="hud-amplitude">—</b></span><span>Time <b id="hud-time">—</b></span></div>
<div class="scene-caption"><span id="node-count">0 points</span><span>Labels: amplitude / frequency</span></div>
<div class="plot-footer"><span id="main-axis">Frequency, texture &amp; amplitude · feature projection</span><span class="legend"><i></i> position → colour</span></div><div class="camera-hint">Drag to orbit <span>·</span> Scroll to zoom</div>
</article>
<div class="secondary-plots"><article class="plot small-plot"><div class="plot-heading"><div><h2>Tone map</h2><p>Frequency leaves a fingerprint</p></div><span class="mini-badge">Hz</span></div><div id="tone-canvas" class="canvas-host" aria-label="3D tone map"></div><div class="plot-footer"><span>Frequency × amplitude × time</span><span class="spectrum-line"></span></div></article><article class="plot small-plot"><div class="plot-heading"><div><h2>Modulation space</h2><p>The movement between moments</p></div><span class="mini-badge">Δ</span></div><div id="mod-canvas" class="canvas-host" aria-label="3D modulation map"></div><div class="plot-footer"><span>Frequency change × level change × time</span></div></article></div>
</div>
<div class="viewbar"><div class="view-tabs" role="group" aria-label="Main visualization"><button data-mode="constellation" class="active" aria-pressed="true">Constellation</button><button data-mode="spread" aria-pressed="false">Texture</button><button data-mode="tone" aria-pressed="false">Frequency</button><button data-mode="mod" aria-pressed="false">Modulation</button></div><span class="view-note">Explore the structure inside your sound.</span></div>
<div class="metrics"><div><span>Amplitude</span><strong id="metric-rms">—<small> dB</small></strong></div><div><span>Centroid</span><strong id="metric-centroid">—<small> Hz</small></strong></div><div><span>Spread</span><strong id="metric-spread">—<small> Hz</small></strong></div><div><span>Entropy</span><strong id="metric-entropy">—</strong></div><canvas id="spectrum" aria-label="Live frequency spectrum"></canvas></div>
<div class="player"><button id="play" class="play" aria-label="Play demo">▶</button><div class="track"><strong id="track-name">Blackbird song</strong><span id="track-detail">Diana Tudor · field recording · press play</span></div><div class="timeline"><span id="current-time">0:00</span><input id="seek" aria-label="Seek" type="range" min="0" max="70" step="0.1" value="0" disabled><span id="duration">1:10</span></div><div class="volume"><button id="mute" aria-label="Mute">♪</button><input id="volume" type="range" aria-label="Volume" min="0" max="1" step="0.01" value="0.65"></div></div>
<p id="message" role="status" aria-live="polite"></p>
</section>
</main>
<footer class="page-footer"><span>A living study of sound.</span><a href="https://www.lucioarese.net/seeing-birdsong/" target="_blank" rel="noreferrer">Inspired by Lucio Arese’s Seeing Birdsong ↗</a></footer>
</div>
<div id="drop-overlay" hidden><div>Drop your sound here<span>Let’s see what it looks like.</span></div></div>
<dialog id="about"><button id="close-help" aria-label="Close" class="quiet">×</button><h2>Sound, translated into space.</h2><p>Each point is a moment in your audio. Its position comes from the sound’s measured frequency content and amplitude. Older points form a trail through time.</p><p><strong>Constellation</strong> places detected tones into an expressive 3D shape using frequency, amplitude and texture. Floating labels show peak amplitude and frequency. Brighter lines connect successive sound samples, following similar tones where possible; softer lines connect nearby points. These are visual links, not proof of an acoustic relationship. Points stay in place and fade smoothly to zero over the trail length (six seconds by default). The final notes fade out after playback ends. The camera settles around the shape and gently widens as needed until you drag or zoom; Reset camera restores automatic framing.</p><p><strong>Texture</strong> maps spectral entropy (how dispersed the energy is) and spectral spread (frequency width). <strong>Frequency</strong> shows the strongest tones. <strong>Modulation</strong> shows frame-to-frame changes in frequency centroid and amplitude.</p><p>Drag any plot to orbit. Scroll or pinch to zoom. Sensitivity magnifies the display; it does not change your audio. Measurements use the original signal before the volume control.</p><p>This is an interpretation of the reference, not the artist’s original algorithm. The initial preview is illustrative; pressing play switches to real audio analysis. The default demo is a real blackbird field recording. It streams from Wikimedia Commons and needs an internet connection.</p><p>Birdsong: <a href="https://commons.wikimedia.org/wiki/File:Common_Blackbird_song_(Turdus_merula).ogg" target="_blank" rel="noreferrer">Common Blackbird song (Turdus merula)</a> by Diana Tudor, used unmodified under <a href="https://creativecommons.org/licenses/by/4.0/" target="_blank" rel="noreferrer">CC BY 4.0</a>.</p><p>MP3, WAV, M4A, FLAC and MP4 playback depends on your browser’s codecs. Protected streaming songs cannot be opened as local files.</p></dialog>
<script type="module" src="/src/main.js"></script>
</body></html>

1173
package-lock.json generated Normal file

File diff suppressed because it is too large Load Diff

9
package.json Normal file
View File

@@ -0,0 +1,9 @@
{
"name": "sonora-audio-space",
"private": true,
"version": "1.0.0",
"type": "module",
"scripts": { "dev": "vite --host 127.0.0.1", "build": "vite build", "preview": "vite preview --host 127.0.0.1", "test": "node --test tests/*.test.js" },
"dependencies": { "three": "^0.183.0" },
"devDependencies": { "vite": "^7.3.1" }
}

38
src/analysis.js Normal file
View File

@@ -0,0 +1,38 @@
export function analyzeSpectrum(db, waveform, sampleRate) {
const n = db.length;
const step = sampleRate / (n * 2);
const power = new Float64Array(n);
let total = 0, weighted = 0, squares = 0;
for (const value of waveform) if (Number.isFinite(value)) squares += value * value;
const rms = Math.sqrt(squares / (waveform.length || 1));
for (let i = 0; i < n; i++) {
power[i] = Number.isFinite(db[i]) ? 10 ** (Math.min(0, db[i]) / 10) : 0;
total += power[i]; weighted += power[i] * i * step;
}
if (total < 1e-12) return { rms, centroid: 0, spread: 0, entropy: 0, flatness: 0, peaks: [] };
const centroid = weighted / total;
let variance = 0, entropy = 0, logSum = 0;
const peaks = [];
for (let i = 0; i < n; i++) {
const p = power[i] / total;
variance += p * (i * step - centroid) ** 2;
if (p > 0) entropy -= p * Math.log2(p);
logSum += Math.log(Math.max(power[i], 1e-30));
if (i > 0 && power[i] > (power[i - 1] || 0) && power[i] >= (power[i + 1] || 0) && db[i] > -75) {
peaks.push({ frequency: i * step, amplitude: Math.sqrt(power[i]), power: power[i] });
}
}
peaks.sort((a, b) => b.power - a.power);
return { rms, centroid, spread: Math.sqrt(variance), entropy: n > 1 ? entropy / Math.log2(n) : 0,
flatness: Math.min(1, Math.exp(logSum / n) / (total / n)), peaks: peaks.slice(0, 12) };
}
// A deterministic illustration for the empty state, never presented as measured audio.
export function previewFrame(t) {
const phrase = (Math.sin(t * 1.7) + 1) / 2;
const centroid = 1900 + Math.sin(t * 2.1) * 1200 + phrase * 1700;
const rms = 0.04 + phrase * 0.12 + Math.abs(Math.sin(t * 7)) * 0.07;
return { rms, centroid, spread: 300 + (Math.sin(t * 3.2) + 1) * 1000,
entropy: 0.25 + phrase * 0.26 + Math.sin(t * 4.5) * 0.12, flatness: phrase * 0.2,
peaks: Array.from({ length: 6 }, (_, i) => ({ frequency: centroid * (0.5 + i * 0.35), amplitude: rms * (1 - i / 7) })) };
}

16
src/color.js Normal file
View File

@@ -0,0 +1,16 @@
const clamp = (value) => Math.max(0, Math.min(1, value));
// A fixed color field in plot coordinates: orbiting the camera never recolors data.
export function colorStyle(x, y, z, intensity, palette, mode) {
const amplitude = clamp(intensity);
const spatial = mode === 'position';
const height = clamp(y / 7), depth = clamp((z + 6) / 12);
const v = spatial ? clamp(0.5 + 0.75 * x / 6 + 0.32 * (height - 0.5) + 0.38 * z / 6) : amplitude;
let h, s, l;
if (palette === 'spectrum') { h = 0.76 - v * 0.76; s = 0.95; l = 0.58; }
else if (palette === 'glacier') { h = 0.64 - v * 0.17; s = 0.8; l = 0.4 + v * 0.4; }
else if (palette === 'orchid') { h = 0.7 + v * 0.23; s = 0.85; l = 0.48 + v * 0.25; }
else { h = 0.005 + v * 0.15; s = 0.95; l = 0.43 + v * 0.35; }
if (spatial) { s *= 0.88 + depth * 0.12; l = clamp(l + (height - 0.5) * 0.08); }
return { h, s, l, energy: spatial ? 0.55 + amplitude * 0.65 : 1 };
}

137
src/constellation-data.js Normal file
View File

@@ -0,0 +1,137 @@
const clamp = (v, min, max) => Math.max(min, Math.min(max, v));
export const NODE_LIMIT = 2400;
const CELL_SIZE = 2.8;
const distanceSquared = (a,b) => (a[0]-b[0])**2+(a[1]-b[1])**2+(a[2]-b[2])**2;
const cellKey = (x,y,z) => `${x},${y},${z}`;
function projectPeak(frame, peak, sensitivity) {
const frequencyUnit=clamp(Math.log2(Math.max(30,peak.frequency)/30)/Math.log2(20000/30),0,1);
const level=clamp(Math.sqrt(peak.amplitude*sensitivity),0,1);
const entropy=clamp(frame.entropy||0,0,1), spread=clamp((frame.spread||0)/5000,0,1);
const angle=frequencyUnit*Math.PI*3.5+entropy*2.8+frame.time*.12;
const radius=1+level*7+entropy*2;
return {frequencyUnit,level,position:[
Math.sin(angle)*radius+Math.sin(spread*5)*1.3,
3.5+Math.cos(angle*1.23+spread*2)*radius*.66+(level-.3)*2.5,
Math.sin(angle*1.71+entropy*3.2)*radius*.8+(entropy-.5)*4,
]};
}
class SpatialIndex {
constructor(){this.cells=new Map(); this.nodes=new Map();}
add(node){
const cell=node.position.map(v=>Math.floor(v/CELL_SIZE));
node.cell=cellKey(...cell);
if(!this.cells.has(node.cell))this.cells.set(node.cell,new Set());
this.cells.get(node.cell).add(node.id); this.nodes.set(node.id,node);
}
remove(node){const cell=this.cells.get(node.cell);cell?.delete(node.id);if(!cell?.size)this.cells.delete(node.cell);this.nodes.delete(node.id);}
nearest(position,exclude,limit=2){
const cell=position.map(v=>Math.floor(v/CELL_SIZE));
// Keep only the few closest candidates; no per-node sorting or temporary bucket copies.
const best=[];
for(let dx=-1;dx<=1;dx++)for(let dy=-1;dy<=1;dy++)for(let dz=-1;dz<=1;dz++){
const bucket=this.cells.get(cellKey(cell[0]+dx,cell[1]+dy,cell[2]+dz));
if(!bucket)continue;
let examined=0;
for(const id of bucket){
if(++examined>24)break;
if(id===exclude)continue;
const node=this.nodes.get(id), distance=distanceSquared(position,node.position);
if(distance<=.006||distance>=CELL_SIZE**2)continue;
let at=0;while(at<best.length&&best[at].distance<distance)at++;
if(at<limit){best.splice(at,0,{id,distance});if(best.length>limit)best.pop();}
}
}
return best.map(candidate=>candidate.id);
}
}
// Tracks detected tones by frequency, independently of their rank by loudness.
// Each historical point and its parent links are created once, then remain fixed.
export class ConstellationModel {
constructor(){this.revision=0;this.reset();}
reset(){
this.history=null;this.geometryKey='';this.lastTime=-Infinity;this.lastFrameTime=-Infinity;
this.lastSampleTime=-Infinity;this.nextId=0;this.nextTrackId=0;
this.nodes=[];this.links=[];this.tracks=[];this.previousFrame=[];this.spatial=new SpatialIndex();this.revision++;
}
update(history,now,settings){
const geometryKey=`${settings.sensitivity}:${settings.trail}`;
if(history!==this.history||geometryKey!==this.geometryKey||now<this.lastTime-.05){
this.reset();this.history=history;this.geometryKey=geometryKey;
}
this.lastTime=now;
let changed=false;
const cutoff=now-settings.trail;
let expired=0;
while(expired<this.nodes.length&&this.nodes[expired].time<=cutoff){this.spatial.remove(this.nodes[expired++]);}
if(expired){this.nodes=this.nodes.slice(expired);changed=true;}
const interval=Math.max(1/30,settings.trail*6/(NODE_LIMIT-24));
for(const frame of history){
if(frame.time<=this.lastFrameTime||frame.time>now)continue;
this.lastFrameTime=frame.time;
if(frame.time<=cutoff||frame.rms<.00001||frame.time-this.lastSampleTime<interval-.001)continue;
this.lastSampleTime=frame.time;
this.tracks=this.tracks.filter(track=>frame.time-track.time<.3);
const matched=new Set();
const currentFrame=[];
for(const peak of frame.peaks.slice(0,6)){
if(!Number.isFinite(peak.frequency)||!Number.isFinite(peak.amplitude)||peak.frequency<=0||peak.amplitude<=0)continue;
let track=null,best=.38;
for(const candidate of this.tracks){
if(matched.has(candidate.id))continue;
const difference=Math.abs(Math.log2(peak.frequency/candidate.frequency));
if(difference<best){best=difference;track=candidate;}
}
const projected=projectPeak(frame,peak,settings.sensitivity);
let previousId=null;
if(track){
const blend=1-Math.exp(-(frame.time-track.time)/.14);
projected.position=projected.position.map((v,i)=>track.position[i]+(v-track.position[i])*blend);
previousId=track.nodeId;
}else{track={id:this.nextTrackId++};this.tracks.push(track);}
matched.add(track.id);
const node={...projected,id:this.nextId++,trackId:track.id,frequency:peak.frequency,amplitude:peak.amplitude,time:frame.time,parents:[],trailParent:null};
let previous=this.spatial.nodes.get(previousId);
// A call can jump farther in pitch than tone matching allows. Continue its
// visual path from the nearest sample in the preceding frame instead.
if(!previous){
let nearest=Infinity;
for(const candidate of this.previousFrame){
if(frame.time-candidate.time>.3||!this.spatial.nodes.has(candidate.id))continue;
const distance=distanceSquared(candidate.position,node.position);
if(distance<nearest){nearest=distance;previous=candidate;}
}
}
if(previous&&frame.time-previous.time<=.3){
node.parents.push(previous.id);node.trailParent=previous.id;
}
node.parents.push(...this.spatial.nearest(node.position,node.trailParent,2));
Object.assign(track,{frequency:peak.frequency,time:frame.time,position:node.position,nodeId:node.id});
this.nodes.push(node);currentFrame.push(node);this.spatial.add(node);changed=true;
}
this.previousFrame=currentFrame;
}
while(this.nodes.length>NODE_LIMIT){this.spatial.remove(this.nodes.shift());changed=true;}
if(changed){
const indexes=new Map(this.nodes.map((node,i)=>[node.id,i]));
const links=[];
for(let i=0;i<this.nodes.length;i++){
const node=this.nodes[i];
for(const id of node.parents){const parent=indexes.get(id);if(parent!==undefined)links.push([parent,i,id===node.trailParent]);}
}
this.links=links;this.revision++;
}
return {nodes:this.nodes,links:this.links,revision:this.revision};
}
}
export function makeNodes(history,now,settings){
return new ConstellationModel().update(history,now,settings).nodes;
}
export function connectNodes(nodes){
const index=new SpatialIndex(),links=[];
nodes.forEach((node,i)=>{for(const id of index.nearest(node.position,null,3))links.push([id,i]);index.add({...node,id:i});});
return links;
}

161
src/constellation.js Normal file
View File

@@ -0,0 +1,161 @@
import * as THREE from 'three';
import { ConstellationModel, NODE_LIMIT } from './constellation-data.js';
import { colorStyle } from './color.js';
const LABEL_LIMIT=24;
// Life is animated on the GPU at display refresh rate, independently of audio sampling.
const lifeShader=`
float lifeFade(float born,float expires,float now){
float arrive=smoothstep(0.,.18,now-born);
float age=clamp((now-born)/max(.01,expires-born),0.,1.);
return arrive*pow(1.-age,1.6);
}`;
export class ConstellationLayer extends THREE.Group {
constructor(){
super();this.model=new ConstellationModel();
this.geometry=new THREE.BufferGeometry();
this.positions=new Float32Array(NODE_LIMIT*3);this.colors=new Float32Array(NODE_LIMIT*3);
this.sizes=new Float32Array(NODE_LIMIT);this.births=new Float32Array(NODE_LIMIT);
for(const [name,array,size] of [['position',this.positions,3],['color',this.colors,3],['pointSize',this.sizes,1],['born',this.births,1]]){
this.geometry.setAttribute(name,new THREE.BufferAttribute(array,size).setUsage(THREE.DynamicDrawUsage));
}
this.geometry.setDrawRange(0,0);
this.uniforms={resolution:{value:600},glow:{value:1.35},now:{value:0},trail:{value:12}};
this.material=new THREE.ShaderMaterial({vertexColors:true,transparent:true,depthWrite:false,blending:THREE.AdditiveBlending,uniforms:this.uniforms,
vertexShader:`attribute float pointSize; attribute float born; uniform float resolution; uniform float now; uniform float trail; varying vec3 vColor; varying float vFade;
${lifeShader}
void main(){vColor=color;vFade=lifeFade(born,born+trail,now);vec4 p=modelViewMatrix*vec4(position,1.);
gl_PointSize=clamp(pointSize*resolution/-p.z,2.,32.);gl_Position=projectionMatrix*p;}`,
fragmentShader:`varying vec3 vColor;varying float vFade;uniform float glow;
void main(){vec2 uv=abs(gl_PointCoord-.5)*2.;float edge=max(uv.x,uv.y);
float ring=(1.-smoothstep(.65,.76,edge))*smoothstep(.44,.55,edge);
float core=1.-smoothstep(.08,.22,length(uv));float halo=exp(-dot(uv,uv)*4.)*(1.-smoothstep(.8,1.,edge));
gl_FragColor=vec4(vColor,vFade*(ring*.85+core*.85+halo*.12*glow));}`,
});
this.points=new THREE.Points(this.geometry,this.material);this.points.frustumCulled=false;this.add(this.points);
// Screen-width ribbons stay readable on high-density displays; WebGL native
// lines are only one physical pixel wide on many devices.
this.lineGeometry=new THREE.InstancedBufferGeometry();
this.lineGeometry.setAttribute('position',new THREE.Float32BufferAttribute([0,-1,0,0,1,0,1,-1,0,1,1,0],3));
this.lineGeometry.setIndex([0,2,1,2,3,1]);
this.linePositions=new Float32Array(NODE_LIMIT*18);this.lineColors=new Float32Array(this.linePositions.length);this.lineLives=new Float32Array(NODE_LIMIT*12);
const positions=new THREE.InstancedInterleavedBuffer(this.linePositions,6).setUsage(THREE.DynamicDrawUsage);
const colors=new THREE.InstancedInterleavedBuffer(this.lineColors,6).setUsage(THREE.DynamicDrawUsage);
const lives=new THREE.InstancedInterleavedBuffer(this.lineLives,4).setUsage(THREE.DynamicDrawUsage);
this.lineBuffers=[positions,colors,lives];
for(const [name,buffer,size,offset] of [['start',positions,3,0],['end',positions,3,3],['startColor',colors,3,0],['endColor',colors,3,3],['life',lives,2,0]]){
this.lineGeometry.setAttribute(name,new THREE.InterleavedBufferAttribute(buffer,size,offset));
}
this.lineGeometry.instanceCount=0;
this.lines=new THREE.Mesh(this.lineGeometry,new THREE.ShaderMaterial({side:THREE.DoubleSide,transparent:true,depthWrite:false,blending:THREE.AdditiveBlending,uniforms:{now:this.uniforms.now,resolution:this.uniforms.resolution},
vertexShader:`attribute vec3 start;attribute vec3 end;attribute vec3 startColor;attribute vec3 endColor;
attribute vec2 life;uniform float now;uniform float resolution;varying vec3 vColor;varying float vFade;varying float vSide;
${lifeShader}
void main(){
vec4 a=projectionMatrix*modelViewMatrix*vec4(start,1.);
vec4 b=projectionMatrix*modelViewMatrix*vec4(end,1.);
float aspect=projectionMatrix[1][1]/projectionMatrix[0][0];
vec2 direction=(b.xy/max(b.w,.001)-a.xy/max(a.w,.001))*vec2(aspect,1.);
vec2 normal=vec2(-direction.y,direction.x)/max(length(direction),.00001);
vec4 p=mix(a,b,position.x);
p.xy+=normal/vec2(aspect,1.)*position.y*3./resolution*p.w;
vColor=mix(startColor,endColor,position.x);vSide=position.y;
vFade=lifeFade(life.x,life.y,now)*step(.1,min(a.w,b.w));gl_Position=p;
}`,
fragmentShader:'varying vec3 vColor;varying float vFade;varying float vSide;void main(){float edge=1.-smoothstep(.55,1.,abs(vSide));gl_FragColor=vec4(vColor,vFade*edge*.9);}',
}));
this.lines.frustumCulled=false;this.add(this.lines);
this.annotations=new THREE.Group();this.add(this.annotations);this.labelPool=[];
for(let i=0;i<LABEL_LIMIT;i++){
const canvas=document.createElement('canvas');canvas.width=256;canvas.height=96;
const texture=new THREE.CanvasTexture(canvas);texture.colorSpace=THREE.SRGBColorSpace;
const sprite=new THREE.Sprite(new THREE.SpriteMaterial({map:texture,transparent:true,opacity:0,depthTest:false,depthWrite:false}));
sprite.visible=false;this.annotations.add(sprite);this.labelPool.push({canvas,texture,sprite,node:null});
}
this.color=new THREE.Color();this.nodeCount=0;this.linkCount=0;this.lastRevision=-1;this.lastLabelTime=-Infinity;
}
update(history,now,settings,resolution){
const {nodes,links,revision}=this.model.update(history,now,settings);
this.uniforms.resolution.value=resolution;this.uniforms.glow.value=settings.glow;this.uniforms.trail.value=settings.trail;
this.lines.visible=settings.connections;this.annotations.visible=settings.labels;
const appearanceKey=`${settings.palette}:${settings.colorMode}:${settings.glow}`;
const geometryChanged=revision!==this.lastRevision;
const appearanceChanged=appearanceKey!==this.appearanceKey;
if(geometryChanged||appearanceChanged){
for(let i=0;i<nodes.length;i++){
const node=nodes[i],[x,y,z]=node.position;
const intensity=settings.colorMode==='frequency'?node.frequencyUnit:node.level*1.8;
const style=colorStyle(x,y,z,intensity,settings.palette,settings.colorMode);
this.color.setHSL(style.h,style.s,Math.min(style.l,.64)).multiplyScalar(style.energy);
this.positions.set(node.position,i*3);
const at=i*3;this.colors[at]=this.color.r;this.colors[at+1]=this.color.g;this.colors[at+2]=this.color.b;
this.sizes[i]=(.16+node.level*.58)*(1+settings.glow*.22);this.births[i]=node.time;
}
this.geometry.setDrawRange(0,nodes.length);
for(const attribute of Object.values(this.geometry.attributes))attribute.needsUpdate=true;
for(let i=0;i<links.length;i++){
const [a,b,temporal]=links[i],offset=i*6,strength=temporal?1.15:.45;
this.linePositions.set(nodes[a].position,offset);this.linePositions.set(nodes[b].position,offset+3);
for(let axis=0;axis<3;axis++){
this.lineColors[offset+axis]=this.colors[a*3+axis]*strength;
this.lineColors[offset+3+axis]=this.colors[b*3+axis]*strength;
}
const born=Math.max(nodes[a].time,nodes[b].time),expires=Math.min(nodes[a].time,nodes[b].time)+settings.trail;
this.lineLives.set([born,expires,born,expires],i*4);
}
this.lineGeometry.instanceCount=links.length;
for(const buffer of this.lineBuffers)buffer.needsUpdate=true;
this.appearanceKey=appearanceKey;
}
if(geometryChanged){
this.updateBounds(nodes);
// Keep each label on its original point until it expires. No re-ranking/reassignment flicker.
const byId=new Map(nodes.map(node=>[node.id,node]));
for(const label of this.labelPool)if(label.node&&!byId.has(label.node.id)){label.node=null;label.sprite.visible=false;}
if(this.lastHistory!==history||this.lastGeometryKey!==this.model.geometryKey){
for(const label of this.labelPool){label.node=null;label.sprite.visible=false;}
this.lastLabelTime=-Infinity;
}
this.lastHistory=history;this.lastGeometryKey=this.model.geometryKey;
}
if(settings.labels&&(geometryChanged||!this.lastLabels))this.updateLabels(nodes,now,settings.trail);
this.lastLabels=settings.labels;this.lastRevision=revision;this.nodeCount=nodes.length;this.linkCount=links.length;
this.animate(now);return nodes.length;
}
updateBounds(nodes){
if(!nodes.length){this.frameBounds=null;return;}
const min=[Infinity,Infinity,Infinity],max=[-Infinity,-Infinity,-Infinity];
for(const node of nodes)for(let axis=0;axis<3;axis++){min[axis]=Math.min(min[axis],node.position[axis]);max[axis]=Math.max(max[axis],node.position[axis]);}
this.frameBounds={center:new THREE.Vector3(...min.map((v,i)=>(v+max[i])/2)),radius:Math.max(1.4,Math.hypot(...min.map((v,i)=>(max[i]-v)/2)))};
}
updateLabels(nodes,now,trail){
if(now-this.lastLabelTime<.4)return;
this.lastLabelTime=now;
const used=new Set(this.labelPool.filter(label=>label.node).map(label=>label.node.id));
const chosen=this.labelPool.filter(label=>label.node).map(label=>label.node.position);
const candidates=nodes.filter(node=>!used.has(node.id)&&now-node.time<trail*.75).sort((a,b)=>b.level-a.level);
for(const label of this.labelPool){
if(label.node)continue;
const at=candidates.findIndex(node=>chosen.every(p=>Math.hypot(...p.map((v,i)=>v-node.position[i]))>1.35));
if(at<0)break;
const [node]=candidates.splice(at,1);label.node=node;chosen.push(node.position);
const ctx=label.canvas.getContext('2d');ctx.clearRect(0,0,256,96);ctx.textAlign='center';
ctx.font='500 29px monospace';ctx.fillStyle='#d9e1e9';ctx.fillText(node.amplitude.toFixed(4),128,34);
ctx.font='22px monospace';ctx.fillStyle='#8d97a8';ctx.fillText(`${(node.frequency/1000).toFixed(2)} kHz`,128,66);
label.texture.needsUpdate=true;label.sprite.position.set(node.position[0],node.position[1]+.33,node.position[2]);
const size=1.5+node.level;label.sprite.scale.set(size,size*.375,1);label.sprite.visible=true;
}
}
animate(now){
this.uniforms.now.value=now;
if(!this.annotations.visible)return;
const trail=this.uniforms.trail.value;
for(const label of this.labelPool){
if(!label.node)continue;
const age=now-label.node.time;
const arrive=Math.min(1,Math.max(0,age/.18));
label.sprite.material.opacity=.8*arrive*Math.max(0,1-age/trail)**1.6;
}
}
}

7
src/demo.js Normal file
View File

@@ -0,0 +1,7 @@
// Common Blackbird song (Turdus merula), recorded by Diana Tudor.
// Unmodified recording, CC BY 4.0; attribution also appears in the help panel.
export const birdsong = {
url: 'https://upload.wikimedia.org/wikipedia/commons/3/30/Common_Blackbird_song_%28Turdus_merula%29.ogg',
name: 'Blackbird song',
detail: 'Field recording · Diana Tudor · CC BY 4.0',
};

230
src/main.js Normal file
View File

@@ -0,0 +1,230 @@
import './style.css';
import { AudioSpace } from './space.js';
import { analyzeSpectrum, previewFrame } from './analysis.js';
import { birdsong } from './demo.js';
const $ = (id) => document.getElementById(id);
const defaultOrbit = !window.matchMedia('(prefers-reduced-motion: reduce)').matches;
const settings = { sensitivity: 1, trail: 6, palette: 'spectrum', colorMode: 'position', glow: 1.35, rotate: defaultOrbit, connections: true, labels: true };
const media = new Audio(); media.preload = 'metadata'; media.crossOrigin = 'anonymous';
let context, analyser, gain, source, db, waveform, url, loaded = false;
let history = [], visualTime = 12, previousFeature = null, lastSample = 0, lastMetrics = 0;
let dirty = true, preview = true, loadVersion = 0, lastVolume = 0.65;
let spaces = [];
const message = (text) => { $('message').textContent = text; };
const time = (seconds) => { const s = Number.isFinite(seconds) ? Math.max(0, seconds) : 0; return `${Math.floor(s / 60)}:${String(Math.floor(s % 60)).padStart(2, '0')}`; };
function clearHistory() { history = []; previousFeature = null; dirty = true; }
function addFrame(f, t) {
f.time = t;
f.deltaFrequency = previousFeature ? f.centroid - previousFeature.centroid : 0;
f.deltaAmplitude = previousFeature ? f.rms - previousFeature.rms : 0;
previousFeature = f; history.push(f);
// Keep a bounded 30-second history even if a tab is left playing for hours.
while (history.length > 910 || (history.length && t - history[0].time > 30)) history.shift();
dirty = true;
}
try {
spaces = [new AudioSpace($('main-canvas'), 'constellation'), new AudioSpace($('tone-canvas'), 'tone', true), new AudioSpace($('mod-canvas'), 'mod', true)];
for (let i = 0; i <= 360; i++) addFrame(previewFrame(i / 30), i / 30);
} catch (error) {
message('3D graphics are unavailable. Enable hardware acceleration or try Chrome, Edge or Safari.');
console.error(error);
}
function initAudio() {
if (context) return;
const AudioContextClass = window.AudioContext || window.webkitAudioContext;
if (!AudioContextClass) throw new Error('This browser does not support audio analysis. Try a current browser.');
context = new AudioContextClass(); analyser = context.createAnalyser(); gain = context.createGain();
analyser.fftSize = 4096; analyser.smoothingTimeConstant = 0.55;
source = context.createMediaElementSource(media);
source.connect(analyser); analyser.connect(gain); gain.connect(context.destination);
gain.gain.value = Number($('volume').value);
db = new Float32Array(analyser.frequencyBinCount); waveform = new Float32Array(analyser.fftSize);
}
function syncPlayer() {
const playing = !media.paused && !media.ended;
document.body.classList.toggle('playing', playing);
$('play').textContent = playing ? 'Ⅱ' : '▶';
$('play').setAttribute('aria-label', playing ? 'Pause' : loaded ? 'Play' : 'Play demo');
if (loaded) $('status').textContent = playing ? 'Analyzing live audio' : media.ended ? 'Playback complete' : 'Paused';
}
function loadBlob(blob, name, detail) { loadTrack(URL.createObjectURL(blob), name, detail, true); }
function loadTrack(audioUrl, name, detail, local = false) {
loadVersion++;
media.pause();
if (url) URL.revokeObjectURL(url);
url = local ? audioUrl : null; media.src = audioUrl;
loaded = true; preview = false;
clearHistory(); visualTime = 0; lastSample = 0;
$('track-name').textContent = name; $('track-detail').textContent = detail;
$('status').textContent = 'Loading audio…'; $('seek').disabled = true;
$('seek').value = 0; $('current-time').textContent = '0:00'; $('duration').textContent = '—';
$('sample-info').textContent = local ? 'Local audio' : 'Birdsong recording'; resetMetrics(); message(''); media.load();
}
function loadDemo() { loadTrack(birdsong.url, birdsong.name, birdsong.detail); }
async function play() {
const version = loadVersion;
try {
initAudio(); await context.resume();
if (version !== loadVersion) return;
if (media.ended) { media.currentTime = 0; clearHistory(); }
await media.play();
if (version !== loadVersion) return;
message('');
} catch (error) {
if (version !== loadVersion || error.name === 'AbortError') return;
message(error.name === 'NotAllowedError' ? 'Press play to allow audio playback.' : media.src === birdsong.url ? 'Birdsong could not load. Check your internet connection, or open a song.' : 'This file could not be played. Try an MP3 or WAV file.');
syncPlayer();
}
}
async function togglePlay() {
if (!loaded) loadDemo();
if (media.paused) await play(); else media.pause();
}
function loadFile(file) {
if (!file) return;
if (!file.size) { message('This file is empty. Choose another audio file.'); return; }
if (!/^(audio\/|video\/)/.test(file.type) && !/\.(mp3|wav|m4a|flac|ogg|aac|mp4|webm|aiff|aif)$/i.test(file.name)) {
message('Choose an audio file, such as MP3, WAV or M4A, or an MP4 video.'); return;
}
loadBlob(file, file.name.replace(/\.[^.]+$/, ''), `${file.name.split('.').pop().toUpperCase()} · ${(file.size / 1048576).toFixed(1)} MB · on your device`);
void play();
}
$('upload').addEventListener('click', () => $('file').click());
$('file').addEventListener('change', (event) => { loadFile(event.target.files[0]); event.target.value = ''; });
$('demo').addEventListener('click', () => { loadDemo(); void play(); });
$('play').addEventListener('click', togglePlay);
media.addEventListener('loadedmetadata', () => {
if (!Number.isFinite(media.duration) || media.duration <= 0) { message('No playable duration was found. Try another file.'); return; }
$('seek').max = media.duration; $('seek').disabled = false; $('duration').textContent = time(media.duration);
syncPlayer();
});
for (const event of ['play', 'pause', 'ended']) media.addEventListener(event, syncPlayer);
media.addEventListener('error', () => {
$('seek').disabled = true;
message(media.src === birdsong.url ? 'Birdsong could not load. Check your internet connection, or open a song.' : 'Your browser could not decode this file. Try an unprotected MP3 or WAV.'); $('status').textContent = 'Unable to play file';
});
media.addEventListener('seeking', () => { clearHistory(); visualTime = media.currentTime; });
media.addEventListener('timeupdate', () => {
$('current-time').textContent = time(media.currentTime); $('seek').value = media.currentTime;
});
$('seek').addEventListener('input', (event) => { if (loaded && Number.isFinite(media.duration)) { media.currentTime = Number(event.target.value); clearHistory(); $('current-time').textContent = time(media.currentTime); } });
function setVolume(value) {
$('volume').value = value;
if (gain) gain.gain.setTargetAtTime(value, context.currentTime, 0.015);
$('mute').textContent = value === 0 ? '×' : '♪'; $('mute').setAttribute('aria-label', value === 0 ? 'Unmute' : 'Mute');
}
$('volume').addEventListener('input', (event) => { const value = Number(event.target.value); if (value) lastVolume = value; setVolume(value); });
$('mute').addEventListener('click', () => { const current = Number($('volume').value); if (current) lastVolume = current; setVolume(current ? 0 : lastVolume); });
$('sensitivity').addEventListener('input', (event) => { settings.sensitivity = Number(event.target.value); $('sensitivity-value').textContent = `${settings.sensitivity.toFixed(1)}×`; dirty = true; });
$('trail').addEventListener('input', (event) => { settings.trail = Number(event.target.value); $('trail-value').textContent = `${settings.trail} s`; dirty = true; });
for (const id of ['rotate', 'connections', 'labels']) $(id).addEventListener('change', (event) => { settings[id] = event.target.checked; dirty = true; });
function updateColorMode() {
settings.colorMode = $('color-mode').value; dirty = true;
const spatial = settings.colorMode === 'position';
$('color-description').textContent = spatial ? 'Colour follows position. Brightness follows sound.' : settings.colorMode === 'frequency' ? 'Low tones are violet. High tones are warm.' : 'Colour and brightness follow the sound level.';
document.querySelector('.legend').lastChild.textContent = spatial ? ' position → colour' : settings.colorMode === 'frequency' ? ' low → high frequency' : ' soft to loud';
}
$('color-mode').addEventListener('change', updateColorMode);
$('glow').addEventListener('input', (event) => { settings.glow = Number(event.target.value); $('glow-value').textContent = `${settings.glow.toFixed(2)}×`; dirty = true; });
function selectPalette(palette) {
settings.palette = palette; dirty = true;
$('palette-name').textContent = palette[0].toUpperCase() + palette.slice(1);
for (const button of document.querySelectorAll('[data-palette]')) { const selected = button.dataset.palette === palette; button.classList.toggle('selected', selected); button.setAttribute('aria-pressed', String(selected)); }
document.querySelector('.legend i').style.background = { ember: 'linear-gradient(90deg,#cb322c,#fff9b6)', spectrum: 'linear-gradient(90deg,#602cde,#3dc6c6,#fdb84a)', glacier: 'linear-gradient(90deg,#354bc1,#b6fbff)', orchid: 'linear-gradient(90deg,#6024c7,#f8c5ed)' }[palette];
}
document.querySelectorAll('[data-palette]').forEach((button) => button.addEventListener('click', () => selectPalette(button.dataset.palette)));
$('reset-settings').addEventListener('click', () => {
Object.assign(settings, { sensitivity: 1, trail: 6, glow: 1.35, rotate: defaultOrbit, connections: true, labels: true });
$('sensitivity').value = 1; $('sensitivity-value').textContent = '1.0×'; $('trail').value = 6; $('trail-value').textContent = '6 s';
$('glow').value = 1.35; $('glow-value').textContent = '1.35×'; $('color-mode').value = 'position'; updateColorMode();
$('rotate').checked = defaultOrbit; $('connections').checked = true; $('labels').checked = true; selectPalette('spectrum'); spaces.forEach(space => space.reset());
});
const modes = { constellation: ['Acoustic constellation', 'A living architecture of sound', 'Frequency, texture & amplitude · feature projection'], spread: ['Spread–entropy space', 'The evolving texture of a sound', 'Entropy × spectral spread × time'], tone: ['Frequency space', 'The strongest tones, suspended in time', 'Frequency × amplitude × time'], mod: ['Modulation space', 'The movement between moments', 'Frequency change × level change × time'] };
document.querySelectorAll('[data-mode]').forEach((button) => button.addEventListener('click', () => {
const mode = button.dataset.mode; spaces[0]?.setMode(mode); dirty = true;
document.querySelector('.workspace').classList.toggle('constellation-view', mode === 'constellation');
[$('main-title').textContent, $('main-description').textContent, $('main-axis').textContent] = modes[mode];
document.querySelectorAll('[data-mode]').forEach(b => { const selected = b === button; b.classList.toggle('active', selected); b.setAttribute('aria-pressed', String(selected)); });
}));
$('reset-camera').addEventListener('click', () => spaces.forEach(space => space.reset()));
$('fullscreen').addEventListener('click', async () => {
try { if (document.fullscreenElement) await document.exitFullscreen(); else if ($('main-plot').requestFullscreen) await $('main-plot').requestFullscreen(); else message('Fullscreen is unavailable in this browser.'); } catch { message('Fullscreen is unavailable in this browser window.'); }
});
$('help').addEventListener('click', () => $('about').showModal()); $('close-help').addEventListener('click', () => $('about').close());
document.addEventListener('keydown', (event) => {
if (event.code === 'Space' && !['INPUT', 'BUTTON', 'TEXTAREA', 'A'].includes(event.target.tagName) && !$('about').open) { event.preventDefault(); void togglePlay(); }
});
let dragDepth = 0;
document.addEventListener('dragenter', (event) => { if (event.dataTransfer.types.includes('Files')) { event.preventDefault(); dragDepth++; $('drop-overlay').hidden = false; } });
document.addEventListener('dragover', (event) => { if (event.dataTransfer.types.includes('Files')) event.preventDefault(); });
document.addEventListener('dragleave', () => { if (--dragDepth <= 0) { dragDepth = 0; $('drop-overlay').hidden = true; } });
document.addEventListener('drop', (event) => { event.preventDefault(); dragDepth = 0; $('drop-overlay').hidden = true; loadFile(event.dataTransfer.files[0]); });
window.addEventListener('blur', () => { dragDepth = 0; $('drop-overlay').hidden = true; });
function resetMetrics() {
$('hud-amplitude').textContent = '—'; $('hud-time').textContent = '—';
$('metric-rms').innerHTML = '—<small> dB</small>'; $('metric-centroid').innerHTML = '—<small> Hz</small>';
$('metric-spread').innerHTML = '—<small> Hz</small>'; $('metric-entropy').textContent = '—';
}
function updateMetrics(f) {
$('hud-amplitude').textContent = f.rms.toFixed(4); $('hud-time').textContent = `${media.currentTime.toFixed(2)} s`;
$('metric-rms').innerHTML = `${f.rms > 1e-6 ? (20 * Math.log10(f.rms)).toFixed(1) : '−∞'}<small> dB</small>`;
$('metric-centroid').innerHTML = `${Math.round(f.centroid).toLocaleString()}<small> Hz</small>`;
$('metric-spread').innerHTML = `${Math.round(f.spread).toLocaleString()}<small> Hz</small>`;
$('metric-entropy').textContent = f.entropy.toFixed(3);
$('sample-info').textContent = `${(context.sampleRate / 1000).toFixed(1)} kHz · live analysis`;
}
const spectrum = $('spectrum'), spectrumCtx = spectrum.getContext('2d');
function drawSpectrum() {
const width = spectrum.clientWidth, height = spectrum.clientHeight;
if (!width) return;
if (spectrum.width !== width * 2 || spectrum.height !== height * 2) { spectrum.width = width * 2; spectrum.height = height * 2; }
spectrumCtx.clearRect(0, 0, spectrum.width, spectrum.height);
for (let i = 0; i < 36; i++) {
const value = db && !preview ? Math.max(0, ((db[Math.floor(2 ** (i / 35 * 10))] || -100) + 85) / 70) : (Math.sin(i * 1.8) * 0.3 + 0.45) * Math.sin(i / 36 * Math.PI);
spectrumCtx.fillStyle = `rgba(211,177,133,${0.25 + value * 0.65})`;
spectrumCtx.fillRect(i * spectrum.width / 36, spectrum.height * (1 - value), Math.max(1, spectrum.width / 36 - 3), spectrum.height * value);
}
}
let lastRender = null, fpsStart = null, fpsFrames = 0;
function frame(timestamp) {
requestAnimationFrame(frame);
if (document.hidden) { lastRender = null; fpsStart = null; fpsFrames = 0; return; }
const delta = lastRender === null ? 1 / 60 : Math.min(.05, Math.max(.001, (timestamp - lastRender) / 1000));
lastRender = timestamp;
if (loaded && !media.seeking) {
if (media.ended) {
// Let the last notes disappear after playback finishes, even with no new FFT frames.
if (visualTime < media.currentTime + settings.trail) {
visualTime = Math.min(media.currentTime + settings.trail, Math.max(visualTime, media.currentTime) + delta);
dirty = true;
}
} else visualTime = media.currentTime;
}
if (analyser && !media.paused && !media.ended && !media.seeking && media.readyState >= 3 && timestamp - lastSample >= 1000 / 30) {
lastSample = timestamp - ((timestamp - lastSample) % (1000 / 30));
analyser.getFloatFrequencyData(db); analyser.getFloatTimeDomainData(waveform);
const f = analyzeSpectrum(db, waveform, context.sampleRate);
// Digital silence carries no texture; do not invent points at the origin.
if (f.rms > 0.00001) addFrame(f, visualTime); else dirty = true;
if (timestamp - lastMetrics > 100) { updateMetrics(f); drawSpectrum(); lastMetrics = timestamp; }
}
if (dirty) {
for (const space of spaces) if (space.host.clientWidth && space.host.clientHeight) space.update(history, visualTime, settings);
const counts = $('main-canvas').dataset;
$('node-count').textContent = `${counts.nodes || 0} points · ${settings.connections ? counts.links || 0 : 0} links`;
dirty = false;
}
for (const space of spaces) space.render(visualTime, delta);
if (fpsStart === null) fpsStart = timestamp;
else fpsFrames++;
if (timestamp - fpsStart >= 1000) {
$('fps').textContent = `${Math.round(fpsFrames * 1000 / (timestamp - fpsStart))} fps`;
fpsStart = timestamp; fpsFrames = 0;
}
}
$('rotate').checked = defaultOrbit; selectPalette(settings.palette); updateColorMode(); drawSpectrum(); requestAnimationFrame(frame);
window.addEventListener('pagehide', (event) => { media.pause(); if (event.persisted) return; if (url) URL.revokeObjectURL(url); context?.close(); spaces.forEach(space => space.dispose()); });

158
src/space.js Normal file
View File

@@ -0,0 +1,158 @@
import * as THREE from 'three';
import { OrbitControls } from 'three/addons/controls/OrbitControls.js';
import { colorStyle } from './color.js';
import { ConstellationLayer } from './constellation.js';
const clamp = (v, lo, hi) => Math.max(lo, Math.min(hi, v));
const MAX_POINTS = 12000;
export class AudioSpace {
constructor(host, mode, compact = false) {
this.host = host; this.mode = mode; this.compact = compact;
this.scene = new THREE.Scene();
this.renderer = new THREE.WebGLRenderer({ antialias: true, alpha: true });
this.renderer.setPixelRatio(Math.min(devicePixelRatio, 2));
this.renderer.setClearColor(0x000000, 0);
host.append(this.renderer.domElement);
this.camera = new THREE.PerspectiveCamera(40, 1, 0.1, 150);
this.controls = new OrbitControls(this.camera, this.renderer.domElement);
this.controls.enableDamping = true; this.controls.enablePan = false;
this.controls.minDistance = 10; this.controls.maxDistance = 65;
this.controls.autoRotateSpeed = 0.25;
this.controls.addEventListener('start', () => { this.autoFrame = false; });
this.frameOffset = new THREE.Vector3();
this.reset();
this.gridGroup = new THREE.Group(); this.scene.add(this.gridGroup);
const grid = new THREE.GridHelper(12, 12, 0x34383c, 0x1b2025);
grid.position.y = -0.02; this.gridGroup.add(grid);
const box = new THREE.EdgesGeometry(new THREE.BoxGeometry(12, 7, 12));
const outline = new THREE.LineSegments(box, new THREE.LineBasicMaterial({ color: 0x272b30, transparent: true, opacity: 0.65 }));
outline.position.y = 3.5; this.gridGroup.add(outline);
this.labels = new THREE.Group(); this.scene.add(this.labels); this.setLabels();
this.geometry = new THREE.BufferGeometry();
this.positions = new Float32Array(MAX_POINTS * 3); this.colors = new Float32Array(MAX_POINTS * 3); this.sizes = new Float32Array(MAX_POINTS);
this.geometry.setAttribute('position', new THREE.BufferAttribute(this.positions, 3).setUsage(THREE.DynamicDrawUsage));
this.geometry.setAttribute('color', new THREE.BufferAttribute(this.colors, 3).setUsage(THREE.DynamicDrawUsage));
this.geometry.setAttribute('pointSize', new THREE.BufferAttribute(this.sizes, 1).setUsage(THREE.DynamicDrawUsage));
this.geometry.setDrawRange(0, 0);
this.material = new THREE.ShaderMaterial({ transparent: true, depthWrite: false, vertexColors: true, blending: THREE.AdditiveBlending,
uniforms: { resolution: { value: 450 }, glow: { value: 1.35 } },
vertexShader: 'attribute float pointSize; varying vec3 vColor; uniform float resolution; void main(){vColor=color; vec4 p=modelViewMatrix*vec4(position,1.); gl_PointSize=clamp(pointSize*resolution/-p.z,1.,38.); gl_Position=projectionMatrix*p;}',
fragmentShader: 'varying vec3 vColor; uniform float glow; void main(){float d=length(gl_PointCoord-0.5)*2.; if(d>1.) discard; float core=exp(-d*d*28.); float halo=exp(-d*d*5.)*(1.-smoothstep(.65,1.,d)); float a=core*.85+halo*.32*glow; vec3 c=mix(vColor,vColor+vec3(.22),core*.3); gl_FragColor=vec4(c,a);}' });
this.points = new THREE.Points(this.geometry, this.material); this.points.frustumCulled = false; this.scene.add(this.points);
this.lineGeo = new THREE.BufferGeometry(); this.linePositions = new Float32Array(1800 * 3); this.lineColors = new Float32Array(1800 * 3);
this.lineGeo.setAttribute('position', new THREE.BufferAttribute(this.linePositions, 3).setUsage(THREE.DynamicDrawUsage));
this.lineGeo.setAttribute('color', new THREE.BufferAttribute(this.lineColors, 3).setUsage(THREE.DynamicDrawUsage)); this.lineGeo.setDrawRange(0, 0);
this.lines = new THREE.LineSegments(this.lineGeo, new THREE.LineBasicMaterial({ vertexColors: true, transparent: true, opacity: 0.5, depthWrite: false, blending: THREE.AdditiveBlending }));
this.lines.frustumCulled = false; this.scene.add(this.lines);
this.color = new THREE.Color();
this.constellation = mode === 'constellation' ? new ConstellationLayer() : null;
if (this.constellation) this.scene.add(this.constellation);
this.resizeObserver = new ResizeObserver(() => this.resize()); this.resizeObserver.observe(host); this.resize();
}
reset() { this.autoFrame = true; this.framingEnvelope = null; this.camera.position.set(this.mode === 'constellation' ? 10 : this.compact ? 14 : 15, this.mode === 'constellation' ? 7 : this.compact ? 11 : 10, this.compact ? 21 : 22); this.controls.target.set(0, this.mode === 'constellation' ? 3.5 : 2.4, 0); this.camera.position.sub(this.controls.target).multiplyScalar(this.fitFactor || 1).add(this.controls.target); this.controls.update(); }
resize() {
const { width, height } = this.host.getBoundingClientRect(); if (!width || !height) return;
this.renderer.setSize(width, height); this.camera.aspect = width / height;
const factor = Math.max(1, 1 / this.camera.aspect);
this.camera.position.sub(this.controls.target).multiplyScalar(factor / (this.fitFactor || 1)).add(this.controls.target);
this.fitFactor = factor; this.camera.updateProjectionMatrix();
this.material && (this.material.uniforms.resolution.value = height * Math.min(devicePixelRatio, 2));
if (this.constellation) this.constellation.uniforms.resolution.value = height * Math.min(devicePixelRatio, 2);
}
label(text, position, width = 4) {
const canvas = document.createElement('canvas'); canvas.width = 768; canvas.height = 96;
const ctx = canvas.getContext('2d'); ctx.font = '30px ui-monospace, monospace'; ctx.fillStyle = '#899099'; ctx.textAlign = 'center'; ctx.fillText(text, 384, 56);
const texture = new THREE.CanvasTexture(canvas); texture.colorSpace = THREE.SRGBColorSpace;
const sprite = new THREE.Sprite(new THREE.SpriteMaterial({ map: texture, transparent: true, depthTest: false }));
sprite.position.set(...position); sprite.scale.set(width, width / 8, 1); this.labels.add(sprite);
}
setLabels() {
for (const child of [...this.labels.children]) { child.material.map.dispose(); child.material.dispose(); this.labels.remove(child); }
if (this.mode === 'constellation') return;
const x = this.mode === 'spread' ? '0 entropy 1' : this.mode === 'tone' ? '20 Hz frequency (log) 20 kHz' : '−1 Δ frequency (kHz) +1';
const y = this.mode === 'spread' ? 'spread · 8 kHz' : this.mode === 'tone' ? 'amplitude' : 'Δ amplitude';
this.label(x, [0, -0.7, 6.8], this.compact ? 8 : 7);
this.label(y, [-6.6, 7.4, 0], 5);
this.label('past ← time → now', [7, -0.35, 0], 5);
}
setMode(mode) { this.mode = mode; this.setLabels(); }
update(history, now, settings) {
const constellation = this.mode === 'constellation';
if (constellation && !this.constellation) { this.constellation = new ConstellationLayer(); this.scene.add(this.constellation); }
if (this.constellation) this.constellation.visible = constellation;
this.gridGroup.visible = !constellation;
this.points.visible = !constellation; this.lines.visible = !constellation && settings.connections && this.mode !== 'tone';
this.controls.autoRotate = settings.rotate;
if (constellation) {
if (this.framingHistory !== history) { this.framingHistory = history; this.framingEnvelope = null; }
const count = this.constellation.update(history, now, settings, this.material.uniforms.resolution.value);
this.host.dataset.nodes = count;
this.host.dataset.links = this.constellation.linkCount;
return;
}
this.material.uniforms.glow.value = settings.glow;
let count = 0, lineCount = 0, previous = null;
const add = (x, y, z, value, age, size, frequency) => {
if (count >= MAX_POINTS) return;
this.positions.set([x, y, z], count * 3);
const colorValue = settings.colorMode === 'frequency' ? clamp(Math.log2(Math.max(30, frequency) / 30) / Math.log2(20000 / 30), 0, 1) : value;
const style = colorStyle(x, y, z, colorValue, settings.palette, settings.colorMode);
const c = this.color.setHSL(style.h, style.s, style.l).multiplyScalar(style.energy);
const fade = 0.16 + 0.84 * (1 - age) ** 0.7;
this.colors.set([c.r * fade, c.g * fade, c.b * fade], count * 3);
this.sizes[count] = size * (0.65 + fade * 0.35) * (1.15 + settings.glow * 0.25); count++;
};
for (const f of history) {
const age = (now - f.time) / settings.trail;
if (age > 1 || age < 0) continue;
const z = 6 - age * 12, gain = settings.sensitivity;
if (this.mode === 'tone') {
for (const peak of f.peaks) {
const x = clamp(Math.log2(Math.max(20, peak.frequency) / 20) / Math.log2(1000), 0, 1) * 12 - 6;
add(x, clamp(Math.sqrt(peak.amplitude * gain) * 8, 0, 7), z, Math.sqrt(peak.amplitude * gain) * 2, age, 0.22 + Math.sqrt(peak.amplitude) * 0.3, peak.frequency);
}
} else {
const x = this.mode === 'spread' ? clamp(f.entropy, 0, 1) * 12 - 6 : clamp(f.deltaFrequency / 1000 * gain, -1, 1) * 6;
const y = this.mode === 'spread' ? clamp(f.spread / 8000 * gain, 0, 1) * 7 : 3.5 + clamp(f.deltaAmplitude * gain * 14, -3.5, 3.5);
add(x, y, z, Math.sqrt(f.rms * gain) * 1.85, age, 0.27 + Math.sqrt(f.rms * gain) * 0.5, f.centroid);
if (previous && f.time - previous.time < 0.2 && lineCount < 1798) {
this.linePositions.set([...previous.position, x, y, z], lineCount * 3);
const currentColor = this.colors.subarray((count - 1) * 3, count * 3);
this.lineColors.set(previous.color, lineCount * 3);
this.lineColors.set(currentColor, (lineCount + 1) * 3); lineCount += 2;
}
previous = { position: [x, y, z], time: f.time, color: this.colors.slice((count - 1) * 3, count * 3) };
}
}
this.geometry.setDrawRange(0, count); this.lineGeo.setDrawRange(0, lineCount);
for (const attr of Object.values(this.geometry.attributes)) attr.needsUpdate = true;
for (const attr of Object.values(this.lineGeo.attributes)) attr.needsUpdate = true;
this.lines.visible = settings.connections && this.mode !== 'tone';
this.controls.autoRotate = settings.rotate;
}
render(now, delta = 1 / 60) {
if (!this.host.clientWidth || !this.host.clientHeight) return;
if (this.mode === 'constellation') this.constellation?.animate(now);
const bounds = this.constellation?.frameBounds;
if (this.mode === 'constellation' && this.autoFrame && bounds) {
if (!this.framingEnvelope) this.framingEnvelope = { center: bounds.center.clone(), radius: bounds.radius, elapsed: 0 };
const envelope = this.framingEnvelope;
envelope.elapsed += delta;
// Settle the centre once, then only widen the view when sound grows beyond it.
// A rolling trail must not continually pull the camera around or pump the zoom.
if (envelope.elapsed < 2) envelope.center.lerp(bounds.center, 1 - Math.exp(-delta / .7));
envelope.radius = Math.max(envelope.radius, bounds.radius + bounds.center.distanceTo(envelope.center));
const halfFov = this.camera.fov * Math.PI / 360;
const limitingFov = Math.min(halfFov, Math.atan(Math.tan(halfFov) * this.camera.aspect));
const desiredDistance = clamp(envelope.radius / Math.sin(limitingFov) * 1.14, this.controls.minDistance, this.controls.maxDistance);
this.frameOffset.subVectors(this.camera.position, this.controls.target);
const blend = 1 - Math.exp(-delta / 1.2);
const distance = THREE.MathUtils.lerp(this.frameOffset.length(), desiredDistance, blend);
this.controls.target.lerp(envelope.center, blend);
this.camera.position.copy(this.controls.target).add(this.frameOffset.setLength(distance));
}
this.controls.dampingFactor = 1 - Math.exp(-delta / .18);
this.controls.update(delta); this.renderer.render(this.scene, this.camera);
}
dispose() { this.resizeObserver.disconnect(); this.controls.dispose(); this.scene.traverse(o => { o.geometry?.dispose(); if (o.material) { o.material.map?.dispose(); o.material.dispose(); } }); this.renderer.dispose(); }
}

14
src/style.css Normal file

File diff suppressed because one or more lines are too long

30
tests/analysis.test.js Normal file
View File

@@ -0,0 +1,30 @@
import test from 'node:test';
import assert from 'node:assert/strict';
import { analyzeSpectrum } from '../src/analysis.js';
test('silence produces finite zero features and no peaks', () => {
const f = analyzeSpectrum(new Float32Array(32).fill(-Infinity), new Float32Array(64), 48000);
for (const key of ['rms', 'centroid', 'spread', 'entropy', 'flatness']) assert.equal(f[key], 0);
assert.deepEqual(f.peaks, []);
});
test('a single spectral tone has its exact frequency and zero spread and entropy', () => {
const db = new Float32Array(32).fill(-Infinity); db[4] = -6;
const f = analyzeSpectrum(db, new Float32Array([0.5, -0.5]), 48000);
assert.equal(f.centroid, 3000); assert.equal(f.spread, 0); assert.equal(f.entropy, 0);
assert.equal(f.rms, 0.5); assert.equal(f.peaks[0].frequency, 3000);
});
test('uniform spectral power has maximum normalized entropy and flatness', () => {
const f = analyzeSpectrum(new Float32Array(32).fill(-20), new Float32Array(64), 48000);
assert.ok(Math.abs(f.entropy - 1) < 1e-10);
assert.ok(Math.abs(f.flatness - 1) < 1e-10);
assert.ok(Math.abs(f.centroid - 11625) < 1e-8);
});
test('centroid weights spectral power, not decibel values', () => {
const db = new Float32Array(4).fill(-Infinity); db[1] = 0; db[3] = -10;
const f = analyzeSpectrum(db, new Float32Array(8), 8000);
assert.ok(Math.abs(f.centroid - 1300 / 1.1) < 1e-7);
});
test('nonfinite inputs cannot contaminate 3D coordinates', () => {
const f = analyzeSpectrum(new Float32Array([NaN, Infinity, -Infinity, -30]), new Float32Array([NaN, 0]), 48000);
for (const key of ['rms', 'centroid', 'spread', 'entropy', 'flatness']) assert.ok(Number.isFinite(f[key]));
});

28
tests/color.test.js Normal file
View File

@@ -0,0 +1,28 @@
import test from 'node:test';
import assert from 'node:assert/strict';
import { colorStyle } from '../src/color.js';
test('each spatial axis changes the color at equal loudness', () => {
const base = colorStyle(0, 3.5, 0, 0.5, 'spectrum', 'position');
for (const position of [[3, 3.5, 0], [0, 6, 0], [0, 3.5, 4]]) {
const next = colorStyle(...position, 0.5, 'spectrum', 'position');
assert.notEqual(next.h, base.h);
}
});
test('loudness changes brightness without moving spatial hue', () => {
const quiet = colorStyle(0, 3.5, 0, 0.1, 'spectrum', 'position');
const loud = colorStyle(0, 3.5, 0, 0.9, 'spectrum', 'position');
assert.equal(quiet.h, loud.h);
assert.ok(loud.energy > quiet.energy);
});
test('loudness mapping is independent of spatial position', () => {
assert.deepEqual(colorStyle(-6, 0, -6, 0.4, 'ember', 'amplitude'), colorStyle(6, 7, 6, 0.4, 'ember', 'amplitude'));
});
test('all palette values remain finite and bounded at plot extremes', () => {
for (const palette of ['ember', 'spectrum', 'glacier', 'orchid']) {
for (const x of [-6, 0, 6]) for (const y of [0, 7]) for (const z of [-6, 6]) {
const c = colorStyle(x, y, z, 1, palette, 'position');
for (const key of ['h', 's', 'l']) assert.ok(c[key] >= 0 && c[key] <= 1);
}
}
});

View File

@@ -0,0 +1,76 @@
import test from 'node:test';
import assert from 'node:assert/strict';
import { ConstellationLayer } from '../src/constellation.js';
test('rendered links keep correct endpoints after expiry, with labels fixed and fades independent of uploads', () => {
// Exercise real Three.js buffers without requiring a browser or GPU.
const originalDocument = globalThis.document;
globalThis.document = { createElement: () => ({ getContext: () => ({ clearRect() {}, fillText() {} }) }) };
let layer;
try {
layer = new ConstellationLayer();
const settings = { trail: 3, sensitivity: 1, palette: 'spectrum', colorMode: 'position', glow: 1.35, connections: true, labels: true };
const frame = time => ({ time, rms: .2, entropy: .4, spread: 900, peaks: [{ frequency: 1500 + time * 10, amplitude: .2 }] });
const history = Array.from({ length: 60 }, (_, i) => frame(i / 30));
layer.update(history, 2, settings, 600);
const label = layer.labelPool.find(label => label.node);
assert.ok(label);
const labelNode = label.node;
const labelPosition = label.sprite.position.clone();
const uploadVersion = layer.geometry.attributes.position.version;
layer.animate(2.01);
assert.equal(layer.uniforms.now.value, 2.01);
assert.equal(layer.lines.material.uniforms.now.value, 2.01);
assert.equal(layer.geometry.attributes.position.version, uploadVersion);
layer.update(history, 2.01, settings, 600);
assert.equal(layer.geometry.attributes.position.version, uploadVersion);
history.push(frame(2.05));
layer.update(history, 2.05, settings, 600);
assert.equal(label.node, labelNode);
assert.deepEqual(label.sprite.position, labelPosition);
// Several old points expire; line indices must still reference the surviving points.
history.push(frame(3.1));
layer.update(history, 3.1, settings, 600);
assert.ok(layer.model.nodes.every(node => node.time > .1));
assert.equal(layer.lineGeometry.instanceCount, layer.model.links.length);
assert.equal(layer.lineGeometry.index.count, 6);
assert.ok(layer.model.links.some(([, , temporal]) => temporal));
for (const [i, [a, b]] of layer.model.links.entries()) {
assert.deepEqual(Array.from(layer.linePositions.slice(i * 6, i * 6 + 6)), Array.from(new Float32Array([...layer.model.nodes[a].position, ...layer.model.nodes[b].position])));
}
settings.connections = false;
layer.update(history, 3.1, settings, 600);
assert.equal(layer.lines.visible, false);
layer.update([], 0, settings, 600);
assert.equal(layer.geometry.drawRange.count, 0);
assert.equal(layer.lineGeometry.instanceCount, 0);
assert.ok(layer.labelPool.every(label => !label.node && !label.sprite.visible));
} finally {
layer?.traverse(object => { object.geometry?.dispose(); object.material?.map?.dispose(); object.material?.dispose(); });
if (originalDocument === undefined) delete globalThis.document;
else globalThis.document = originalDocument;
}
});
test('old annotations fade gradually to zero without another audio sample', () => {
const originalDocument=globalThis.document;
globalThis.document={createElement:()=>({getContext:()=>({clearRect(){},fillText(){}})})};
let layer;
try {
layer=new ConstellationLayer();
layer.update([{time:0,rms:.2,entropy:.4,spread:900,peaks:[{frequency:1500,amplitude:.2}]}],.2,{trail:6,sensitivity:1,palette:'spectrum',colorMode:'position',glow:1,connections:true,labels:true},600);
const label=layer.labelPool.find(label=>label.node);
const fresh=label.sprite.material.opacity;
layer.animate(3);
const halfway=label.sprite.material.opacity;
assert.ok(halfway<fresh*.5);
layer.animate(5.9);
assert.ok(label.sprite.material.opacity<.01);
layer.animate(6);
assert.equal(label.sprite.material.opacity,0);
} finally {
layer?.traverse(o=>{o.geometry?.dispose();o.material?.map?.dispose();o.material?.dispose();});
if(originalDocument===undefined)delete globalThis.document;else globalThis.document=originalDocument;
}
});

View File

@@ -0,0 +1,91 @@
import test from 'node:test';
import assert from 'node:assert/strict';
import { makeNodes, connectNodes, ConstellationModel } from '../src/constellation-data.js';
const frame = (time, frequency = 1500) => ({time, rms: 0.2, entropy: 0.4, spread: 900, flatness: 0.03, peaks: [{frequency, amplitude: 0.2}]});
const settings = {trail:12, sensitivity:1};
test('constellation uses measured peak data and excludes expired or silent frames', () => {
const nodes = makeNodes([frame(-20), {...frame(0), rms:0}, frame(1)], 2, settings);
assert.equal(nodes.length, 1);
assert.equal(nodes[0].frequency, 1500); assert.equal(nodes[0].amplitude, 0.2); assert.equal(nodes[0].time, 1);
});
test('node positions are stable as they age and respond to changed frequency', () => {
const node = makeNodes([frame(1)], 2, settings)[0];
assert.deepEqual(node.position, makeNodes([frame(1)], 3, settings)[0].position);
assert.notDeepEqual(node.position, makeNodes([frame(1, 4000)], 2, settings)[0].position);
});
test('dense tracks have a bounded point count and finite positions', () => {
const frames = Array.from({length:3000}, (_, i) => ({...frame(i / 100), peaks:Array.from({length:12}, (_, j) => ({frequency:200+j*400, amplitude:0.1}))}));
const nodes = makeNodes(frames, 30, {...settings, trail:30});
assert.ok(nodes.length <= 2400); assert.ok(nodes.length > 100);
for (const node of nodes) assert.ok(node.position.every(Number.isFinite));
});
test('network connections are local, bounded and never self-connect', () => {
const nodes = [0, 1, 2, 50].map(x => ({position:[x,0,0]}));
const links = connectNodes(nodes);
assert.ok(links.length > 0);
for (const [a,b] of links) { assert.notEqual(a,b); assert.ok(Math.abs(nodes[a].position[0]-nodes[b].position[0]) < 3); }
assert.ok(links.length <= nodes.length * 3);
});
test('rolling history retains the identity, position and parents of surviving points', () => {
const model = new ConstellationModel();
const history = Array.from({length:60}, (_,i) => frame(i/30,1500+i*2));
const first = model.update(history,2,{...settings,trail:3});
const survivor = first.nodes.find(n=>n.time>1);
const saved = {id:survivor.id,position:[...survivor.position],parents:[...survivor.parents]};
history.splice(0,20); history.push(frame(2.1,1630));
const next = model.update(history,2.1,{...settings,trail:3});
const same = next.nodes.find(n=>n.id===saved.id);
assert.deepEqual(same.position,saved.position); assert.deepEqual(same.parents,saved.parents);
});
test('frequency tracks survive peak loudness reordering and create continuous links', () => {
const model = new ConstellationModel(), history=[];
history.push({...frame(0),peaks:[{frequency:1000,amplitude:.3},{frequency:2000,amplitude:.2}]});
const first = model.update(history,0,settings);
const tone = first.nodes.find(n=>n.frequency===1000);
history.push({...frame(.05),peaks:[{frequency:2000,amplitude:.4},{frequency:1005,amplitude:.1}]});
const next = model.update(history,.05,settings);
const continuation = next.nodes.find(n=>n.frequency===1005);
assert.equal(continuation.trackId,tone.trackId);
assert.ok(continuation.parents.includes(tone.id));
});
test('unchanged audio does not rebuild geometry or connections', () => {
const model = new ConstellationModel(), history=[frame(1)];
const first=model.update(history,1,settings);
const next=model.update(history,1.01,settings);
assert.equal(next.revision,first.revision);
assert.equal(next.nodes,first.nodes); assert.equal(next.links,first.links);
});
test('a new source resets tracks and never links back to the previous song', () => {
const model=new ConstellationModel(); model.update([frame(1)],1,settings);
const next=model.update([frame(2)],2,settings);
assert.equal(next.nodes.length,1); assert.deepEqual(next.nodes[0].parents,[]);
});
test('new samples are smoothly connected through sudden amplitude changes', () => {
const model=new ConstellationModel(), history=[{...frame(0),peaks:[{frequency:1500,amplitude:.01}]}];
const start=model.update(history,0,settings).nodes[0];
history.push({...frame(.05),peaks:[{frequency:1500,amplitude:.8}]});
const next=model.update(history,.05,settings).nodes.at(-1);
const raw=makeNodes([history[1]],.05,settings)[0];
const distance = (a,b) => Math.hypot(...a.position.map((v,i)=>v-b.position[i]));
assert.ok(distance(start,next)<distance(start,raw)*.5);
});
test('rapid pitch changes still connect consecutive sound samples', () => {
const model=new ConstellationModel(), history=[frame(0,700)];
const first=model.update(history,0,settings).nodes[0];
history.push(frame(.1,6000));
const next=model.update(history,.1,settings);
assert.ok(next.nodes.at(-1).parents.includes(first.id));
assert.ok(next.links.some(([, , trail])=>trail));
});
test('sound paths break across silence and all points and links expire', () => {
const model=new ConstellationModel(), history=[frame(0,700),frame(1,6000)];
const result=model.update(history,1,settings);
assert.equal(result.nodes.at(-1).trailParent,null);
const expired=model.update(history,14,settings);
assert.equal(expired.nodes.length,0);
assert.equal(expired.links.length,0);
});