Season 9 · Article 09

Music-Sync Interactives

Beat detection. Phase-locked animations. Rhythm games. The player's movement, the scene's changes, perfectly on the downbeat.

1. Know the BPM

If you authored the music: store BPM in metadata. If not: beat-detect with an algorithm.

2. Scheduling events to the beat

const BPM = 120;
const beatDuration = 60 / BPM;  // 0.5s per beat
const startTime = ctx.currentTime;

function beatIndex(now) {
  return Math.floor((now - startTime) / beatDuration);
}

let lastBeat = -1;
function tick() {
  const beat = beatIndex(ctx.currentTime);
  if (beat > lastBeat) {
    onBeat(beat);  // trigger animation
    lastBeat = beat;
  }
  requestAnimationFrame(tick);
}

3. Sample-accurate scheduling

Browsers can jitter setTimeout by 4-16ms. For music, too much.

Schedule audio events ~50-100ms ahead via setValueAtTime(). Audio engine plays them exactly.

function scheduleAhead() {
  // Schedule all beats up to currentTime + 0.1
  while (nextBeatTime < ctx.currentTime + 0.1) {
    playKickAt(nextBeatTime);
    nextBeatTime += beatDuration;
  }
}
setInterval(scheduleAhead, 25);

4. Phase-locked visuals

function tick() {
  const t = ctx.currentTime - startTime;
  const phase = (t % beatDuration) / beatDuration;  // 0..1 per beat

  // Scale pulse on downbeat
  sphere.scale.setScalar(1 + Math.max(0, 1 - phase * 4) * 0.5);
}

Envelope peaks at phase=0 (downbeat), decays to 0.25.

5. Beat detection (no BPM known)

// Simple: energy spike over rolling bass average (S9-03)
// Better: autocorrelation over bass history to find dominant period
// Best: BeatDetectr / Essentia.js libraries

6. Live demo — 4/4 with synced visuals

7. Rhythm game patterns

• Notes spawned at noteTime - travelTime.
• Target zone at fixed screen position.
• Player input timed against ctx.currentTime.
• Scoring: |inputTime - noteTime| buckets (Perfect / Good / Miss).

8. Song playback + sync

// Start audio
const source = ctx.createBufferSource();
source.buffer = decodedSongBuffer;
source.start(ctx.currentTime);
const songStartTime = ctx.currentTime;

// Game time = ctx.currentTime - songStartTime
// Compare to note chart times

9. Pitch detection

For Rock Band-style vocal: analyze mic input with autocorrelation to find fundamental frequency.

Libraries: Pitchy (UMD, simple), YIN algorithm implementations.

10. Takeaways

• BPM known → schedule by time. BPM unknown → detect energy spikes.
• Schedule 50-100ms ahead for sample-accurate audio.
• Phase (fractional beat position) drives phase-locked visuals.
• Rhythm games: fixed note chart, scored against ctx.currentTime.
• Pitch detection via autocorrelation for vocal games.