Article 3

Advanced Audio Visualisation

Drive 3D frequency visualisations, beat detection, and spectral analysis using Minim's FFT and BeatDetect classes inside Processing.

⏱ 50 min Minim FFT audio beat detection P3D visualisation

Minim is Processing’s bundled audio library. It provides an FFT class for spectral analysis, BeatDetect for onset detection, and AudioInput for live microphone capture. Together they give you the raw numbers to build reactive audio visualisations with no external dependencies.

FFT Fundamentals

The Fourier transform decomposes a time-domain audio buffer into frequency-domain magnitude values. Minim’s FFT class operates on a buffer of a power-of-two size; 1024 is common for real-time work.

import ddf.minim.*;
import ddf.minim.analysis.*;

Minim minim;
AudioPlayer player;
FFT fft;

void setup() {
  size(1024, 400, P2D);
  minim = new Minim(this);
  player = minim.loadFile("track.mp3", 1024);
  player.loop();

  fft = new FFT(player.bufferSize(), player.sampleRate());
  fft.logAverages(22, 3);  // 22 Hz base, 3 bands per octave
}

void draw() {
  background(10);
  fft.forward(player.mix);  // compute FFT from current buffer

  stroke(255, 100, 50);
  noFill();
  for (int i = 0; i < fft.specSize(); i++) {
    float mag = fft.getBand(i) * 4;
    line(i, height, i, height - mag);
  }
}

fft.forward() must be called every draw loop before reading any band values. getBand(i) returns the raw magnitude; values are roughly 0–100 for normal music levels. getFreq(hz) returns the magnitude at a specific frequency in hertz.

Log Averages and Linear Averages

Raw bins are spaced linearly but human hearing is logarithmic. fft.logAverages(minBandwidth, bandsPerOctave) groups bins into perceptually even bands. Access them with fft.getAvg(i) and count them with fft.avgSize():

fft.logAverages(22, 8);  // 8 bands per octave — good for visualisation

for (int i = 0; i < fft.avgSize(); i++) {
  float mag  = fft.getAvg(i);
  float hue  = map(i, 0, fft.avgSize(), 0, 360);
  float barW = width / float(fft.avgSize());
  colorMode(HSB, 360, 100, 100);
  fill(hue, 90, map(mag, 0, 60, 20, 100));
  rect(i * barW, height - mag * 3, barW - 2, mag * 3);
}

Beat Detection

BeatDetect identifies onsets in the signal. It has two modes: SOUND_ENERGY measures overall loudness spikes, and FREQ_ENERGY analyses independent frequency sub-bands for kick, snare, and hat detection.

import ddf.minim.*;
import ddf.minim.analysis.*;

Minim minim;
AudioInput input;
BeatDetect beat;

float kickSize  = 1.0;
float snareSize = 1.0;
float hatSize   = 1.0;

void setup() {
  size(600, 400, P2D);
  minim = new Minim(this);
  input = minim.getLineIn();  // live microphone

  beat = new BeatDetect(input.bufferSize(), input.sampleRate());
  beat.setSensitivity(100);   // milliseconds of silence after onset
  beat.detectMode(BeatDetect.FREQ_ENERGY);
}

void draw() {
  beat.detect(input.mix);

  if (beat.isKick())  { kickSize  = 3.5; }
  if (beat.isSnare()) { snareSize = 3.5; }
  if (beat.isHat())   { hatSize   = 3.5; }

  kickSize  = lerp(kickSize,  1.0, 0.12);
  snareSize = lerp(snareSize, 1.0, 0.12);
  hatSize   = lerp(hatSize,   1.0, 0.12);

  background(15);
  noStroke();

  // Kick — red circle, centre-left
  fill(255, 60, 60);
  ellipse(150, height/2, 80 * kickSize, 80 * kickSize);

  // Snare — green circle, centre
  fill(60, 255, 120);
  ellipse(300, height/2, 80 * snareSize, 80 * snareSize);

  // Hat — blue circle, centre-right
  fill(60, 150, 255);
  ellipse(450, height/2, 40 * hatSize, 40 * hatSize);
}

beat.isOnset(band) lets you query any specific FFT band by index for custom frequency ranges.

Spectral Centroid and Flux

The spectral centroid is the weighted average frequency — perceptually the “brightness” of a sound. The spectral flux measures how much the spectrum changes between frames, useful for detecting transients.

// Compute spectral centroid manually
float spectralCentroid(FFT f) {
  float num = 0, den = 0;
  for (int i = 0; i < f.specSize(); i++) {
    float mag  = f.getBand(i);
    float freq = f.indexToFreq(i);
    num += freq * mag;
    den += mag;
  }
  return (den > 0) ? num / den : 0;
}

// Spectral flux (store previous magnitudes)
float[] prevMags;

float spectralFlux(FFT f) {
  if (prevMags == null) prevMags = new float[f.specSize()];
  float flux = 0;
  for (int i = 0; i < f.specSize(); i++) {
    float diff = f.getBand(i) - prevMags[i];
    flux += diff * diff;
    prevMags[i] = f.getBand(i);
  }
  return sqrt(flux);
}

Full Example: 3D FFT Visualiser

This sketch renders frequency bars extruded along the Z axis in P3D, with colour mapped to amplitude and hue mapped to frequency band. A rotating camera orbits the display.

import ddf.minim.*;
import ddf.minim.analysis.*;

Minim minim;
AudioPlayer player;
FFT fft;

final int BANDS   = 64;
final int HISTORY = 80;

float[][] history = new float[HISTORY][BANDS];
int histHead      = 0;
float camAngle    = 0;

void setup() {
  size(1280, 720, P3D);
  colorMode(HSB, 360, 100, 100, 100);
  hint(ENABLE_DEPTH_SORT);

  minim  = new Minim(this);
  player = minim.loadFile("track.mp3", 2048);
  player.loop();

  fft = new FFT(player.bufferSize(), player.sampleRate());
  fft.logAverages(22, 4);  // produces ~BANDS averages
}

void draw() {
  background(240, 20, 8);
  fft.forward(player.mix);

  // Sample current averages into history ring buffer
  for (int b = 0; b < min(BANDS, fft.avgSize()); b++) {
    history[histHead][b] = fft.getAvg(b);
  }
  histHead = (histHead + 1) % HISTORY;

  // Camera orbit
  camAngle += 0.008;
  float camX = sin(camAngle) * 600;
  float camZ = cos(camAngle) * 600;
  camera(camX, -300, camZ, 0, 0, 0, 0, 1, 0);

  // Draw 3D bar grid
  float bandW = 12;
  float depthStep = 14;
  float totalX = BANDS * (bandW + 2) * 0.5;
  float totalZ = HISTORY * depthStep * 0.5;

  for (int z = 0; z < HISTORY; z++) {
    int bufIdx = (histHead + z) % HISTORY;
    float zPos = z * depthStep - totalZ;

    for (int b = 0; b < BANDS; b++) {
      float mag  = history[bufIdx][b];
      float h    = map(b, 0, BANDS, 200, 360);
      float bri  = map(mag, 0, 60, 10, 100);
      float barH = mag * 4;
      float xPos = b * (bandW + 2) - totalX;

      pushMatrix();
      translate(xPos, barH * 0.5, zPos);

      // Age-based transparency (older = more transparent)
      float alpha = map(z, 0, HISTORY, 20, 100);
      fill(h, 85, bri, alpha);
      noStroke();
      box(bandW, barH, depthStep * 0.9);
      popMatrix();
    }
  }

  // HUD
  camera();
  hint(DISABLE_DEPTH_SORT);
  noLights();
  fill(0, 0, 100);
  text(nf(frameRate, 0, 1) + " fps", 20, 30);
}

void stop() {
  player.close();
  minim.stop();
  super.stop();
}

Working with AudioInput (Microphone)

For live performance or installations, swap AudioPlayer for AudioInput. The API is identical for mix, left, and right buffers:

AudioInput input = minim.getLineIn(Minim.STEREO, 2048);
// Feed FFT the same way:
fft.forward(input.mix);

Close the input in stop() with input.close() to release the audio device cleanly.

Tips for Smooth Visuals

Raw FFT values are noisy frame-to-frame. Apply exponential smoothing to each band value before rendering:

float[] smoothed = new float[BANDS];
float smoothFactor = 0.2;  // 0 = no smoothing, 1 = frozen

// Inside draw():
for (int i = 0; i < BANDS; i++) {
  float target = fft.getAvg(i);
  smoothed[i] = lerp(smoothed[i], target, smoothFactor);
}

Use smoothed[i] for display, not fft.getAvg(i) directly. This removes flickering while preserving transient punch.