Audio Spectrum and Wave Form

processing audio waveform & spectrum 1

processing audio waveform & spectrum 2

This is a quick little audio visualizer I put together with Processing and the ESS Sound Library. The audio spectrum is analyzed with an FFT and spectrum bands are plotted as vertical bars. The Waveform is drawn over the bars in white, adding a lot of interest to the image. To create the fading effect of the object a transparent, white rectangle is drawn over the whole sketch instead of using a background. Previous frames are left on the screen and are slowly covered up by white.

To create an interesting color scheme each bar is colored based on its own height and its neighbors. Combined with the overlapping shapes a broad range of tones and hues is created.

processing audio waveform & spectrum 3

The code isn’t perfect but I’ll share it anyway. You can see some more examples and learn more about the library on the ESS home page. Let me know if I can clarify anything.

processing code
code formatterAudio Spectrum & Waveform Script

import krister.Ess.*;

FFT fft;
AudioInput input;
int bufferSize=256;

void setup() {
  size(1300,400);
  stroke(0);
  noFill();
  smooth();
  frameRate(30);

  Ess.start(this);
  input=new AudioInput(bufferSize);
  fft=new FFT(bufferSize*2);
  input.start();

  fft.damp(.3);
  fft.equalizer(true);
  fft.limits(.005,.01);

  background(255);
}

void draw() {
  pushStyle();
  noStroke();
  fill(255,50);
  rect(0,0,width,height);
  popStyle();

  renderSpectrum(50, height/2, 5, 400);
}

public void audioInputData(AudioInput input) {
  fft.getSpectrum(input);
}

void renderSpectrum(int X, int Y, int S, int A) {
  pushStyle();
  noStroke();
  for (int i=1; i<(width-2*X)/S; i++) {
    float v=fft.spectrum[i];
    fill((v-fft.spectrum[i-1])*200,v*200,(v-fft.spectrum[i+1])*600,30);
    rect((i*S)+X,Y+v*(A/2),3,v*-A);
  }
  popStyle();

  pushStyle();
  stroke(255,200);
  strokeWeight(2);
  noFill();
  int interp=max(0,(((millis()-input.bufferStartTime)/input.duration)*input.size));
  beginShape();
  for (int i=0;i<(width-2*X)/S;i++) {
    float pos=Y;
    if (i+interp+1<input.buffer2.length) {
      pos-=input.buffer2[i+interp]*A*2;
    }
    curveVertex(X+i*S,pos);
  }
  endShape();
  popStyle();
}

void keyPressed() {
  if (key=='s') {
    saveFrame("waveform-####.png");
  }
}

8 comments

  • Pingback: Some Random Dude

  • noman
    05.05.09

    Tried it out, all I get is a white background, with a few very small gray dots.


  • tony
    05.05.09

    If your not getting any errors it’s probably an issue with your audio input. The script checks for the default audio input. My computer has in internal microphone. Check the sound settings in your system preferences and make sure it’s picking something up and the input volume is turned up.

    If your still not getting anything you can try scaling up the signal within processing by increasing the fourth parameter of the renderSpectrum function (currently 400) within the draw loop. Other than than that it might help to yell at your computer.


  • noman
    05.05.09

    Thanks. It was my audio input.

    For others having a problem with this example:
    for the ESS library – make sure you restart processing after you extracted
    for the audio input in windows – turn on/unmute stereo mix as input, in volume settings.


  • John Seales
    03.20.10

    Hey – your visualization is beautiful.

    I’m a composer, and sometimes I work on transcribing performances. I’ve started using a method of scoring to a video of a performance. I can get a really accurate transcription by watching performer’s fingers. Problem is, I need a video of the performers.

    I have an idea though: if I could take a sound file and turn it into a video file, the visual part being a waveform visualization (a clear and basic one) I could make very accurate transcriptions from a sound file.

    input: sound file (wav or aiff)
    output: video file (mpeg or mp2 or whatever) with original sound file as audio component and waveform visualization as visual component.

    Does a tool like that exist, or would it be an easy extension of what you have here?


  • tony
    03.27.10

    Thanks.

    There are definitely some tools like that out there. If you search for spectrum analyzers you should be able to find something. I think there might even be one as a visualizer for itunes. Audacity, an free audio editor has a spectrum view as well.

    Most of these applications will display a grid which represents time in one direction and frequency in the other. The pixels will be colored based on the volume of that frequency at each time.

    Unfortunately I don’t think this sort of visualization will be as easy to understand as you would hope. A single note from an instrument will not be seen as a blip and the base frequency but a more complex structure of overtones. If you’re only looking at one instrument at a time, it might be possible, depending on the instrument, but it would be easy.


  • Pedro
    10.06.10

    Muito bom .
    Very good!
    thanks


  • meeble
    07.24.11

    I think your visualizer looks exactly like what I am looking for – but I’m not really sure how to run it. Do I run it in the “Processing” app? What role does Ess play? Any specific help you can offer is appreciated…