Anthony Mattox Interaction Design + Digital Art

Although Processing does not have the ability to process audio on it’s own there are a few libraries which can be used to add such features. I’ve been experimenting with the ESS library but you can find others from the processing libraries page. Gathering Data from live or recorded sound can be a powerful tool for artists and designers in creating interactive applications, audio visualizers, music videos, or anything else that could involve an interaction with sound. Getting the data is fairly straight forward and the script returns an array of data updated each frame. This demonstration with display the data very simply, however, the numbers could be applied to any attribute in any system to produce different effects.

Here are the steps to creating a simple script to get the spectrum data from a microphone into a processing sketch. Using pre-recorded sound is similar. Once the data is in the sketch it can be manipulated and used to generate graphics or effect other aspects of a program including complex forms in 3d space using the OpenGL or P3D libraries. Read On »

Programmers have many languages at their disposal, all of which have their strengths and weaknesses and are most useful for particular applications. However, most of them have very similar structures. In order to communicate ideas between languages, to sketch out how a program should be structured, or even to explain the function of a program to non-programmers, scripts can be written in “pseudo-code.” This is just a fancy term for fake code which mimics the structure of code in simple easily understood terms. Pseudo-Code is written to be read by humans rather than computers, but still follows certain conventions. Read On »

A variation on a script I’ve been working with to generate a 3d visualization of an audio spectrum. The color and position of the first row of cubes is based on spectrum values and then passed down the array with a decay factor. Created with Processing using the OpenGL library to create the 3d environment and ESS to gather sound data.

Another variation renders a 3d grid rather than an array of cubes.

Simple recursive functions can created complex geometric fractals. A function draws a graphic and then by calling itself any number of times creates branches in the image. Each of the branches follows the same algorithm and creates the fractal. The transformations which occur between iterations, the graphic drawn, and any other steps or even randomized algorithms within the process determine the resulting fractals structure. This video of a fractal programmed in Processing is one of the simplest geometric fractals. Each iteration produces two copies of itself at it’s end. More complex versions of such a function are used to generate trees and other structures in video games and other virtual environments. The rotations of these two branches are changed slightly each frame and the function is recalculated. Within an object oriented programming language like Processing, each iteration could also create a new object or pass values into an array allowing the data to be easily reused without re-running the entire function.

This Processing application uses the audio spectrum values to generate a 3d visualization. The first row of the 2d array gets it’s values from the audio input and all subsequent rows take the previous one’s each frame, multiplied by a decay factor. the result is this undulation 3d grid. The script uses the OpenGL library to handle the 3d environment and the ESS sound library to get the audio spectrum.

I’m working to create a section of my site for interactive applets, but in the meantime i have also gotten a vimeo account to post videos of what I’m working on, especially my work with processing. There’s not much there yet, but I’ll be putting things up as i go.

Read On »

I created a relatively simple recursive algorithm in Processing which allows me to control any number of parameters of a geometric fractal. The function repeats it self changing parameters for the size and position and then calls another function to render a predefined graphic using the resulting data. Based on a set percentage and a randomized number, the function also recalls itself twice, causing the graphic system to branch.

Adjusting the initial parameter for the first call to the function, those for how each generation is calculated, and the graphic function can produce innumerable different forms and textures. These are a few images I have created using the scripts. Processing also has the ability to export graphics as vector graphics, which can be scaled and used in vector software such as Illustrator. I am working on an application which will allow the parameters to be changed through an interface within the application.

Read On »

Shutter feed is a project of mine from last spring which is currently on display in the Rosenberg Gallery on the MICA campus. The piece is a program which takes a webcam feed and continually overlays each frame and adjusts the image. The result reacts differently depending on the the lighting. In a dark area one can paint with a small light source and in a lit environment it produces slowly evolving ghostly images.

Read On »