Philip Glass String Quartet #5, 2nd Movement

We recently came across Anita Lillie’s beautiful music visualization and MusicBox projects. The music visualization project was especially intriguing to us, since it was the only music visualization project we’d seen that did not require video playback—the visualizations were strictly in 2D. Unfortunately, the source code for Lillie’s visualizations was not available, so we decided to build our own.

Our goal was to build a 2D visualization that could be approached like one of Edward Tufte’s sparklines: a quick, snapshot overview with high information density. In addition, having an image snapshot of a song could be useful for visually-minded people who often find themselves thinking of music as in spatial or pictorial terms. In the same way that Cicero used different rooms in his home to memorize different sections of his oratories, a 2D song-picture could provide a memorable structure for interpreting and contextualizing moments in a piece of music.

Here is the fruit of our labors. The top segment represents volume across the duration of the song. The bottom segments represent the 12 pitches of the song’s primary key, and the weight of each block corresponds to that pitch’s volume at that point in the song. Details, including Echo Nest integration and source code, are below.

Pink Floyd, Dark Side of the Moon, "Speak to Me/Breathe in the Air". The opening crescendos of this album are clearly visible.

Crystal Method, Vegas, "High Roller". We like the seeing that perfect volume curve at the beginning of this song. There's a pronounced industrial feeling here.

John Williams, Star Wars: A New Hope, "Imperial Attack". This visualization should probably be in black.

Goldfrapp, Supernature, "Ride a White Horse". Volume leveling clearly visible. Pop at its finest.

Mozart, Requiem, "Kyrie"

George Maasry, "Permutations". Mathematical permutations on a theme.

Making Of

The Echo Nest has developed a brilliant piece of software called Analyze. It “listens” to music as if it were human, and extracts information about rhythm, tone, pitch, etc. The amount and quality of information available is stunning. A simple web service API provides anyone with access to its vast power.

Adam Lindsay has written a fantastic overview of the API service.

Our code, below, requires the Python Imaging Library, aggdraw, and Echo Nest Remix with a valid API key. This is a rough sketch of what is possible, and there are probably bugs, so don’t be surprised if something breaks.

Please feel free to use and modify this code as you see fit, we’d just ask that you provide a link back to this site if you happen to use our code in any of your projects.

import echonest.audio as audio
import Image
import aggdraw
import math

def main( input_filename, output_filename, width=None, height=None ):
    """
    Simple song visualization using Echo Nest
    published by Chris Mueller, http://visualmotive.com
    """
    audio_file = audio.LocalAnalysis( input_filename )
    segments = audio_file.analysis.segments

    if width is None:
        width = int( math.ceil( audio_file.analysis.duration ) ) * 2
    width_scale = float( width ) / audio_file.analysis.duration
    if height is None:
        height = 216
    height_part = height/3

    img = Image.new( "RGBA", ( width, height ), "#ffffff" )
    draw = aggdraw.Draw( img )
    draw.setantialias( False )
    brush = aggdraw.Brush( "#0088aa", 255 )
    pen = aggdraw.Pen( "#ffffff", 1, 0 )

    for segment in segments:
        left = width_scale * segment.start
        right = width_scale * ( segment.start + segment.duration )
        rect = ( left,
                 height_part,
                 right,
                 segment.loudness_max * -1 )
        draw.rectangle( rect, pen, brush )
        for i in xrange( len( segment.pitches ) ): # assumes a constant key
            pitch_loudness = segment.pitches[ i ]
            pitch_brush = aggdraw.Brush( "#0088aa", int( 255 * pitch_loudness ) )
            pitch_height = 2 * height_part * ( 1.0 / 12.0 )
            top = height_part + i * pitch_height
            rect = ( left,
                     top,
                     right,
                     top + pitch_height )
            draw.rectangle( rect, pen, pitch_brush )

    draw.flush()
    del draw
    img.save( output_filename, "PNG")

This is only scratching the surface of possible 2D music visualizations. We look forward to exploring this concept more in the future. Many thanks to Echo Nest for their great service.