We recently came across Anita Lillie’s beautiful music visualization and MusicBox projects. The music visualization project was especially intriguing to us, since it was the only music visualization project we’d seen that did not require video playback—the visualizations were strictly in 2D. Unfortunately, the source code for Lillie’s visualizations was not available, so we decided to build our own.
Our goal was to build a 2D visualization that could be approached like one of Edward Tufte’s sparklines: a quick, snapshot overview with high information density. In addition, having an image snapshot of a song could be useful for visually-minded people who often find themselves thinking of music as in spatial or pictorial terms. In the same way that Cicero used different rooms in his home to memorize different sections of his oratories, a 2D song-picture could provide a memorable structure for interpreting and contextualizing moments in a piece of music.
Here is the fruit of our labors. The top segment represents volume across the duration of the song. The bottom segments represent the 12 pitches of the song’s primary key, and the weight of each block corresponds to that pitch’s volume at that point in the song. Details, including Echo Nest integration and source code, are below.
The Echo Nest has developed a brilliant piece of software called Analyze. It “listens” to music as if it were human, and extracts information about rhythm, tone, pitch, etc. The amount and quality of information available is stunning. A simple web service API provides anyone with access to its vast power.
Adam Lindsay has written a fantastic overview of the API service.
Our code, below, requires the Python Imaging Library, aggdraw, and Echo Nest Remix with a valid API key. This is a rough sketch of what is possible, and there are probably bugs, so don’t be surprised if something breaks.
Please feel free to use and modify this code as you see fit, we’d just ask that you provide a link back to this site if you happen to use our code in any of your projects.
import echonest.audio as audio import Image import aggdraw import math def main( input_filename, output_filename, width=None, height=None ): """ Simple song visualization using Echo Nest published by Chris Mueller, http://visualmotive.com """ audio_file = audio.LocalAnalysis( input_filename ) segments = audio_file.analysis.segments if width is None: width = int( math.ceil( audio_file.analysis.duration ) ) * 2 width_scale = float( width ) / audio_file.analysis.duration if height is None: height = 216 height_part = height/3 img = Image.new( "RGBA", ( width, height ), "#ffffff" ) draw = aggdraw.Draw( img ) draw.setantialias( False ) brush = aggdraw.Brush( "#0088aa", 255 ) pen = aggdraw.Pen( "#ffffff", 1, 0 ) for segment in segments: left = width_scale * segment.start right = width_scale * ( segment.start + segment.duration ) rect = ( left, height_part, right, segment.loudness_max * -1 ) draw.rectangle( rect, pen, brush ) for i in xrange( len( segment.pitches ) ): # assumes a constant key pitch_loudness = segment.pitches[ i ] pitch_brush = aggdraw.Brush( "#0088aa", int( 255 * pitch_loudness ) ) pitch_height = 2 * height_part * ( 1.0 / 12.0 ) top = height_part + i * pitch_height rect = ( left, top, right, top + pitch_height ) draw.rectangle( rect, pen, pitch_brush ) draw.flush() del draw img.save( output_filename, "PNG")
This is only scratching the surface of possible 2D music visualizations. We look forward to exploring this concept more in the future. Many thanks to Echo Nest for their great service.