A Visual Instrument That Turns Colors Into Music

Music may be something that you hear, but it does have a visual element. That visual aspect is different for everyone—I see melodies and harmonies as shapes, with a chord laid out as a fragment of its parent-scale. It's asily moved around and re-arranged, but it's always a part of a larger thing.


Other people see sound as color—it's called "Sound -> Color Synesthesia," and it sounds intense as all get out. I've known a couple of people whose brains worked like this—their musical hearing went beyond perfect pitch into a visual realm. It left them able to hear notes and harmonies in a unique and mystifying (to me) way.

This camera, called "Audible Color," is also a synesthete. It's a musical instrument designed by Hideaki Matsui and Momo Miyazaki that "reads" colors on a spectrum that assigns various colors to various notes. It's a nifty way to generate (and observe) music, and the melodies it creates are soothing and unpredictable.


From Matsui's website:

Audible color is an audio-visual instrument. Sound is generated based on color detected by a web cam connected to a computer. Red, green and blue correspond with certain music notes. When the colors are mixed, the resulting secondary colors produce different notes. The size of the colors influences the volume and frequency of the notes played. Color detection and sound generation were created and are controlled using Processing code.

If you've played a game like Flower or Sworcery, where musical tones are generated by touching things in the environment, you've probably seen something similar. The world is made into natural music, and the resulting melodies are just as beautiful as notes on a page.

Audible Color [Hideaki Matsui, thanks G.A.!]

Share This Story

Get our newsletter


This is pretty interesting, albeit a bit limited by the grouping of specific colors to notes, which makes it a one-key ordeal.

I can imagine one could use video to generate more complex compositions. Imagine taking different sample averages of a given image (1x1, 2x2, 4x4, etc), with each size determining a different layer of the composition. Colors can be linked to moods, which can be linked to keys and modes. And each layer adds a specific flavor depending on the colors present.