Augmented World

Music production software can be really complicated to use, which is why most people attach all kinds of external control surfaces. Anything to make playing music more like playing music and less like writing a term paper. It's better! But still not that intuitive.

Mallett's completely done away with button-pushing and knob-turning with his Pensato virtual reality controller, made for the popular music production suite Ableton Live. To build it, he hacked together an Oculus Rift and custom set of VR gloves. The Oculus displays the software's interface, and using the gloves you can reach out and interact with different elements of the interface.

The gloves use the magnetic tracking sensors from Razer's Hydra controller from a few years ago to determine the absolute position of the gloves. They've also been outfitted with bend sensors so the gloves can sense the grabbing motion.

Pensato is focused entirely on the software's Session View, which is designed for live performance. In addition to touching and grabbing elements in Live, he's programmed his custom interface for a few gestures.

Much like most of the interactive virtual reality programs we've seen and played with, Pensato is a little clumsier than a mass audience would probably accept. Sure, he did away with the pushing buttons, but he just replaces it with some awkward waving. Still, the potential is apparent in a few moments throughout the performance. In particular, the ability to literally reach out, grab something, and move it somewhere else seems like the kind of intuitive interaction that electronic music production often lacks.

Besides the interface making music easier or more interesting to produce, VR also has the potential to spice up the stage a little bit. If you've ever watched some dude in front of laptop do a set, you know it's not very exciting. I like the idea of standing on stage, waving your arms around like a a grand wizard as you're raining music on an audience.