Teaching a computer to play an instrument

Two professors -- Mark Bocko, who teaches electrical and computer engineering, and Dave Headlam, who teaches music theory -- got funding for the University of Rochester from the National Science Foundation to teach a computer how to reproduce what a musician does when he or she plays a clarinet -- the blowing pressure at every instant, the clamping force on the reed, the fingering patterns and so on.

As you can see in the video below, Headlam plays the instrument in an echo-free room while Bocko takes acoustic measurements. The computer then combines all this data to create an audio file that's 1000 times smaller than an MP3.

How? By having a better understanding of which information is critical and which is not. Bocko explains:

"If you look at how much data there is in an audio file, 1.5 million bits per second, and then you look at a musician playing an instrument--then you ask, well, how much information can a human actually transmit in a second? A musician can change their blowing pressure, the force of their lips on the reed, their fingerings, and all of that. But there's no way that there's one and a half million bits of information being imparted by the musician to the instrument."

Tiny audio files have many applications, from improved telepresence -- conversations and collaborations over the Internet would be more realistic because the sound would lag less -- to improved music education.

The computer can use the clarinet data to create a visual display of the clarinet's music that a student can follow as she plays. The closer she can make her sound pattern match her teacher's sound pattern, the better she's playing.

Here's a video from NSF:

This post was originally published on Smartplanet.com