A new kind of music search engine from the Jacobs School of Engineering at UC San Diego allows for users to discover music by typing in free-text semantic queries rather than song titles, album names or artist names. For example, if you're in the mood for "groovy driving music with lots of banjo" you'd just use that as your query and the experimental system would return specific song suggestions that best match the description.
The system uses machine learning to "listen" to songs it has never heard before and then label them based on the actual sounds in the song. Then, when people type descriptive words—like "mellow jazz" or "melodic techno"—into the team's experimental search engine, it retrieves the appropriate songs.
To accomplish this, the search engine uses a song dissection algorithm that can capture both the timbral (texture and color) and dynamic elements of a song and then automatically segment it into homogeneous sections such as verses, choruses and bridges. Each of those chunks are then labeled appropriately. (If you want to see the technical details of the the team's approach to automatic song segmentation, download this four-page paper.)
The engine currently works for more than 100 words that cover music genres, emotions and instruments, and is planned to be available to the public later this year.
The team of electrical engineers, including Gert Lanckriet, a professor and machine learning expert and Ph.D. students Luke Barrington and Antoni Chan developed the new segmentation algorithm after they came across Queen's "Bohemian Rhapsody," since it proved to be "too heterogeneous" of a song for the original search system to label accurately.
At first, the team paid cash-strapped undergrads to listen to songs and manually label to serve as as a training aid for the machine learning algorithm. But that got too expensive. Then, they had a brilliant idea of creating a fun and easy music trivia game for Facebook, and called it "Herd It." I gave it a spin myself, and it really is cool playing against other online users. And you don't have to download the app until after you try it out. You can get the Herd It app here and watch a video here.
"The Facebook games are a lot of fun and a great way to discover new music. At the same time, the games deliver the data we need to teach our computer audition system to listen to and describe music like humans do," said Lanckriet.
Exposing the search engine to hundreds of examples of each individual genre will help train it to find patterns in the songs and accurately label them.
"Once enough people play our new music discovery games on Facebook, I'll have the data I need to both improve our search engine and finish my Ph.D.," said Barrington.
Additionally, the song-word combinations collected by the game will enable the team to grow their music search engine's vocabulary and increase its coverage in genres and classes of music.
The research was presented at the 2009 International Conference on Acoustics, Speech, and Signal Processing (ICASSP) in Taiwan last month.