Spectralmind 07/09/12

This week’s Tools Spotlight is a follow-up interview with Thomas Lidy, founder and CEO of Spectralmind. We first interviewed him this spring, but wanted to reach back out to speak with him again after finding out about some of the company’s big recent developments. The Spectralmind Audio Intelligence Platform is powered by intelligent audio technology that automatically analyzes media in large catalogues to detect beat, rhythm, mood, genre, timbre and other acoustic properties, understanding sounds just as humans would. Spectralmind technology also powers their iOS app Sonarflow, which is a fun, simple and interactive way to discover music directly on your iPhone and iPad. 

For our readers who didn’t read our first interview, give us a quick re-introduction of Spectralmind and why you started it.

So we’re doing this pretty exciting thing with music analysis, specifically, semantic music analysis, which you can do a number of things with. You can, for the first time, search sound by sound. You can find music by its actual sound – which wasn’t possible before, because so far you’ve needed to search by keywords. So I was doing research in this area, along with some of my co-founders, and we put our team together and started Spectralmind to bring innovative products into the market, both for professional users and consumers which digs into this exciting music discovery space. We have products out there now for professional users to search music for production purposes, but we also have apps out for discovering new music from large music services. The second thing our company stands for is a high degree of usability and user experience, so we’ve added designers to our team to help deliver the best experience in searching music out there.

That’s really good to hear! The biggest complaint for a lot of software companies is that they may have great features, there may be great code there but if users are getting frustrated they’re never going to discover how great the code is.

Right – in the early days when we founded the company, we were like, okay we’re a bunch of engineers, we need somebody who takes care of the designs. So we added that and we got a whole new concept on how to interact with these interfaces. We were introducing a new form of search here: audio-based search. We don’t have an interface where you sing into it – that would be another possibility – but the heavy task for us is to have a visual interface to search for audio. You’re going from one modality to another – that’s the challenge here, and I think we did a good job in tackling it.

Since our first interview you added a new dimension to the Spectralmind Audio Intelligence Platform. How would you describe each one of those modules (categories) and the new music search portal?

I think the modules were described pretty well in the first interview, so I’d refer people to that. But the new thing is that, so far, you could look for entire songs and search similar songs based on similarity from one song to another. So you think of a song in a certain mood, you can find it by giving an example, uploading it or searching it with textual search and then say that you wanted to find similar songs. So that was in the previous release, but now we can go inside the song. This was a request by many producers and editors, because typically you want an excerpt or part of a song to use in production for a certain scene or advertisement. We have two ways of getting into the song and searching by segment. So at the query you can specify “segment inside a song” – you think, okay that sounds like the sound I’m looking for, give me search results that sound exactly like that portion of the song. Then the search by sound interface will give you the results of not only matching songs but it marks the segment within that song that is the best match.

That’s cool, because for a lot of songs, the bridge – which occurs in the last third of the song – can be so different from the rest of the song.

Right, and that’s something that was a difficulty before, too. As I mentioned, before, when you uploaded a song it would deliver you whole songs as matches – but as you just said, a song varies over time. You have a chorus, a bridge, you have a lead-in and a fade-out. So you’ll want to have a search more detailed, more specific. Which part of the song do you want to match? We’re now giving people that possibility, and they really love it.

Yeah, that kind of granularity is great.

I think it was really important to get the producers’ and editors’ feedback, because they work with parts of songs.

As a video editor who needs to use royalty-free music, it’s great to see Spectralmind technology powering the Getty Images music library. Have you gotten any initial feedback from users?

Yes, we started that with Getty Images – we have a music library, about 100K tracks there that are browse-able and searchable right now in the catalogue. The nice thing is with the latest releases, we’ve added backlinks to Getty where you can directly license that song. You have to login to their portal, but you don’t have to do another search. So you search in our sound interface, find the sound you need, click on “license” and you’re directed to the Getty page where you can download the song.

That’s the starting point of our partnership with Getty. We’re now discussing the next steps, how to go further and integrate it more tightly. I’ll be at the Mobile Beat conference next week in San Francisco presenting our new Sonarflow app, which we’ll talk about a little bit later. I’m meeting somebody from Getty there to discuss the next steps. At the same time, we’re looking for more partnerships with other providers of royalty-free music.

In the last few months we’ve been talking to studios and networks about how they would search and license music, and we’ve been getting more and more requests and feedback – like it would be great to have one space where you find all the assets, all the different catalogues. We’re currently weighing how we’ll achieve that so that we can deliver the best experience in that respect.

I know the portal is still in beta and that there is a feedback tab on the side of the page, but what are the other ways that users can comment on this new feature?

For the search by sound portal, we’d prefer it if you would leave feedback in the tab there because we have set an automatic system that ranks the suggestions so that we have a higher possibility of really figuring out what the most requested features are and what we should implement next. But really, anybody should feel free to contact us in any way – Twitter, Facebook, email, or our blog.

Sonarflow, your visualized music discovery iOS app, is also powered by Spectralmind technology. The big news is your partnership with Spotify. What are the details?

We are now launching Sonarflow for Spotify, which adds this entire concept of visual music discovery to the entire catalogue of Spotify. Spotify has 16 million titles in their catalogue, and their subscribers have the opportunity to access them all, but nobody will ever be able to listen to 16 million tracks. So it’s all about finding the right music for you, for yourself or for the right moment. Typically, when you’re a consumer – or let’s say a music aficionado or music listener – in your mind, you pretty much have a feeling for what kind of music you want in a certain moment. You want different music in the morning, different music during dinner, different music when you’re working out. It’s very hard to get that music quickly. But with our visual music discovery approach we want to enable that; you’ll be able to find and access the music you want very quickly. We start out with the music and artists that a person is already familiar with; typically on Spotify or iTunes you have music already. Based on these artists, our brand new discovery feature brings you suggestions for new artists that are similar. You can reach out and find more artists and titles in the Spotify catalogue. The nice thing is, right from within the Sonarflow for Spotify app, you can just double tap on these bubbles as if they were on your local device. These bubbles start streaming the music instantly, you don’t have to go into the Spotify app or change over to another app, you just double tap in the Sonarflow app and start streaming immediately.

The app also delivers background information about each artist – a biography, images, and links to youtube videos. You can also start youtube videos right from inside the app.

When will the Sonarflow for Spotify app be available?

That will be released July 10, 2012. I’ll be demoing it on stage at the Mobile Beat conference in San Francisco. Soon after the Spotify version there will be a special version for iTunes, not only for a local iTunes library but also as recommendations from the full iTunes catalogue, from the iTunes store. That will also be coming out in July, a week after the Spotify version.

When will the Android version of Sonarflow be available?

That’ll be sometime this winter.

Spectralmind has the potential to change the expectations that people have for interacting with music. What do you think will be the timeline of this transformation? 6 months? 1 year? 5 years?

I see a timeline of something like 3 years. I mean, our app is ready. We have some ideas that would improve it. But already there are a lot of other companies in this space, ranging from music services to device manufacturers, so I think that a version of this kind of browsing and directing with music and accessing music can come rather soon. We can establish it within one year, but then it needs a while to spread out. I think in a timeframe of 2 to 3 years, it could become very widespread.

You don’t really have any competitors in this space because what you do is pretty unique. But do you see bigger companies in this space, the Dolbys of this world and what have you, who have that kind of R&D muscle – do you feel like what you all are doing is starting to get on their radar in terms of them wanting to explore this space more?

Yes, we’ve seen that already. We are being approached by companies like that. As I mentioned, music services, platform providers and device manufacturers – there are several that we’re working on different projects with. We have our own IP (intellectual property) within the team so we do have this very unique combination of content analysis, audio analysis, with visual organization. We actually have a patent filed for the hierarchical media clustering which is fundamental for the visual organization that we’re doing. We’re more interested in working with other companies than being afraid of anybody copying our approach. We’re happy to work with anybody who would like to work with us. It’s more about collaboration than competition.

You mentioned in your May 25th blog post on music API’s that restrictions around music licenses can be a challenge when trying to improve your analysis engine. How much of a challenge is this for improving your analysis engine and what other challenges exist? Things like technological challenges, legal challenges, adoption challenges – what are your challenges as a company?

I don’t see challenges as very dramatic; it’s all about working with the right people. Getting the right partners on board. So I’m very optimistic. We’re not in any gray area or illegal space. We provide a technology for other partners in the space who typically have the music licenses and the platforms for that. So as I mentioned before, it’s a matter of collaboration. So when we find the right partners who have the music licenses and who would want to apply our engine to the content they have, it’s a perfect situation.

Are there any other potential partnerships you can speak to at this point?

At this point I can only mention that we are working more and more in the hardware device space, that we’re starting projects there. We’re working with a white label platform provider. I can’t mention any names.

What question(s) do you have for content creators and specifically video editors?

We would like to find out more about how, specifically, you search for music. How do you look for music? What would you want to see to improve what we’re doing with audio-based search? We want to learn more about the procedure, the process of how editors look into music. How do they start? How they fine-tune the search results? How long does it take for them? Where are the steps that we can really optimize and make their life better by improving their work – by having them find what they need faster?

I invite everybody to try it – we have the search by sound portal now. Previously it was a private beta, but now it’s a public beta – so everybody is invited to test it out and give us their feedback.

They can give specific feedback there at that portal, but can they also give you feedback on Facebook and Twitter?

Yes, or on the website there is a contact form. You can email us at info@spectralmind. If you’re an editor in any of these spaces and you have ideas, just contact us and we’re happy to talk about them.

 

Spectralmind is currently looking for investors for funding to grow their company.