Silicon Valley might never solve the Web's impossible problem: taste
There's no accounting for taste, but Pandora keeps trying to do just that anyways.
Let's say I'm a music snob. (in reality I'm more like a wanna-be music snob.) I have a respectable rock critic-approved vinyl collection with a corresponding iTunes catalog of indie hits spanning Joanna Newsom to Joanna Gruesome. I've spent years pruning and editing my Pandora playlists with strategic thumbs up and thumbs down signals to reflect my perfectly curated taste.
There is one problem, though. I'll it call the "Party in the U.S.A." problem. I freaking love that song, and not in a guilty pleasure way. It is a great song, and, even though my liking it confounds logic and rationality, I can't help it. Oh, and that "Hey, Ho" song? The adorably twee one they play at all the weddings and in commercials and whatnot? I hate that song. It should be right in my taste wheelhouse, but I can't stand it, and I'm not sure why. And that confuses the hell out of an algorithm.
Taste is irrational and unpredictable, but in most categories, algorithms can get pretty damn close based on our past preferences. Erin bought an ugly cardigan? Show her more ugly cardigans! Erin went to a burger place? Show her more burger places! I don't mind sorting through these recommendations to pick out which ones I like best.
But music is a whole different beast. We place a huge emotional significance on songs we like and hate. Pick the wrong song, and you've ruined the mood. There's also the issue of over-played songs that get old fast, and the problem of playing too much stuff that's too obscure. Music taste is the most unscientific problem there is, and yet, countless music startups aspire to solve it with algorithms.
Last year, we saw a backlash to the algorithm, in music and in every other category, too. Curation exploded -- suddenly every startup from Fab and Birchbox to Behance and Songza was preaching its gospel. There was a whole conference dedicated to curation. People started taking jobs as professional good-taste-havers, with titles like"Chief Curator" and "Chief Creative Officer." I called the whole trend a Cure-gasm.
This was all very alien to Silicon Valley. There's nothing tech geeks hate more than throwing expensive, unpredictable, unscalable humans at a problem.
At least algorithm-happy Pandora knows it can't truly predict anyone's taste. "I do not believe this is a solvable problem," says Eric Bieschke, the company's Chief Scientist. "But that's what makes it so interesting to me."
Indeed, it's interesting enough to keep him at it for 13 years. Bieschke was Pandora's second employee. As the company has grown to 950 employees, Pandora has in fact thrown a few humans at the problem. Bieschke has a team of 55 people working under him on the playlist team. That includes data scientists, recommendation system specialists, statisticians, software engineers and an astrophysicist. There are also 25 music analysts and 10 curators who listen to music and categorize it.
But notably, those humans don't have an opinion on "Party in the U.S.A.;" they're not getting paid for their taste. The whole point of Pandora's algorithm is that it doesn't have a voice or an opinion. Even the Pandora's explanations of why it played a certain song -- "This track has similar blues influences, great lyrics, repetitive melodic phrasing, extensive vamping and minor key tonality," for example -- are written by a robot.
When it comes to taste, Pandora is as Silicon Valley as it gets (even though it's headquartered in Oakland). The company believes only an algorithm can create the best, most personalized listening experience for its users. As Pandora has grown that user base to 200 million, the company has obsessively tweaked and improved its algorithm along the way. Bieschke's team still ships updates to the playlist algorithm every Tuesday.
In the beginning, the music Pandora listeners heard was chosen based on Pandora's song mapping technology and basic linear algebra. But now, with almost a decade worth of listener data and 30 billion thumbs-up and thumbs-down indications from users, playlists are a big data game. "No one else has this data from eight years." Bieschke says.
The company uses its data to get users returning to Pandora, often employing repetition to get people familiar with songs and hooked on hearing them. The algorithm is doing its job if it knows "when this and only this listener is ripe for a burst of discovery, or a familiarly beloved song from their childhood, or their favorite song from last summer," Bieschke says. "It's all about hitting people with the perfect song in the perfect moment."
Regarding companies mimicking Pandora's offerings, Bieschke says, "Even if you have access to our algorithms, you'd still need our data." That eight years of listening data is the biggest thing that differentiates Pandora from its newest competitor, Apple Radio.
I'm always happy to talk about curation and music, but it's no surprise that Pandora wanted to talk to me about this now. Apple's iTunes Radio launched six weeks ago, and in that time it has gained 20 million listeners. Pandora CFO Mike Merring called the service a "credible threat" even though 92 percent of iTunes Radio listeners also still use Pandora.
Pandora plans to fend off competition the same way it always has -- by continually making its service better. "Any improvement (in the listening experience) is big for us," Bieschke says. "Amazon is still working on it. Pandora is still working on it."
[Illustrations by Hallie Bateman for Pandodaily]