How Spotify Will Battle Taylor Swift (and Kanye West and Adele and Beyoncé)

Y ou do not want to get on Taylor Swift’s bad side. Your tour may be canceled, or your romantic failures could be blasted on the radio, or you might have to work on a Sunday. So when the pop superstar decided to pull her entire catalog from Spotify in 2014, citing issues with how the company compensates artists, it looked like a massive blow for the world’s largest paid music streaming service.

More than a year later, Swift’s music is still nowhere to be seen on Spotify. But the streaming service experienced its biggest growth year ever in 2015, adding 29 million active users. There’s no specific star or revolutionary business plan behind this success: In large part, its growth is thanks to the Echo Nest, a music data start-up that Spotify acquired just months before Swift’s exodus. Echo Nest alums have conceived and shepherded virtually every major product update Spotify has rolled out over the last year, from Discover Weekly to its Running mixes. These features, centered on personalization, are part of Spotify’s big bet that crafting killer, user-friendly playlists will keep its followers loyal.

But in the age of streaming album exclusives, loyalty can be tough to come by. As more streaming services have entered the market — most notably, Apple Music and Tidal — major artists have started brokering deals to keep their work on certain platforms and off others. In one case, a frustrated fan resorted to filing a lawsuit against an artist. And Spotify users have it particularly rough, with many of the year’s best-selling albums showing up on the service late, or not at all.

Spotify’s answer to these glaring omissions? Algorithms. Thanks to the Echo Nest, Spotify is now better than any other streaming service at helping users discover new songs they might love. But now the question is whether that perk will be enough to satisfy users missing out on music and album launches from superstars such as Swift, Beyoncé, and Drake. Can an algorithm — even a really, really good algorithm — replace them?

Nesting Period

T o understand how the Echo Nest came to play a pivotal role in the way we listen to music, you need to first know about Blitter and the James Brown Machine.

In the late ’90s, Blitter (real name: Brian Whitman) was a New York musician specializing in IDM (intelligent dance music), an electronic subgenre known for its unconventional sonic arrangements and computerized bleeps and bloops. His song “Catgut Kelly Hears the Tone” is the sound of a superintelligent alien species turning on their dial-up internet, with a bass line that Blitter believes worthy of a Gucci Mane track. According to Whitman, Faith No More frontman Mike Patton once called him “the biggest geek he’s ever met.”

He never hit it big. But as he was working New York nightclubs, Whitman was also cultivating an interest in computer science, particularly machine learning. He decided to enroll in a doctoral program at the Massachusetts Institute of Technology’s Media Lab in 2001.

At a conference the next year in Sweden, the birthplace of Spotify, Whitman met his musical match. Tristan Jehan, another Media Lab student, was also intensely interested in the ways machines could be used to organize music. But the two men wanted to tackle the problem in different ways.

Whitman was fascinated by the way people describe and write about music. He once studied Pitchfork reviews to measure their ratio of actual music criticism to personal musings about the writers’ lives. (“It was the style at the time,” he says now, diplomatically.) Was there a way to convert this flowery writing into usable data? If a music critic or a kid on a random blog wrote that a new indie band sounded like “David Bowie when he was in Berlin,” Whitman wanted to craft a way to algorithmically map that connection. “I wanted to have some computer program read the same thing I was reading,” he says.

Jehan (who prefers jazz to electronic) opted for a more technical approach. He was interested in deconstructing music itself, analyzing the digital signals of waveforms to categorize types of sounds. While at MIT, he developed the James Brown Machine, a computer program that, as its title implies, can compose “new” James Brown songs. After being fed dozens of actual tracks by the soul star, the computer attempts to algorithmically derive the “essence” of James Brown and output new compositions in the singer’s style. You can judge for yourself how well machine imitates man.

“We had this kind of funny fight between us about how to best model music,” Jehan says. “In the end we realized that you need both.”

After earning their respective PhDs at MIT in 2005, the pair launched the Echo Nest, a music analytics business. The start-up not only analyzed the audio of songs, but also trawled the web for writing about those songs. Combined, the data points provided a more comprehensive picture of the ways different tracks are connected, both from a sonic and a human perspective. “We worked on different parts of the engine,” Jehan says. “I was mostly working around audio stuff, and [Whitman] was mostly working around the text initially and then the overall architecture of the system.”

The Echo Nest’s technology could not have come at a more essential time. Thanks to piracy, internet radio, and iTunes, listeners were suddenly awash in an unprecedented amount of easily accessible music. “People had so many MP3 downloads from Napster,” Jehan says. “They were probably not listening to even, like, 5 percent of it. That was a natural thing — let’s just organize it all.”

As the Echo Nest began to grow, so did music streaming. The companies behind streaming platforms including iHeartRadio and Rhapsody, eager to imbue smarts into their user-recommendation algorithms, paid the Echo Nest to access the company’s “music intelligence.”

Eventually one longtime customer, Spotify, decided the Echo Nest’s talent and tech were too unique to share. The streaming service bought the Echo Nest, including its approximately 70 employees, for a reported $100 million in 2014. Whitman and Jehan came on as principal scientists; Whitman oversaw Spotify’s personalization and discovery teams, while Jehan helmed audio R&D projects. Whitman is now based in New York, but Jehan still works at the same office building where the Echo Nest launched just over a decade ago in the Boston suburb of Somerville. Initially, the company had space for only a single small office that accommodated about five workers. Today, rebranded as Spotify Boston, the company occupies three of the building’s floors. “The people that were there are still there,” Jehan says. “The same group is in that same office.”

While most of its employees are still in the same building, the Echo Nest is working on a much larger scale. At the time of the acquisition, Spotify CEO Daniel Ek pledged to use the Echo Nest to build the “best music intelligence platform on the planet.”

The Friendliest Algorithm

A t Spotify’s lower Manhattan office, a listening lounge styled like a Victorian library leads into a sprawling open workspace housing many of the company’s engineers. Instead of literary works, the room’s bookshelf is lined with classical vinyl records: The Best of Gershwin, Brahms: Symphony No. 1, The Rite of Spring. A vintage-styled Delmonico record player stands next to the couch, and in the corner of the room sits an ancient Roman-style bust of mascot of Major Lazer, whose smash hit “Lean On” became Spotify’s most streamed song ever in 2015.

The “old meets new” angle extends to Spotify’s products. Discover Weekly, one of the company’s most successful recommendation features, mixes human touch with algorithmic scale. Each week, every user gets a roughly two-hour playlist of songs they likely haven’t heard but may be predisposed to enjoying. The songs are culled from the actual playlists of other users with similar tastes (Spotify has built an intricate listening dossier of each of its users that it calls a Taste Profile). With more than 2 billion user-generated playlists on the service already, Spotify’s algorithms have a wide range of options to appease both Top 40 lovers and audiophiles looking for the next hot — preferably obscure — thing.

“We’re obviously relying on our expert users who know so much about music to make that work really well,” Whitman says.

It’s a winning strategy. Since launching in July, Discover Weekly has been tried by more than 40 million of Spotify’s estimated 100 million users. More than half of Discover Weekly listeners play at least 10 songs per week. Overall the feature has racked up 5 billion streams in less than a year. Every Monday, you can find people on Twitter gushing about how a list of songs knows them better than their boyfriend, their best friend, or their own inner selves.

A key part of Discover Weekly’s appeal is its intimacy. Spotify initially tested a 100-song version of the feature but found that combing through that many songs was exhausting for users. At about 30 tracks, the weekly playlist isn’t too much longer than the mixtapes friends traded on cassettes and CD-Rs in generations past — there’s almost a nostalgic element to it. [“Discover Weekly] taps into a key way in which people discover new music,” says Don Knox, a senior lecturer in audio technology at Glasgow Caledonian University. “Research has shown that one of the main ways in which we discover new music is through friends and family.”

The mixes also appeal to users trying to discern what Spotify’s algorithms may have divined about their personalities. “[Discover Weekly] indicates that Spotify cares about you,” says Catherine Moore, a music business professor at New York University. ”It’s like having a friend that you check in with, and you know some things are never gonna change, but you know there’s going to be new things on a regular basis.”

Discover Weekly is just one of the personalized products Spotify has pushed over the last year. A feature called Running, which was initially prototyped by Jehan, picks upbeat songs based on a listener’s jogging tempo and adjusts tracks based on the user’s listening history. Another playlist, Fresh Finds, uses Echo Nest tech to scour the web for lesser-known acts critics and fans are buzzing about, then finds which of Spotify’s hippest users are also talking about these new artists. These users are dubbed tastemakers, and their listening habits help power the Fresh Finds lists. According to Whitman, Spotify’s hope is that up-and-coming artists can first gain traction through Fresh Finds, and then, as they are placed in more users’ playlists, land in more Discover Weekly offerings. The company has been aggressively trying to market these products as a boon for artists seeking their big break, amid ongoing criticism that the royalty rates it pays musicians are too low.

Spotify may actually be able to help launch some superstars (it’s happened before). But despite all its tech-heavy smarts, the company still has to wrestle with the fact that a lot of today’s biggest names are choosing to keep their work off the service.

Star Power Problems

S ince its 2014 spat with Swift, Spotify’s content problem has only gotten worse. In 2016 alone, Rihanna, Kanye West, Drake, Radiohead, Chance the Rapper, and The 1975 all debuted major releases on competing services Tidal or Apple Music, often waiting weeks to bring their albums to Spotify. Swift, Adele, and Beyoncé — arguably the three biggest music stars in the world today — have all chosen to keep their newest albums off Spotify indefinitely.

The reason, of course, is money. Drake and Swift have both inked promotional deals with Apple, appearing in Apple Music ads and offering exclusive content for the service. Kanye, Rihanna, and Beyoncé are all partial owners in Tidal, so putting their titles on that platform has obvious financial benefits.

These exclusives have spurred subscriptions, at least in the short term. Last year, Tidal was growing at a glacial pace, reaching 1 million subscribers in September. After Kanye debuted The Life of Pablo on Tidal in February, the service shot to the top of the iOS App Store. Two months later, when Beyoncé dropped Lemonade as a Tidal exclusive, the service gained 1.2 million new users in the week following its release. (The jolt of a big new album still hasn’t led to consistent growth, though; Tidal isn’t currently in the top 100 most-downloaded apps, while Spotify is floating in the top 15.)

Despite increased competition, Spotify is not bending. The company has been reticent to open its wallet for big-name albums, and it hasn’t ceded to artist demands to make some music exclusive to its premium tier. Some famous artists believe the proliferation of free music on platforms such as Spotify depresses album sales — last year Adele told Time that streaming makes music feel “disposable.” When asked in February how the company planned to lure back Swift and Adele, CEO Daniel Ek offered only vague pledges to pay artists fairly and build ties to the creative community.

Ek wants to avoid bidding wars with his competitors, and for good reason: Spotify is already bleeding money. The company has been in the red for years, and in 2015 it lost $206 million. The year before, it lost $184 million. So what is Spotify spending all that money on? The vast majority of its expenses go to licensing costs paid to labels and publishers, which amounted to $1.8 billion last year. Adding sweetheart deals with individual artists on top of these already sky-high costs is the last thing the streaming service wants to do, especially as it mulls an IPO.

Despite a serious lack of current star power and steady hum of Twitter outrage, Spotify isn’t exactly hurting. With more than 30 million paying subscribers, it’s far ahead of competitors Apple Music, with 13 million subscribers, and Tidal, with 3 million (there are, of course, other competitors such as Google Play Music, Rhapsody, and Deezer, but they are thought to be trailing the three larger platforms and generally don’t land A-list exclusives).

Building smart recommendation algorithms is a better long-term strategy than bidding for specific albums, according to Moore. “To me, exclusive content is a short-term strategy, partly because artists will want to reserve the right to change their minds,” Moore says, citing Kanye’s The Life of Pablo ending up on Apple Music after he vowed it never would. “Music fans can [also] go somewhere else. They can go to unauthorized sites, bittorrent sites — all these other places — and still get the music.”

Expect Spotify to double down on discovery in the future. Whitman is interested in rolling out new versions of Discover Weekly, perhaps focusing specifically on new releases. Ek has identified sleep as a daily routine that Spotify wants to improve. Of course, just because Spotify perfected the algorithm first doesn’t mean competitors won’t catch on. New recommendation features will likely be part of the revamped version of Apple Music that’s expected to be unveiled at Apple’s Worldwide Developers Conference later this month.

Heightened competition will just give engineers like Jehan and Whitman more incentive to deliver a better product. The days of a streaming service serving as a stand-in for your hippest friend are just beginning.

“I hope that we get everybody obviously, and I hope that we don’t have to deal with a world in which you have to go to a certain service to use a certain artist,” Whitman says. “I just see discovery as really important no matter what.”

An earlier version of this piece incorrectly stated that Spotify added 89 million users in 2015; the correct number is 29 million. Also, the piece said that Spotify’s lower Manhattan office contains a bust of Major Lazer; the bust is actually of the group’s mascot.