Advertisement
Sports

Inside Yahoo's Fantasy Sports Mission Control on NFL Game Day

We spent the opening NFL Sunday inside Yahoo Fantasy Sports headquarters with the engineers who make sure that everything runs smoothly and updates constantly.

by Tim and Dave Simpson
Oct 18 2016, 2:00pm

Tim Simpson

"This is our Super Bowl," said Kelly Hirano, who oversees the engineering department for all of Yahoo's sports enterprises. "This is our fantasy football season. If we screw it up this year then we can't fix it until next year."

It was just past 8 AM on the NFL's opening Sunday and an hour north, San Franciscans were jockeying for seats in sports bars. The Yahoo campus in Sunnyvale was abandoned except for one room, the fantasy sports section, where dozens of engineers watched huge televisions broadcasting pregame shows or, in most cases, the main event for the people who work in Yahoo's fantasy sports operation: the undulating graphs that monitor their system's response to raucous web traffic.

Would this year's product keep up with the demands of the country's millions of general managers? The engineers, who were brought in from departments across the company for this inaugural Sunday, were relaxed and excited; most of them wore jerseys. But as kickoff approached, and as desktop traffic spiked as meticulous GMs sought last minute lineup changes, the tenor in the room changed.

Read More: Kelli Masters Is an Agent for Change

The room looked like NASA's Launch Control Center before an Apollo liftoff, but with a banquet table covered in doughnuts, Warriors jerseys hanging from the rafters, and Peyton Manning, Lionel Messi, and Stephen Curry Fatheads pasted to the wall. The engineers—more than 40 of them, mostly men—stared at their screens. After kickoff, the desktop traffic would drop to almost zero, as nearly everyone switched over to mobile. Graphs were spiking on the monitors but the system was holding.

"When things go bad, everybody just focuses immediately on that problem," said Hirano, who was wearing a Doug Baldwin Seahawks jersey. "You can see the graphs here. You'll see something go wrong. We'll start to see it on Twitter. We'll start to see it ourselves. And then it just kind of goes quiet and then there's this general focus."

Hirano has developed DEFCON levels for big days like this, and his team runs practice scenarios for each level of potential disaster.

"You'd probably see a group peel off to room somewhere over there to get away from the larger group and kind of war room on it," said Guy Lake, the director of product management (red Pats jersey, red Pats hat). "I'm the social person so I'm checking out Twitter and responding to people. The mood in general in this room won't change much unless it's a catastrophe. We're not anticipating that. If it did though, yeah, you'd feel that for sure."

The rise of fantasy sports is common grist for hot takes: what it says about us as fans, the way we consume sports, male bonding, etc. Kelly Hirano and Guy Lake look at that world with X-ray goggles—a clear view into the heart of darkness of the fantasy GM. Hirano, in his pursuit to keep the system running, is responding to our reptilian fantasy needs.

Yahoo Fantasy Sports is, like many of society's great institutions, powered by donuts. Photo by Tim Simpson

Back in the spring of 1999, when Hirano helped launch Yahoo's first game, fantasy baseball, they updated stats overnight. "Today, expediency is critical," he said. "You see that touchdown on TV and you want to see that score on your phone or the website."

On opening Thursday, which in many ways can be a greater test than Sunday because everyone is watching the same game and therefore checking the app after the same big plays, Yahoo's scoring provider was about a minute or two behind the game. Twitter users let them know. The company switched providers and soon the scores were updating faster than the broadcast, which was operating on a seven-second delay.

"It's not that big of a deal," Lake said. "It has no impact on your scoring or the picks you made, but they really want that gratification as they're watching the game."

Lake's job is to find the fantasy id—to add new features that will keep users overloading Hirano's system.

Sometimes, this requires descending into the bowels of the internet, asking Reddit users what they'd like to see changed.

"[You] basically have to do that with armor plating because you're going to get absolutely shit on," Lake said. "It's just what happens. Basically, half the people will dump on you about a very specific feature that they want that you don't provide them. You will then have a good chunk of people who will say thank you for showing up to talk at all. That's cool. Honestly, I don't see anybody else from any of the other [places] like ESPN, or any of the large sites doing this. Then the other piece is you have people who say, Hey this bug happened. These things happened in our league. And that's actually very helpful for us."

Where Hirano has the calm demeanor of someone who has been weathering the ever-growing traffic pressure placed on his systems for 18 years, Lake is excitable and often philosophical when talking about the direction fantasy is heading. As a kid, he created his own college football rankings and made up players' stats. He affirms the notion that fantasy sports are a slightly less nerdy version of Dungeons and Dragons.

"I was an avid D&D guy back in the day, but luckily I also liked box scores and things worked out for me in the end," he said.

For many years, he worked as a product manager for an education software company, moonlighting as a fantasy sports blogger until he was able to merge those two skills at Yahoo.

"Every single time I go to a school event for my children and (people) ask what do you do for a job: I run the Yahoo fantasy sports team. Yes that's a real job," he said. "And they'll ask me all these questions about what's that like or say, My husband does that. You get a wide variety of responses, [from] feature requests to incredulity that that's actually a real game. I'm pretty self-aware that we're just doing fake sports."

Lake has watched the fantasy games change over the years, as the fan interest or the sports themselves have shifted, and he can rattle off stats about the state of fantasy sports.

Hirano and Michael La Guardia, head of product, monitor the monitors. Photo by Tim Simpson

Only 15 percent of fantasy leagues are public, and head-to-head has long since overtaken rotisserie-style scoring. As passing in the NFL increases, more than a third of private leagues use the points per receptions (PPR) stat, up from about 20 percent a few years ago. About 16 percent of the private leagues have slots for independent defensive players.

In baseball, which popularized fantasy sports but now pales in comparison to football, users are more statistically inclined, following the trend of sabermetricians who loath batting average statistic; on-base percentage and OPS are on the rise.

At the end of the day, however, the social nature of the game is the glue, Lake said.

Sunday's early games kicked off and the systems weathered an early Aaron Rodgers touchdown. Half a dozen kids were in the office, put to work by their dads, watching games and their phones, checking for immediate updates. Engineers stood, arms folded, staring at a wall covered with five large televisions in the shape of an X, the center one switched to RedZone, the outer four showing the green and black and white graphs.

The mood at Yahoo shifted steadily from focus toward celebration as elite quarterbacks hit elite wide receivers, as hundreds of thousands of GMs reached for their phones for that dopamine hit, and as the graphs bent but didn't break.

It's a year-round job. On Thanksgiving in 2013, Lake was at his in-laws' house when he got word that the You Pick'em scores weren't coming in and spent the night helping sort it out. But this moment, as the NFL season begins, and as the system is repeatedly tested and repeatedly passes, is really the end of a season for Yahoo's fantasy team. Most of the legwork is done. For better or worse, the system can't be changed drastically until next year. March Madness, and the days leading up to the tournament, is one of team's busiest stretches. The other is the summer leading into the football season. If a draft goes down, you've got 12 college friends who've flown in from across the country, sitting in a living room with nothing to talk about. If the system goes down on opening Sunday, it makes headlines.

Hirano remembers one 2 AM work session in the early years, when the programmer responsible for writing that year's draft code ran into problems out of the gate. Frustrated, the guy smashed his mouse on his desk, rendering it unusable, and Hirano had to race around the office, hunting for a replacement so the programmer could continue to code.

"That I think that states the pressure of the situation, where people on the outside don't realize because people on the outside, they see a draft or they go to any modern website, and they just expect the whole damn thing to work," he said. "And sometimes it's a well-oiled machine—I think we're there largely today—and sometimes it's a bunch of people with duct tape and string."

Late one Friday night a few years ago, under the impression that a Derek Jeter single had been incorrectly scored a strikeout, I sent an condescending e-mail to some Yahoo support address, largely venting after a bad fantasy week, under the assumption I was yelling into the void. A few minutes later, I got a polite response with a video of Jeter beating out a dropped third strike.

"You don't necessarily feel like you own a regular website even if you go there every single day," Lake said. "Fantasy? You own it. It is yours. If something is screwed up it's like someone went into your living room and messed with your stuff."

"That's why this level of attention is here," he added

The intimacy of the relationship the fantasy team feels with their obsessive users became even clearer to me on opening Sunday. The graphs, these vital signs of our obsession, are surging after every play for all to see. Someone tweeted at the Yahoo account that he was having a minor issue—something that might have only been impacting him and a handful of other people—but because they were operating so smoothly (within the 99th percentile, Hirano said proudly), an engineer had time to flip a few switches and solve the problem within minutes.

"People don't believe we read these things," Hirano said of some of the nastier comments. "We actually read almost all of it. So it does literally make me feel bad going home at night that we sort of failed the audience."

On the coming Sundays, if all continues to go well, the in-office staff will shrink, but Yahoo keeps staff members monitoring the graphs and scores overnight 365 days a year.

Competition within the office is strong, too. Most of the members of the team are avid sports fans but even if they're not, they have to play to test the system. The Development League was formed for that reason. It was initially intended only for developers, but other staffers like Lake wormed their way in and now there are multiple tiers for most sports. The bottom three finishers in the Development League get sent down to the Develegation League. Hirano was relegated last year. For football, there's even relegation league for the relegation league.

I asked Lake if, in leagues full of people with access to the backend of the game, anyone exploits loopholes. Insider trading, so to speak.

"We all can catch each other doing that, so that's the problem," he said. "There's no good way to pull that off."

One impatient Development League player invented what is arguably the most significant technological advancement in fantasy history, the thing that defines the new multi-screen way we ingest sports: "StatTracker," Hirano said. "That engineer decided that he wanted to get live stats. He wanted to get something that would update live on the screen. That actually was fairly revolutionary in the space to have that product."

In StatTracker, the drug that is fantasy sports had its syringe—the perfect vessel for delivery.

"We have millions of people hitting you all at once, going, 'How am I doing?' anytime there's a touchdown," Lake said. "And we're sending a push notification telling you that Isaiah Crowell scored. What do I do? Tap on it. Open the app and I look at it."

Want to read more stories like this from VICE Sports? Subscribe to our daily newsletter.