The Atlanta Scooter Ban Study Doesn’t Prove What It Thinks It Does

On August 9, 2019, Atlanta enacted a night-time ban on shared e-scooters and e-bikes after four scooter riders were killed by drivers, three at night and one in the early morning. Users were unable to unlock the devices from 9 p.m. until 4 a.m. Professor Omar Asensio at Georgia Tech saw an opportunity for a natural experiment. He could “test what users decide to do when suddenly micromobility devices were no longer available,” he told Motherboard. Do they take more car trips? Or do they walk or take transit instead?

The results of that question were published last week in the journal Nature Energy. The study attracted lots of attention, in part because the press release announcing the study from the university claimed to provide a “definitive answer” as to whether “electric scooters reduce car use.” It was posted in the r/science subreddit with almost 29 million members and got 32,000 upvotes. Axios ran a story on it under the headline “Study: Atlanta’s overnight scooter ban added hours to everyone’s commute” including by 37 percent after big events like soccer games, and bans like this around the country could cost $536 million in lost productivity every year 

But all of these headline-grabbing conclusions came not from the empirical findings of the study itself but by extrapolating very small changes in a very specific setting and applying them universally to the entire city and country. As a result, it is not a “definitive” answer to any important question surrounding e-scooter use and public policy, questions that have already been the subject of ample study from scholars around the world. 

There is a big difference between a scientific study that makes a valuable contribution to a body of work on a specific subject and one that provides a definitive answer to anything. The latter is exceedingly rare. The former is a daily occurrence. Asensio’s study is very much the former: a good study that will be cited by academics in the future and worth the time and energy he and his research team put into it. It also demonstrates there are few definitive answers to urban transportation questions.

Before we go any further, it is worth being absolutely clear about what the study actually investigated and what the results of that investigation were. 

Asensio and his research team had two important pieces of data. First was the night-time scooter ban enacted on a specific date, providing what he called a natural experiment. Before August 9, Atlantans could unlock and ride shared scooters and e-bikes at night. After that night, they could not. They also had access to the Uber Movement dataset for Atlanta, which provides public data on travel times around the city based on Uber ride data, anonymized to group census tracts, or geographic areas with populations between 1,200 and 8,000 people. The researchers then looked at travel times for 45 days before and after the night ban for three types of trips: In the city center (midtown), around Metropolitan Atlanta Rapid Transit Authority (MARTA) stations, and around Atlanta United Major League Soccer games at Mercedes-Benz Stadium.

Based on this data, the researchers found that, for the average commute in Fulton County, evening commutes increased approximately 2.3 to 4.2 minutes per trip. For the stadium experiment specifically, the researchers estimated much greater travel time increases of 11.9 minutes on average. 

The researchers admit these don’t sound like very impressive numbers: “Although a 2-to-5-minute delay for evening commuting and a 12-minute delay for special events could appear to be a minor inconvenience,” the study says, “the cost of additional time in traffic quickly adds up when aggregated across large commuter populations.” They then present high-level numbers: Between 327,000 and 784,000 additional commuting hours for Atlantans per year (this is where Axios’ headline comes from), or a $10.5 million annual loss in productivity. That sounds more meaningful.

However, there are two important points to make. The first is about the soccer games, which resulted in the biggest congestion increases. The problem is there weren’t many soccer games in the data set. In the 45 days prior to the scooter ban, Atlanta United played four home games. In the 45 days after, they played three, for a total of seven events. Four of these games took place in the afternoon and therefore would not have been affected by the scooter ban. There is no mention of this sample size issue in the study. And the attendance at these games varied from about 44,000 people to 68,000. Asensio said they “used a fix effects statistical estimator” so that “All observations for each game day (i.e., travel times to and from the stadium per surrounding stadium tracts) would be treated as a separate experiment that is then averaged across all games pre and post intervention” to account for differences in attendance. This resulted in a wider range of statistical outcomes than the other scenarios (7.4 to 16.3 minutes travel time increase). But three games affected by the scooter ban with widely varying attendance is not enough to draw any kind of definitive conclusion from. 

The midtown and MARTA station findings are more robust, but the travel time impacts are also much smaller. A two-to-four minute trip time increase must be put into wider context. And the way to do that is not to multiply trip time increases by population to come up with a very large number of total hours lost. Ironically, traffic engineers have been doing this for decades to argue against building the very kind of protected bike infrastructure that would make e-scooter and e-bike riding safer. In those scenarios, engineers calculate that it would cost drivers a few minutes per trip, extrapolate that to say it will cost drivers hundreds of thousands or even millions of lost hours per year, and decide not to do it. 

This methodological approach of adding up all the two-minute delays to reach one huge person-hours-lost figure misses the point of traveling in cities. The study is about changes to people’s individual behavior. Few people, if anyone, make meaningful changes to their behavior based on two or four minutes per trip. For years now, it has become increasingly common for engineers and planners at city and state transportation departments to put aside those degrees of travel time changes in favor of safety and environmental benefits when considering, for example, bike infrastructure or highway teardown projects. Cycling and micromobility advocates regularly fight for projects that would slow down car trips by more than five minutes as an acceptable trade-off to save lives. 

Again, this isn’t to criticize the study itself or any particular policy. Personally, I think it’s silly to ban e-scooters because drivers keep hitting the people on them. This is a problem with cars and road design, not the scooters. But the way the study is being presented and the reaction to it could well apply to drivers who don’t want their precious lanes repurposed for cyclists or commuters who want to keep their highway. And advocates for safer streets have been fighting for decades against that kind of reasoning.

In other words, the study doesn’t prove what people think it does. It does provide strong evidence traffic marginally increased after e-scooters and e-bikes were banned during the time the ban was in effect. The study also showed the travel time increases from the scooter ban are within the realm of acceptability for a safety intervention that saves lives. Whether the safety intervention itself is the right way to tackle the problem is a different question entirely, and one without any definitive answers. It is a political question about what Atlantans want their streets for.