Are Duolingo Users Actually Learning Anything Useful?

When I first tried the free language learning app Duolingo, which calls itself "the most popular way to learn languages in the world," I was delighted. I told it that I wanted to beef up my Latin American Spanish, and it launched into a brief quiz that took me on a whirlwind tour of the language. The quiz took me from insultingly basic phrases like "hola," to the stuff I got sick of in 9th grade like "¿Dónde está mi bolígrafo?," to tougher stuff in a few minutes. At the end, I was told I was 65 percent fluent.

That's a grade worth bragging about, considering my Spanish is complete trash.

Anyone who has ever heard me try to communicate in Spanish knows I barely muddle through it. My attempts at conversation in the language often fail completely; if the other person can speak English, the conversations end up taking place in English instead. I encounter opportunities to speak Spanish every day of my life, but I break out in a cold sweat when I have to—so getting better at speaking my city's second pseudo-official language seemed essential.

I enjoyed accruing Duolingo's currency, lingots, and got a rush from competing with myself thanks to the app's lauded gamification. I also enjoyed the cartoon graphics and weirdly dark sentences embedded in the lessons ("He can't swim!" "I have no nationality!"). But after a few months with Duolingo, the most popular educational app in iTunes, when it came time to speak Spanish, I hadn't noticeably improved. What gives?

To find out, I checked in with linguists who focus on language acquisition. Most of them weren't into Duolingo at all, nor did they love its more expensive cousins like Wanikani and Rosetta Stone. "Many of us in academia have quite a bit of disdain for the endless series of gadgets and apps that are supposedly going to solve all our problems," Robert M. De Keyser, professor of second language acquisition at University of Maryland's School of Languages, Literatures, and Cultures, told me in an email.

According to Roumen Vesselinov, assistant professor of statistics at Queens College at The City University of New York, "Generally [Duolingo] seems to work." In an experiment commissioned by Duolingo itself, Vesselinov and his team found that using Duolingo for 34 hours is about the equivalent of a one-semester language class. It's an impressive result to be sure.

However, Vesselinov also told me that the evaluation process relied on a text-based online quiz called WebCAPE. "In other studies, with other software packages, we were able to use oral tests," he told me, referring to comparable tests of the language applications Rosetta Stone and Busuu. "You give a recording to evaluators, they listen, and they evaluate your communication skills," he explained, and he said this produces a result that is "more objective." Vesselinov said it "would be nice to test Duolingo for both vocabulary and oral proficiency."

What's more, he said, "the biggest measurable progress is when you start from zero." According to his report, the difference between beginner and advanced progress was "noteworthy" and not "statistically significant," because of the small sample size. But with beginners achieving 9.2 test points of progress on the WebCAPE, and more advanced students achieving a paltry 0.6 points of progress, Vesselinov wrote that, "with [a] larger sample, the result would have been significant."

"It's more difficult when you're at an advanced stage to improve further," he told me point blank.

Since I'm not able to have what I think is a normal conversation, I'm not at what I consider an advanced stage in my Spanish knowledge—but I seem to have reached a plateau. This is most likely because, as MIT linguist Suzanne Flynn told me in an email, apps like Duolingo are "good for learning new vocabulary at best," and don't have "what is needed for true language acquisition to take place—immersion or immersion-like language experiences."

Immersion is, of course, the gold standard of language teaching methods; generally, anyone who teaches a language recommends moving for a while so you can be among native speakers if you want to make real progress. But I'd imagined that, at the intermediate and upper levels, an app like Duolingo (or Rosetta Stone, if I wanted to spend more money) could get me close enough, and that I'd only need immersion if I wanted to speak with the panache of a native.

I now know from experience that an app won't even let me successfully eavesdrop on a conversation between two native speakers.

Robert Daland, a UCLA linguistics professor, explained that my ability to form a sentence pieced together from snippets, as learned from Duolingo, isn't even fighting half the battle. "An equally important part of language acquisition—in fact, I would say maybe even a more important component—is perception, the ability to understand what someone means when they say something to you." He said each language comes with unique challenges when it comes to just understanding a normal sentence—things he referred to as "lazinesses or sloppinesses," that make everyday speech baffling to a non-native speaker.

Let's take an example, in English: Go ahead and say the elementary sentence, "It's in that green box over there," out loud. According to Daland, you just confused any non-native speaker by referring to a "greem box." The unwritten rules of spoken English say that when you hear an "m" at the end of green, that's actually an "n," and you've probably never given it a second thought. And it's not just that all languages allow for muddled consonants, he explained. "The way languages allow this is different," and currently, no app really teaches this kind of thing, even though the problems it presents materialize at a very basic level of speech.

"Any kind of strategy that's going to help you learn words is going to beneficial for you in the long run, as long as you're learning frequent and useful words," Daland told me. But I'll never, ever get anywhere close to fluent with Duolingo. "It is going to possibly be a very helpful part of a multi-component program," he said.

Follow Mike Pearl on Twitter

.