Wikimedia: We’re Really Really Not Building a Search Engine

The nonprofit that runs Wikipedia says the idea it was competing a Google competitor has been a big misunderstanding.

Feb 16 2016, 9:23pm

Image: Ewan McIntosh/Flickr

The Wikimedia Foundation, which founded and operates Wikipedia, is most definitely not building a search engine with an aim to compete with Google, the nonprofit organization emphatically declared Tuesday.

In a blog post designed to clear up a month of confusion among its volunteer community members, Wikimedia executive director Lila Tretikov and vice president of product Wes Moran wrote "we're not building a global crawler search engine. We're not building another, separate Wikimedia project. We're committed to our mission of helping the world access and interact with free knowledge."

Why the confusion? Because that statement (and other, previous public-facing statements about the project) is in direct contrast to a number of internal documents and a grant proposal to the Knight Foundation for the "Wikimedia Knowledge Engine," now known as "Wikimedia Discovery."

Leaked internal Wikimedia documents pitch the Knowledge Engine as a "new site [that] will be the internet's first transparent search engine, and the first one that carries the reputation of Wikipedia and the Wikimedia Foundation."

"The core issue here is about transparency"

As we explained on Monday, the prospect of Wikimedia getting into the more algorithmically-minded search engine game is a scary one for the human editors of its encyclopedia. But more importantly, those in the community say that Wikimedia hasn't been forthcoming with what its plan for Wikimedia Discovery actually is; a reversal from the organization's normal ethos of complete transparency.

The organization says that the current version of Wikimedia Discovery will specifically focus on improving search within Wikimedia products such as Wikipedia, which has been what the group has said outwardly several times.

Much of the backlash against the idea has come from the community, which is upset that Wikimedia considered committing itself to a several-year project that will cost millions of dollars (via a grant application) without consulting volunteers and without including the project in its annual plan, which has always been publicly available.

In an email to Motherboard, Wikimedia spokesperson Samantha Lien said it did not initially publish the Knight Foundation grant proposal as a matter of policy.

"Generally, we do not post donor documents without advance agreement as a matter of donor privacy," the spokesperson said.

In the blog post, Moran and Tretikov said, basically, that plans change.

"It is true that our path to this point has not always been smooth, especially through the ideation phase," Moran and Tretikov wrote. "By sharing our publicly available product goals, planning, status updates and the Knowledge Engine grant agreement, we are making an effort to improve our collaboration."

Whether the blog post is enough to temper unrest in the community is another question altogether. Several people who I spoke to said that they now believe Wikimedia isn't developing a "secret search engine," but believe that's only the case because Wikimedia explored the idea without input from the community and deemed it unfeasible or unpopular.

"I think the Foundation's leadership would like to give the impression that this is all just a miscommunication, but at some point a failure to communicate becomes secrecy," William Beutler, a longtime Wikipedia editor, journalist, and author of The Wikipedian blog, told me. "Leaving aside whether a search engine is a good idea, let alone feasible, the core issue here is about transparency. The irony is that the Wikimedia Foundation failed to observe one of the movement's own core values, and that's what led to this."

In any case, it appears as though Wikimedia Foundation is moving forward with a plan to fix the site's search, using the $250,000 grant it received from the Knight Foundation.

"We have documented longstanding issues around search on Wikipedia and the Wikimedia projects. The Wikimedia community has indicated—through surveys, support tickets, and direct feedback—that our search features could be improved," Lien told me. "Our objective is to improve people's ability to find content across Wikipedia and the Wikimedia projects."