This story is over 5 years old.

Teaching Robots to Appreciate Poetry

Turning metaphor into metadata.
May 20, 2015, 11:00am

Over the course of 1967 and 1968, Argentine writer Jorge Luis Borges delivered a series of lectures at Harvard about the nature of human language. In one of these lectures, he spent a good deal of time ruminating on the importance of metaphor and its limitless possibilities in language. Borges theorized that despite these boundless possibilities for poetic language, there were nevertheless distinct patterns of metaphors that kept cropping up—a favorite example of his being the metaphorical equivalence of "stars" and "eyes."

It was this lecture series given by the surrealist writer that inspired Poetry for Robots, a project launched last week through a partnership between Neologic, Webvisions, and The Center for Science and the Imagination at Arizona State University. The project seeks to put Borges' theory to the test, asking on their website whether it is possible to teach machines the poetic quality of human language.

"I want to experiment with robots thinking more like we do rather than vice versa."

If we really do cluster our metaphors as Borges thought, we should also be able to manipulate metadata with poetry, theoretically allowing Google search results for "eyes" to return images of stars in addition to pupils.

"In an image databank, you're often searching metadata that's really poor," said

Corey Pressman, a partner at Neologic, a digital agency that helps clients build websites, apps, and other digital experiences. "It doesn't match how we actually think about the world. As a species, we think about it metaphorically—poetically."

The Poetry for Robots experiment has two parts. The first is the data entry side, which just launched last week. In this phase, the team is hoping that it can crowdsource poetry in response to a set of 120 images on the site to use as metadata for its image database. Users are asked to input poems up to about 150 characters which respond to a particular image—an endless expanse of sea, a girl holding a sparkler, a bowl of candies, or whatever you find that moves you.

The team will be accepting responses throughout the summer, compiling the poetry into a large collection of metadata for their image database. They plan to showcase their results at Webvisions Chicago in September, where they hope to prove Borges' theory correct, contributing to a paradigm shift from machine learning which is cold, scientific, and literal to ludic, poetic, and human.

According to Pressman, the Poetry for Robots project is much more than a mere novelty—it has the potential to revolutionize the way we interact with machines. As it currently stands, in order to search image databases, humans must "think like machines"—if you want a picture of stars, you search "stars," not "eyes." Despite the efficacy of this utilitarian approach, it begins to break down at the boundary of the human, where abstract experiences such as "sadness" or "beauty" are difficult to quantify as image metadata.

"I want to experiment with robots thinking more like we do rather than vice versa, so that an image search for emotions give us images that resonate with us," said Pressman. "Robots are a tool so I'd rather have them think more like us. Ultimately you can get richer, more realistic search results."

The second application of the Poetry for Robots project largely depends on the success of the metadata portion. If all goes as expected, the ability for robots to search like a human could also lead to the ability for robots to write poetry as well.

While human creative endeavors have often been seen as the only safe refuge from the robot takeover, computers have already successfully demonstrated that they are capable of engaging in creative processes. Machines have had their poetry accepted in literary magazines and a number of bots source tweets to use as fodder for poetry they spew into the twittersphere. Perhaps most poetic of all, there is a computer that scrawls self-effacing poems into the sand.

Pressman, who is himself a poet, is the first to admit that there is something unsettling about robots creating art. However despite the uncanniness of artistic machines, he cites Brian Eno and Peter Chilver's generative music applicationBloom as a particularly instructive example of the beauty that can still be found in machine generated art.

"Generative music doesn't seem cold to me—I dig it," said Pressman. "At the same time, part of the glory of getting turned on by a Pattiann Rogers poem, for example, is that you know someone wrote it. You're resonating with how she sees the world so you feel less alone. People probably get a little uncomfortable with [Poetry for Robots] because poetry is so much about empathy and empathizing with another person."

The Poetry for Robots project has the capacity to drastically improve this relatively rudimentary field of creative machine learning by allowing computers to better understand the essence of metaphorical language, which if Borges is right, is a large part of what makes us human. As neo-Luddites of all stripes lament the perceived mechanization of humanity, Poetry for Robots is striving to reverse this trend by humanizing the machines, for better or worse.

"In 1989 Norman Cousins wrote a piece saying that computers are coming and let's not have them dehumanize us. He recommended that in order to do that, every time they have a technician working on something that they should hire a poet to work with them," said Pressman. "As the robots proliferate in our lives, in order to keep them humanized, we need to not just have computer scientists in the room, but have the arts and humanities side by side with what they are doing. I think that's the real beauty of what is going on here."