The Poem That Passed the Turing Test
They should have sent a computer.
Feb 5 2015, 4:10pm
'For a Bristlecone Pine', by A Computer
More from vice
A home transformed by the lightning
In 2011, the editors of one of the nation's oldest student-run literary journals selected a short poem called "For the Bristlecone Snag" for publication in its Fall issue. The poem seems environmentally themed, strikes an aggressive tone, and contains a few of the clunky turns of phrase overwhelmingly common to collegiate poetry. It's unremarkable, mostly, except for one other thing: It was written by a computer algorithm, and nobody could tell.
Zackary Scholl, then an undergrad at Duke University, had modified a program that utilized a context-free grammar system to spit out full-length, auto-generated poems. "It works by having the poem dissected into smaller components: stanzas, lines, phrases, then verbs, adjectives, and nouns," Scholl explained. "When a call to create a poem is made, then it randomly selects components of the poem and recursively generates each of those."
Scholl's work forms part of a small but burgeoning canon of algorithmically abetted poetry and prose—from bots that mine Twitter to build sonnets in iambic pentameter to poem drones that scrawl lines on sidewalks to automated novel-generators, the gap between man and machine-made art has, ever so slightly, begun to close.
In 2010, Scholl began submitting the output to online poetry websites, in order to gauge reader reaction, which he says was "overwhelmingly positive." The year after that, he sent his auto-generated poems to literary magazines, where they were rejected from the likes of Memoir Journal and First Writer Poetry. Scholl then submitted a battery of poems written by his algorithm to the Duke literary journal, The Archive. One was accepted. Here's its text in full:
A home transformed by the lightning
the balanced alcoves smother
this insatiable earth of a planet, Earth.
They attacked it with mechanical horns
because they love you, love, in fire and wind.
You say, what is the time waiting for in its spring?
I tell you it is waiting for your branch that flows,
because you are a sweet-smelling diamond architecture
that does not know why it grows.
He never told the editors that the poem was 'written' by what he considers to be an artificial intelligence. "I didn't want to embarrass anybody," Scholl told me.
Four years later, Scholl, now a PhD candidate in computational biology, published a blog post revealing his stunt, "Turing Test: Passed, Using Computer-Generated Poetry." The Turing Test, of course, is named after the freshly Oscar-famous mathematician and computer scientist Alan Turing, and describes the renowned thought experiment designed to determine whether an AI could be smart enough to successfully imitate a human. In the most famous iteration, an interrogator tries to discern which of two unseen responders—one of which is a machine—is human. If he or she is fooled into thinking the program is the more human-like of the two, it passes the Turing Test.
"This AI can create poetry indistinguishable from real poets"
Scholl contends his poetry generator satisfies some version of the test. "This AI can create poetry indistinguishable from real poets," he wrote. "The real Turing Test of this AI was to get it accepted to a literary journal, which was accomplished—this poetry was successfully accepted into a literary journal at a prestigious university."
Of course, AI scholars would likely be skeptical—after all, last year, when the much more sophisticated chatbot Eugene Goostman "passed" the Turing Test by posing as a Russian teenager who tapped out answers to human questions in in broken English, many in the AI community cried foul. (It should also be pointed out that many consider the Test itself to be a flawed and highly arbitrary standard for gauging artificial intelligence.) Sneaking a robot-generated poem into an undergraduate literary journal is a similarly insufficient standard for proving the creep of machine intelligence; poetry is often ambiguous and bizarre, and its appraisers were judging 'Bristlecone' for its originality and content as much as its humanity.
"We want to showcase a breadth of authors and a breadth of styles. We're not just publishing the pieces that win the most votes," Elizabeth Beam told me. Beam was one of the editors-in-chief of the Archive when "Bristlecone" was published. She said she didn't remember "anything of substance" about the poem, except that it was original.
Beam graduated from Duke with a BS in neuroscience and a BA in English, and now works as a research assistant at Harvard's Buckner Laboratory, where she studies the neural, genetic, and behavioral differences in people with clinical anxiety. When she dug up her notes on the "Bristlecone" poem, all she found was that she'd underlined the last three lines of the poem, and jotted the terse note, "Yes, type of tree." She wasn't responsible for casting any votes in the selection process; the editors-in-chief refrained from voting after they'd gathered and submitted the poems.
"I think that's why we published this poem—because it was intriguing. It was not trite. And this was the most coherent one."
Scholl had sent in 26 poems, one for each letter of the alphabet, and "Bristlecone" was the only one that was published. The fact that Scholl sent so many poems in also likely influenced the vote count—this was a student publication, after all, and was aimed at inclusivity and encouraging budding writers to keep at it.
So, if this is to be considered a milestone—a marker on the road to autonomous robot artistry—it's a vanishingly little one. Still, it's an interesting little milestone; none of the poets or coders I've spoken to knew of another machine-generated poem that was accepted for publication and published as if authored by a human.
Most algorithmic language artworks are unabashed in their artificiality; the aforementioned Pentametron turns mundane tweets into real-life poetry. Ross Goodwin's fiction generator takes texts (like the CIA torture report, say) and transforms them into bizarre word-salad stories.
But Scholl isn't as interested in the novelty alone. "I do consider it just another way of doing poetry," he told me. Scholl is an avid poetry reader and student, and his knowledge has informed the library—all the possible syntax and word choices are contained in a .bnf file—that the auto-poems draw their language from. Scholl can then tweak the inputs to alter the poem's emotional content.
"This program works on the basis that every word in the English language is either 'positive' or 'negative,'" Scholl explains in the generator's Read Me file. "For instance 'lovely' is positive and 'thorn' is negative. A 'poem' is a group of sentences that are structured in a way to have +1, -1 or 0 in terms of the positivity/negativity. A 'mushy poem' is strictly positive." He sent me a few different "positive" and "negative" poems, and the difference was obvious.
"It's really cool to see the reaction," he said. He notes that it's been interesting to compare the reactions between those who had no idea the poems were written by a machine, and those who did. The comments on the poetry site, for instance, would say things like "What a wonderful piece of writing. you paint a vivid picture and I love the picture you have painted here. well done." On Reddit, where his revelatory blog post made a minor splash on r/futurology, the reaction was "just as beautiful."
Scholl acknowledges that the program is very basic: "The only thing it does is store information about poetic words. The reasoning is very simple."
"Maybe it is an AI," he added, "but a simpler one than speech recognition." And he intends to make it more nuanced yet. "There is room for improvement in this bot. More words, more styles of poetry. Words about buildings, environment, all kinds of classifications."
There's another note of irony at play here, Beam said: This issue of The Archive was explicitly designed to highlight the artifice inherent in the act of reading poetry.
"We were really trying to emphasize that this is an object that you're holding in your hands," she told me. "The words are skewed on the page, and it's hard to read."
"At the same time we were designing the magazine with this intention, someone was writing this non-subjective, authorless poetry," she said. It is indeed fair to call the poems themselves authorless; in fact, Scholl said, they can be 'written' by anyone. The code for the generator is open source, and available on Github. And the curious can go to Scholl's website and give the generator a spin for themselves.
"I think it's an interesting thing to do," Beam said. "It's certainly within the definition of art to create a program that writes a poem."
- Artificial Intelligence
- motherboard show