Apple Has Been Struggling with the Stylus for 20 Years

Motherboard staff is exploring the cultural, political, and social influence of the iPhone for the 10th anniversary of its release. Follow along.

The 2010 release of the iPad, arriving mostly fully formed as a larger variation of the iPhone, had a weird way of glossing over decades of work in tablet computing.

It was not the first tablet, nor was it the first tablet that could be considered popular (that was the GRiDPad, a featured part of RadioShack's recent estate auction that was a hit with businesses in the early '90s, generating tens of millions of dollars in sales for GRiD Systems). But the iPad probably was the first one that any of your friends had ever used.

Certainly, it wasn't the first foray into tablet computing by Apple, which had experimented with a variety of handheld computing devices over the prior two decades. Most people are familiar with the Apple Newton (thanks in part to a comic strip and a cartoon— Doonesbury and The Simpsons, respectively), but at the same time Apple was working on that, it had commissioned work on a tablet-style computer that used many of the basic elements of its PowerBook line.

But little was known about it—rarely reported at the time, it lost out internally to the Newton in a fairly depressing way. Paul Mercer, who led the project, known internally as PenMac, led a team in creating a fairly impressive example.

Called the PenLite, the tablet device was designed to work with a stylus, but Apple never fully followed through. It became an obscure example of a Mac prototype that never saw the light of day. More people know of Apple's notorious Pippin video game console than the PenLite, despite the fact that it was clearly a more influential device on Apple's trajectory.

So what happened? A 2006 New York Times article discussing a career shift by Mercer noted that the vision for the device was lost on Apple's then-CEO, John Sculley:

Business deals are a horrible reason, of course, to stop developing an interesting idea. And the story of the PenLite got me wondering about what we were missing at the time.

Fortunately, the people who actually worked on these devices are still around.

Back in January 2010, as the buzz around the iPad, which had yet to be announced, was just starting to reach a fever pitch, developer Arno Gourdol wrote a blog post pondering whether Apple had finally gotten things right this time. He knew what the prototype looked like, its weaknesses, along with its potential.

"Despite the low resolution of the screen compared to what today's technology can offer, the form factor of the device, the ability to cradle it in your arms, made for a fundamentally different, more intimate, experience than a laptop," Gourdol wrote at the time.

Gourdol, a longtime software developer who spent years working with both Apple and Adobe, has created products you've probably used—he at one point was the lead engineer for the MacOS user interface and also helped Adobe build out, among other things, its Creative Cloud platform. But it was PenMac, a software tool that hasn't seen a whole lot of time in the Sun, that might be the most curious and interesting.

The stylus-based user interface for MacOS was one of his first projects during a decade-long career at Apple, one he started as an intern. The platform was an attempt by Apple to answer a big question about the viability of handheld computing. The company had two separate groups attempting to answer the question at once—the PenLite group, and the Newton group.

"The project started when a brilliant engineer managed to get MacOS to boot on a 68000-based PDA from Sharp (I believe)," Gourdol told me over email. "It was ridiculous, of course, because you had a minuscule black and white screen and the UI was unusable, but it was a proof of concept. The Newton had started with the premise: 'What if we could reinvent the user experience from scratch?' This project was asking: 'What if you could still run the Mac apps you love?' Ultimately it wasn't clear what was the better approach, and so both projects went on."

As history has shown, the Newton won out, but the project did answer a few questions. For one thing, it was clear, even as the project ended prematurely, that simply adding pen capabilities to a desktop computing experience wasn't enough.

In his 2010 blog post, he laid out the problem evocatively: "Repurposing for a tablet a UI designed for a desktop is like eating pea soup with a fork."

While information on the PenLite is somewhat hard to come by online, there is information floating around. Perhaps the most obscure and intriguing is a 2012 YouTube clip, nearly an hour in length, that features early Apple engineer Thomas Gilley going through the paces of the PenLite as part of the long-running patent case involving Apple and Samsung.

The perk of this, of course, is that it shows the layperson exactly what they would be getting into with the PenLite.

Gilley's lengthy clip shows the harsh realities of pea-soup fork eating—which, to be fair, were also problems for Windows-based pen interfaces at the time. Closer in look to the desktop Macintosh than anything resembling the Newton or iOS, the interface is still driven by both window interfaces and Finder.

One fairly significant difference was the stylus, which he used for purposes of writing and text editing in one part of the clip. Writing "this is a test" in pen was close, but no cigar, with the lettering converting to "*hisis_test" instead.

But perhaps the most telling part of the clip involved the complexities of opening up windows. Dragging menus with a pen clearly is not as easy as a mouse. Nor is clicking. And while a touch-based interface was added to later Apple tablet prototypes, it was pen-only on the PenLite.

In some ways, it feels like the early GUI work of Xerox PARC, exposed to the public for the first time, except in tablet form. While technically impressive, it does show, in retrospect, why Apple went with the Newton instead.

When asking Gourdol about the computing experience of a PenLite, he suggested it was usable and offered the familiarity of MacOS, but it would be challenging for the device to compete with the usefulness of a traditional desktop machine for a full work experience. "Overall, it worked pretty well for some simple things, but you'd never really reach 'fluency' with it," he said.

While the PenLite was unceremoniously put out of its misery, Apple did continue to experiment with the form in the years before Steve Jobs returned to the company, but with nowhere near the level of sophistication of the iPad. The most notable experiment of this nature was the Vademecum, a very rough prototype that included a built-in camera and wireless networking functionality, but was far from the slick iPad experience we're familiar with today.

"In 1994 the tablet idea was rethought; it was clear that tablets would exist one day, and the team wanted to experiment with key design factors," designer and writer Marcin Wichary explained on a Flickr page highlighting the prototype. "Hence the handle, hence the camera, hence the wireless… the team felt these would all make the tablet a new proposition."

(Eventually, they all did—minus the handle, of course. It just took a while.)

Clearly a laptop with a tablet form factor wasn't as far along as it could have been, considering the technological limits of the era. But there were probably some big lessons to be taken away from the work that went into the PenLite a quarter-century ago.

If anything, Gougnol says, the iPad was almost too conservative in its approach to the tablet format upon its big public launch.

"The thing that surprises me the most is how long it is taking for Apple to realize the potential of the iPad," Gougnol explained, noting the power and the sophistication of the hardware were truly impressive.

"But I feel like the software has been lagging behind. The first version were just too close to the iPhone," he added. "Apple provided support for UI layouts that would take advantage of the larger screen size, but the rest of the experience remained unchanged. With the form factor of the iPad, there is the potential for a user experience that is radically different than what you could do on the small screen of a phone."

Part of the problem, he says, is that the iPad has traditionally shied away from multitasking, meaning that doing multiple things at once is difficult, if not impossible. He did give Apple credit with iOS 11 for finally moving in this general direction, "but it feels like it's been taking a long time. Maybe I'm just impatient."

This, he says, stands in contrast with what we've seen from Microsoft, the initial target of his "pea soup" comment. "I can see that they struggled with the same questions we did: 'design a brand new UI from scratch, but let go of compatibility with existing apps,' or 'adapt the UI so that existing apps continue to work,'" he suggested. "I think they've tried to navigate a path between those two, but clearly their apps (and to an extent the apps from third party developers) have evolved as well to work much better with touch."

He suggests that Apple's conservatism—its unwillingness to find this common ground between the desktop experience and a tablet experience that isn't simply focused on touch—could be hurting it. For years, Apple was wary of the stylus with the iPad, before finally embracing it with the Apple Pencil and the iPad Pro. And that's still happening with the keyboard and mouse equation, which he notes Microsoft and Google have made much progress on.

"Even today, it doesn't feel like things are settled," he says.

Perhaps the secret to eating pea soup is that you simply need a better fork.