Once upon a time, a company called Apple came out with a great concept: a breakthrough consumer device with a new user interface that left the competitors in the dust. It brought UI to a whole new level by introducing a new visual and gestural language which greatly increased ease of use. In doing so, it lowered the barrier to entry for the general public, created new markets for its products and a revolution occured. Sound familiar? It should. I’m talking about the introduction of the Macintosh in 1984. The new visual language of pointing, clicking, dragging and using overlaping windows gradually became the dominant UI paradigm. But here’s the problem: other companies stole Apple’s great ideas (which Apple had actually stolen from Xerox but never mind). What could have rocketed Apple to market dominance instead became a commodity that anyone could implement.
Flash forward to 2007. Apple again comes out with a new UI paradigm, together with a visual and gestural language, and they release it as part of a breakthrough consumer device; the first of a series of devices in different form factors which they think will undo the last 20 years and rocket them to dominance of all things digital. But this time, they’ve got an ace up their sleeve: a string of patents. As Wired reported in February, Apple is trying to patent the gestures that make up the iPhone UI – the iPhone’s equivalent of “point and click.” In fact, if Apple’s efforts succeed, I think they will be shooting themselves in the foot. Why? Because if we are, en masse, to move to a new user interface paradigm, beyond point and click, we are going to have to have some consistency. If “pinching” means “shrink” on one device and “close” on another device, this would be a disaster from a user experience standpoint, and could turn potential users off in a big way.
In fact, we don’t have to imagine for too long because some of new breed of “iPhone killer” devices now hitting the streets exhibit this very problem. I was just looking at a touch UI device manufactured by an un-named Korean company (that also coincidentally manufactured my fridge which now is on the blink after only 3 years of ownership – not that I hold a grudge). The problem with this device was that it was replicating a non-touch UI (a UI controlled by a four-way rocker switch) with a touch-screen overlaid on top. It wasn’t quite as bad as the Prada phone that I wrote up last year, but it was close. For example, instead of scrolling by simply flicking your finger up and down, it required you to (repeatedly) press soft buttons at the bottom of the screen labeled with up and down arrows. I haven’t actually had the chance to test out the Nokia “touch” Series-60 device, but when I read this article in News.com with accompanying spy shot, my blood chilled. A scroll bar? Menu buttons on the bottom? Could it be that Nokia is falling into the same trap – trying to replicate a button-based UI with a touch screen overlaid on top? I sincerely hope not — indeed, I think Nokia has enough UI expertise to understand that touch needs a new visual gestural language.
But this brings us back to Apple and their patents. I am not a lawyer, but I don’t believe patenting gestures is a good idea. It seems like there’s plenty of prior art – take a look at Jeff Hann’s talk at TED on gesture-based UI as an example – but the main thing is: in order for us to move into this brave new world of touch, I would argue that gestures need to be royalty free, and companies need to know that if they implement commonly used gestures they will not be sued. If anything, we need standardization of gestures so that users can have some kind of consistency between touch-based platforms. The people behind Interactivegestures.com are moving in this direction, but it’s unclear to me what the intellectual property around these gestures (if any) is. What is the way forward to ensure that gestural and touch-based UI can flourish and isn’t hobbled by intellectual property disputes before it’s even properly off the ground? We briefly “touched” on this issue during a discussion on the future of mobile user experience at Over the Air led by UIQ’s David Mery and Idean’s Mikko-Pekka Hanski, but this topic alone needs more discussion. At risk may be the very future of human computer interaction.
