Apple / Interface / Interface innovations / Mobile / Nokia

Lost futures: Unconscious gestures?

moleitau23 Comments

Lamenting lost futures is not that productive, but it doesn’t stop me enjoying it. Whether it’s the pleasure of reading Ellis’s “Ministry of Space” and thinking “what if?” or looking through popculture futures past as in this Guardian article – it’s generally a sentimental, but thought-provoking activity.

Recently, though, I’ve been thinking about a temporarily lost future that’s closer to home in the realm of mobile UI design. That’s the future that’s been perhaps temporarily lost in the wake of the iPhone’s arrival.

A couple of caveats.

Up until June this year. I worked at Nokia in team that created prototype UIs for the Nseries devices, so this could be interpreted as sour-grapes, I suppose.. but I own an iPodTouch, that uses the same UI/OS more-or-less, and love it.

I spoke at SkillSwap Bristol in September (thanks to Laura for the invite) and up until the day I was travelling to Bristol, I didn’t know what I was going to say, but I’d been banging on at people in the pub (esp. Mr. Coates) about the iPhone’s possible impact on interface culture, so I thought I’d put together some of those half-formed thoughts for the evening’s debate.

The slides are on Slideshare (no notes, yet) but the basic riff was that the iPhone is a beautiful, seductive but jealous mistress that craves your attention, and enslaves you to its jaw-dropping gorgeousness at the expense of the world around you.

skillswap250907

This, of course, is not entirely true – but it makes for a good starting point for an argument! Of course, nearly all our mobile electronic gewgaws serve in some small way or other to take us away from the here and now.

But the flowing experience just beyond Johnny Ive’s proscenium chrome does have a hold more powerful than perhaps we’ve seen before. Not only over users, but over those deciding product roadmaps. We’re going to see a lot of attempts to vault the bar that Apple have undoubtedly raised.

Which, personally, I think is kind-of-a-shame.

First – a (slightly-bitter) side-note on the Touch UI peanut gallery.

In recent months we’ve seen Nokia and Sony Ericsson show demos of their touch UIs. To which the response on many tech blogs has been “It’s a copy of the iPhone”. In fact, even a Nokia executive responded that they had ‘copied with pride’.

That last remark made me spit with anger – and I almost posted something very intemperate as a result. The work that all the teams within Nokia had put into developing touch UI got discounted, just like that, with a half-thought-through response in a press conference. I wish that huge software engineering outfits like S60 could move fast enough to ‘copy with pride’.

Sheesh.

Fact-of-the-matter is if you have roughly the same component pipeline, and you’re designing an interface used on-the-go by (human) fingers, you’re going to end up with a lot of the same UI principles.

But Apple executed first, and beautifully, and they win. They own it, culturally.

Thus ends the (slightly-bitter) side-note – back to the lost future.

Back in 2005, Chris and myself gave a talk at O’Reilly Etech based on the work we were doing on RFID and tangible, embodied interactions, with Janne Jalkanen and heavily influenced by the thinking of Paul Dourish in his book “Where the action is”, where he advances his argument for ’embodied interaction’:

“By embodiment, I don’t mean simply physical reality, but rather, the way that physical and social phenomena unfold in real time and real space as a part of the world in which we are situated, right alongside and around us.”

I was strongly convinced that this was a direction that could take us down a new path from recreating desktop computer UIs on smaller and smaller surfaces, and create an alternative future for mobile interaction design that would be more about ‘being in the world’ than being in the screen.

That seems very far away from here – and although development in sensors and other enablers continues, and efforts such as the interactive gestures wiki are inspiring – it’s likely that we’re locked into pursuing very conscious, very gorgeous, deliberate touch interfaces – touch-as-manipulate-objects-on-screen rather than touch-as-manipulate-objects-in-the-world for now.

But, to close, back to Nokia’s S60 touch plans.

Tom spotted it first. In their (fairly-cheesy) video demo, there’s a flash of something wonderful.

Away from the standard finger and stylus touch stuff there’s a moment where a girl is talking to a guy – and doesn’t break eye contact, doesn’t lose the thread of conversation; just flips her phone over to silence and reject a call. Without a thought.

Being in the world: s60 edition from blackbeltjones on Vimeo.

As Dourish would have it:

“interacting in the world, participating in it and acting through it, in the absorbed and unreflective manner of normal experience.”

I hope there’s a future in that.

23 thoughts on “Lost futures: Unconscious gestures?”

  1. You are dead right.

    My new K850 is a lot worse than my broken old K750 because it’s got a little element of touch to it. So it makes me look at it all the time. Which bugs me. I used to be able to do lots on the K750 without looking. That’s why I prefered my old cassette walkman to my ipod, I could feel my way through every control. Didn’t have to look at it.

    I hadn’t worked out why I didn’t like this, but you’ve crystalised it for me. Thanks

    Reply

  2. The next important step is to convince TV manufactures to create devices that connect to the network. With the prices of panels dropping so abruptly, that will be our future world, in a sea of IP based displays.

    I didn’t know you used to work on experimental user interfaces, that is pretty bad ass.

    Reply

  3. Not sure if you were aware of this but Paul Golding first blogged about this concept ages ago and I was talking to Nokia Research Centre about it around the same time(roll-over-and-hush). Pleased to see this coming next year.

    Reply

  4. “it’s likely that we’re locked into pursuing … touch-as-manipulate-objects-on-screen rather than touch-as-manipulate-objects-in-the-world”

    Well, you could argue that the iPhone is the first consumer experience of touch-as-manipulate-DATA-objects-in-the-world”. Music is now more or less a fully digital experience; a “song” is for all intents and purposes now authentically “in-the-world” only as a digital file. When you flip through songs on the iPhone, you are not flipping through “objects on a screen”, you’re flipping through objects in the world that happen to be digital. Photos are nearly there, too–they’re digital things, not digital representations of things.

    “Social phenomena”, the other part of Dourish’s definition of embodiement, are obviously clearly moving this way, too. What’s the appropriate *physical* interaction for Twittering, for instance? Could it be possible to consider a twitter post a “social object in the world” that you could touch like you touch an mp3 file?

    Also, I always thought that Timo’s Address book desk was a great sketch of how interactions like the flip-phone-to-ignore-call example could work:
    http://www.elasticspace.com/2005/12/address-book-desk

    Reply

  5. Great insight here about cultural ownership — something Apple and its design team have done well many times over.

    What is particularly interesting here is the response that other companies who maybe should have (or would have been expected to) done this “first” make these ham-fisted efforts that are essentially driven by conservative and rather uninspiring business practices of dipping into a market of users — let’s get our piece of this market of expectation, they may think — anything will work so long as we borrow the “touch” interaction idiom, now owned by Apple. It’s not the lack of technology nor lack of inspired designers — it’s the lack of will and aspiration and generally scaredy-cat business principles that keep innovation well-muffled.

    Apple has the will and aspiration and moxy — in Ives and Jobs — to do clever and world-changing design that changes the way we live within the world. The other operations..well..someone needs to lose.

    Reply

  7. Great post! A small note: The anecdotal evidence so far is that one thing iPhone users don’t like is that they always have to look at the screen to do anything, unlike older phones where feeling for the right button worked just fine…

    Reply

  11. “it’s likely that we’re locked into pursuing … touch-as-manipulate-objects-on-screen rather than touch-as-manipulate-objects-in-the-world”

    Currently looking for ways of differentiating Touch in these two contexts. The former is getting a lot more attention right now. Thanks for putting it into one sentence.

    Reply

  12. […] Blackbeltjones/Work: » Lost futures: Unconscious gestures? Jones on the iPhone: “…although development in sensors and other enablers continues, and efforts such as the interactive gestures wiki are inspiring – it’s likely that we’re locked into pursuing very conscious, very gorgeous, deliberate touch interf (tags: iphone tangible physical embodied interactiondesign gestures touch mobile) […]

    Reply

  13. […] Matt Jones discusses Nokia’s S60 touch plans and raises the future concept of unconscious gestures with regards to actions on the phone. Referring to a video (viewable from the foot of his blog post), he summarises the observed action:”Away from the standard finger and stylus touch stuff there’s a moment where a girl is talking to a guy – and doesn’t break eye contact, doesn’t lose the thread of conversation; just flips her phone over to silence and reject a call. Without a thought.”  The concept is nothing new, but it reminds us how elegant some phone interactions could be. Matt Jones on Lost futures: Unconscious gestures? […]

    Reply

  18. Yes, iPhone is nice, but forget not that we all ‘stand on the shoulders of giants’, so very few ideas are truly original, more evolution than revolution. Leonardo ‘invented’ the helicopter, but couldn’t make one; same with the iPhone; you could image its UI, but could you have made it successfully even five years ago?

    Reply

Leave a comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.