Menu
the REFERENCE

Gesture Through UI Design

Pulling, pinching, swiping, scrolling, tapping and typing - these tiny movements are just the tip of the iceberg when talking about gestures. They allow us to connect with our interfaces and are slowly becoming more common and more complex. It’s easy to overthink and misuse gestures intended for interactions. It’s also crucial to remember to keep the user interface parallel to the user experience during the design process.

How can we integrate gestures into the design process? We must define what a gesture truly is. Willemien Visser and Mary Lou Maher, a cognitive psychologist and professor of design respectively say in “The role of gesture in designing.”1

Although gesture is most commonly assumed to play a role in communication, it has been shown that gesture also plays an important role in thinking (McNeill, 1992).

By definition it is a non-verbal way of communication, but to communicate we need to think. As primitive as that may sound, how we think is very important to our gesture. In an article, by MIT Media Lab Instructor David Rose writes:

Gestures are generation-specific. When asked to signal turning up the volume, a few people twisted an invisible knob, but most of the under 30s lifted a palm or made a pinching gesture with their fingers.

Inventing Gestures for Common Actions from IDEO on Vimeo.2

Just as we assimilate our interface appearance to demographic/roles and apply it to the relative field, the same rule applies for gesture. Knowing when to use modalities (such as voice, touch and body or face gestures) in general is important. Microsoft has done an extensive amount of research into Natural User Interfaces and Bill Gates states:

Until now, we have always had to adapt to the limits of technology and conform the way we work with computers to a set of arbitrary conventions and procedures. With NUI, computing devices will adapt to our needs and preferences for the first time and humans will begin to use technology in whatever way is most comfortable and natural for us.

—Bill Gates, co-founder of the multinational technology company Microsoft3

As technology grows and adapts so too will the designer. This is not to say that the designer needs to become an expert in multiple disciplines like AI, HCI (human computer intelligence), psychology or advanced pragmatics, but be more aware of those fields involvement when designing for gesture. If we become more aware of gestures and their capabilities we will already have started to make progress.

For example, if a driver is in a moving car with an open mobile application/web app, then it could be made illegal to operate any mobile hand held device according to your location. The design may be based around gestures of driving (hands possibly placed 10 and 2) which could trigger a different mobile experience and interface allowing you to only operate the application through voice commands. How would you design the interface for that driver?

Companies like BMW, LG and eyeSight have already integrated gesture into their own interfaces.



When designers understand the role of gestures and/or modalities it strengthens the end result of the product. Here are some basic tips on how to keep on top of gesture while designing.

Onboarding the User

Simple touch gestures on interfaces are often clean but hidden. If user education does not exist then the user will most likely not use it. Best practices are to keep away from long tutorials and instructional wizards that have a long step by step processes. Instead keep a more gradual learning curve through discovery of the application.

Focus on a single interaction rather than trying to explain every possible action in user interface

Nick Babich explains in depth in his article "how to communicate hidden gestures"4 there he states a few common techniques like hinting, content teases, content command, tutorials and more.

Knowledge of existing gesture guidelines

Knowing the native gestures of the operation system that your application will be running on is important to keep in mind when you start to include touch gestures in your app. Apple thinks going too far away from the native gesture could lead to confusion. This is not to say that we should stick strictly to the standards of an IOS app like Apple but make our choices intuitive, apparent and consistent. For example: To “like” on Instagram you can double tap on an image, but natively if you double tap on an image in a gallery using Apple’s IOS, it will zoom.

Apple’s

Standard Gestures. People generally expect the following standard gestures to work the same across the system and in every app.
https://developer.apple.com/ios/human-interface-guidelines/user-interaction/gestures/

Android

https://developer.android.com/training/gestures/index.html

Google

https://material.io/guidelines/patterns/gestures.html#gestures-touch-mechanics

Microsoft

https://msdn.microsoft.com/en-us/library/windows/desktop/dd940543(v=vs.85).aspx

Understanding your market

Knowing your audience highly affects how you want people to use gesture. When you rough draft and sketch out comps keep in mind possible elements to strengthen not hinder your user. Write down verbs to describe your actions and question the need or appropriate design to match the movement or scenario. Having a better understanding who your client is will affect the gesture and the type of gesture will impact the design.

Don’t be afraid to Experiment

As of right now there is a lack of publicly shared guidelines to gestural control. In order to advance we must experiment and try new techniques that cope with our design principles. With experimentation may come failure and with failure you may be able to create something successful. Most success is often found internally within your company team or lab at a very sustained and manageable rate until valid. It is not a great practice to inflict pain of failures onto your clients or customers.

Keep things Consistent

If you create an action, button, or your own symbolism that contains gesture, do not change the purpose of that button.

In an article written by Donald A. Norman and Jakob Nielsen of the Nielsen Norman group Gestural Interfaces: A Step Backwards In Usability5 they describe in depth some more principles which they feel are disappearing from the designer’s tool kit.

Conclusion

Designers in the end have the responsibility to make gestures really work. They pave the pathways and instruct our users visually. Through discovery of an application we need to slowly educate and not force knowledge on our users. We should send consistent visual cues and pair them with consistent gestures. It is necessary to keep in touch with gesture advancements as our technology progresses so we can push the limits of our designs.

INTERESTED IN UX DESIGN? DOWNLOAD OUR WHITEPAPER! 

  1. “The role of gesture in designing”
  2. Inventing Gestures for Common Actions from IDEO on Vimeo.
    https://www.ideo.com/blog/why-gesture-is-the-next-big-thing-in-design
  3. https://www.interaction-design.org/literature/article/natural-user-interfaces-what-are-they-and-how-do-you-design-user-interfaces-that-feel-natural
  4. "how to communicate hidden gestures"
  5. Gestural Interfaces: A Step Backwards In Usability

Don't miss out

top
The Reference has its office in the heart of Manhattan.
“I want to wake up in that city that never sleeps, and find I'm king of the hill, top of the list, head of the heap” – Frank Sinatra