How can we integrate gestures into the design process? We must define what a gesture truly is. Willemien Visser and Mary Lou Maher, a cognitive psychologist and professor of design respectively say in “The role of gesture in designing.”1
By definition it is a non-verbal way of communication, but to communicate we need to think. As primitive as that may sound, how we think is very important to our gesture. In an article, by MIT Media Lab Instructor David Rose writes:
Inventing Gestures for Common Actions from IDEO on Vimeo.2
Just as we assimilate our interface appearance to demographic/roles and apply it to the relative field, the same rule applies for gesture. Knowing when to use modalities (such as voice, touch and body or face gestures) in general is important. Microsoft has done an extensive amount of research into Natural User Interfaces and Bill Gates states:
—Bill Gates, co-founder of the multinational technology company Microsoft3
As technology grows and adapts so too will the designer. This is not to say that the designer needs to become an expert in multiple disciplines like AI, HCI (human computer intelligence), psychology or advanced pragmatics, but be more aware of those fields involvement when designing for gesture. If we become more aware of gestures and their capabilities we will already have started to make progress.
For example, if a driver is in a moving car with an open mobile application/web app, then it could be made illegal to operate any mobile hand held device according to your location. The design may be based around gestures of driving (hands possibly placed 10 and 2) which could trigger a different mobile experience and interface allowing you to only operate the application through voice commands. How would you design the interface for that driver?
Companies like BMW, LG and eyeSight have already integrated gesture into their own interfaces.
When designers understand the role of gestures and/or modalities it strengthens the end result of the product. Here are some basic tips on how to keep on top of gesture while designing.
Onboarding the User
Simple touch gestures on interfaces are often clean but hidden. If user education does not exist then the user will most likely not use it. Best practices are to keep away from long tutorials and instructional wizards that have a long step by step processes. Instead keep a more gradual learning curve through discovery of the application.
Nick Babich explains in depth in his article "how to communicate hidden gestures"4 there he states a few common techniques like hinting, content teases, content command, tutorials and more.
Knowledge of existing gesture guidelines
Knowing the native gestures of the operation system that your application will be running on is important to keep in mind when you start to include touch gestures in your app. Apple thinks going too far away from the native gesture could lead to confusion. This is not to say that we should stick strictly to the standards of an IOS app like Apple but make our choices intuitive, apparent and consistent. For example: To “like” on Instagram you can double tap on an image, but natively if you double tap on an image in a gallery using Apple’s IOS, it will zoom.
Understanding your market
Knowing your audience highly affects how you want people to use gesture. When you rough draft and sketch out comps keep in mind possible elements to strengthen not hinder your user. Write down verbs to describe your actions and question the need or appropriate design to match the movement or scenario. Having a better understanding who your client is will affect the gesture and the type of gesture will impact the design.
Don’t be afraid to Experiment
As of right now there is a lack of publicly shared guidelines to gestural control. In order to advance we must experiment and try new techniques that cope with our design principles. With experimentation may come failure and with failure you may be able to create something successful. Most success is often found internally within your company team or lab at a very sustained and manageable rate until valid. It is not a great practice to inflict pain of failures onto your clients or customers.
Keep things Consistent
If you create an action, button, or your own symbolism that contains gesture, do not change the purpose of that button.
In an article written by Donald A. Norman and Jakob Nielsen of the Nielsen Norman group Gestural Interfaces: A Step Backwards In Usability5 they describe in depth some more principles which they feel are disappearing from the designer’s tool kit.
Designers in the end have the responsibility to make gestures really work. They pave the pathways and instruct our users visually. Through discovery of an application we need to slowly educate and not force knowledge on our users. We should send consistent visual cues and pair them with consistent gestures. It is necessary to keep in touch with gesture advancements as our technology progresses so we can push the limits of our designs.
- “The role of gesture in designing”
- Inventing Gestures for Common Actions from IDEO on Vimeo.
- "how to communicate hidden gestures"
- Gestural Interfaces: A Step Backwards In Usability