Commentary
How gesture, voice, haptics enhance touchscreen user experiences
All of the technology necessary to enhance touchscreen user experiences with new features is available today. Numerous types of businesses are waiting for innovators to deliver self-service systems that give their customers the flexibility of greater options to interact.
October 4, 2021 by Gene Halsey — Director, Sales and Marketing, Touch Revolution
As computers and mobile devices evolved, the touchscreen emerged as the most natural way for people to interact with technology. Think smartphone UIs that transitioned from keyboards to touchpads and tracker balls, then finally to widespread touchscreen adoption. It's not hard to understand why. It's simply easier to tap, swipe, scroll, pinch or spread than to try to interface in other ways.
Touchscreens have also become a staple in business interactions. They're the interface at ATMs, grocery self-checkouts, fast food ordering kiosks and healthcare facility check-ins. Now that they're more ubiquitous, innovators are beginning to explore how to make good touchscreen user experiences even better.
Imagine the experiences you could engineer by combining touchscreen technology with the following:
Gesture recognition
AI-powered vision systems that learn and interpret motion can augment touchscreens systems with gesture recognition. It could eliminate the need for users to actually touch it for every input, such as allowing them to point to a selection or swipe to move to the next screen.
Voice recognition
Natural language processing, the behavioral tech behind Siri, Alexa and voice-based systems, has won over consumers with its convenience and time savings. Voice in combination with a touchscreen can streamline interactions, for example, allowing a user to confirm information with a spoken "yes," "no" or "next" at self-service check-in or informational kiosks.
Haptics
Haptics — technology that provides a tactile response, such as a vibration, pulse or even sensation of running your finger over a zipper to open a screen — can take touchscreen experiences to a new level. It can bring an alert to a user's attention, warn of incorrect data input, and add a 3D effect to choices on the screen.
Biometric identification
People are accustomed to unlocking their smartphones with a fingerprint or other biometric authentication technique. This functionality can save the time a user spends at a kiosk and decrease the instances of fraud.
Some evangelists of advanced technology believe that AI-based solutions will replace current systems; however, it's hard to imagine voice, gesture or haptics alone replacing the ease and familiarity of a touchscreen's UI. Besides, humans communicate in multiple ways. Giving them that same flexibility — through touch, gesture, voice and sensations via haptics — is likely the formula for optimal user experiences.
Challenges of democratized self-service
Enhancing touchscreen user experiences with advanced technologies also offer innovators a path to solving some challenges that prevent excellent user experiences for everyone. Consider:
- Visually impaired users. Bank ATMs and some other kiosks give these users the option to use headphones and a microphone. However, one downside is security — this workaround may force the users to say sensitive information aloud. Haptics could solve that problem by directing the user to a field where they could enter a PIN or other data with taps or even by a tactile interface where they could feel the shapes of numbers or even read Braille.
- Users with physical handicaps or prosthetic limbs. People with tremors or with limited use of their hands may not be able to use a touchscreen easily. Additionally, capacitive touchscreens rely on the conductive properties of the human body to create an electrostatic field that sends signals to a controller. A prosthetic limb may not work with capacitive touch. Supplementing touch with gesture recognition could enable more users to interact with kiosks, self-checkouts or interactive digital signage.
- Users concerned with security. People are protective of their personally identifiable data and healthcare and payment information — and with good reason. There's no telling who is standing behind you in line at a kiosk and what they're watching. A kiosk integrated with biometric ID and voice recognition technology can enable a user to check in with a healthcare provider with a fingerprint or facial recognition, then answer yes, verbally or with the touchscreen, to confirm information or submit a copay.
- Corporate teams. Touchscreen solutions augmented with gesture, voice and haptics can help create user-friendly collaborative conference systems with voice-activated cameras and digital touchscreen whiteboards. Integrating these technologies can give team members attending virtually the full boardroom experience. Enhanced touchscreen solutions can also make presiding over meetings with in-person and remote members easier and allow more focus on content rather than technology.
Design solutions
Undoubtedly, there are hurdles to cross to optimize touchscreen user experiences. AI needs to be impeccably trained so that it works correctly. Variations in gestures must be accounted for, voice recognition must work in a busy, noisy store or office, and biometric ID must have little to no instances of false positive or false negatives.
The biggest roadblock, though, is probably public perception. You can develop an elegant system that uses AI vision systems to identify loyal shoppers, their demographics and the items they show interest in during a shopping trip. You can use that data to provide these shoppers with information via voice or touchscreen or discounts for products they're most likely to buy. However, those systems may skirt the boundary between personalizing experiences and basically making them creepy. It may be better to limit your technology for merchants to use with people who opt in to receive such offers. You may also want to advise merchants to leverage other data your solution collects to provide insights into traffic flow or interest in displays and use that intel to optimize shopping experiences for everyone, not specific consumers.
Time to innovate
All of the technology necessary to enhance touchscreen user experiences with new features is available today. And numerous types of businesses are waiting for innovators to deliver self-service systems that give their customers the flexibility of greater options to interact — and that will also provide them with a more positive impression of their brands.
Start thinking in terms of wholistic user experiences and how technology can provide the richest interactions. Also, consider how the enhanced touchscreen solution will work within the business's system, integrating with point of sale, analytics, inventory management or CRM for seamless operation and the greatest value. Then take your design from theoretical to prototype to prove that value.
Your creativity, skill and expertise can change self-service user experiences. What could you create when you bring touchscreens and advanced technologies together?
