Loomo is a mobile robot sidekick, both a self-balance transporter and a personal assistant. Loomo went on sale in April 2018 worldwide and has been integrated into the lives of 30,000+ satisfied customers.
PERSONAL SMART ROBOT
Robot, iOS, Android
I joined the Loomo team in May 2017, worked as the Product designer in a team of 2 PMs, 3 Product designers, and 20 engineers. It’s my luck to join the team from an early stage, so I had the chance to went through the whole design, develop and marketing process of Loomo.
As a designer for smart products, I lead the design of multiple features in the end-to-end product design, across the robot gesture, voice interaction, and companion mobile application (iOS, Android) interfaces.
On this page, I'd like to mainly share how I brought the gesture interaction from ideation to launch and solved the outdoor interaction problem.
A natural, quick way for outdoor interaction.
Gesture Quick View
Loomo has using scenarios both indoors and outdoors. Although it already got voice interaction, it's hard for voice commands to work in a noisy outdoor environment.
As users, they need an interaction method to quickly access outdoor high-frequency features like follow, take photos.
There were 2 reasons I thought gesture is an excellent fit:
1. Loomo is designed as a little buddy of users. Just like when people call their dog with gestures, users could call Loomo in the same way.
2. Gesture as a Long-distance interaction fits the high-frequency features like follow, take photos very well.
Goals of Gesture
Natural, quick, safe gestures for 3 feature: Follow, Stop Follow, Take Photo
High accuracy of gesture recognition
Easy to learn
Pre-design: Define Gestures
3 principles to choose gesture positions
Fit to life scenario, match the meaning of commands
Take future gesture plan into consideration
Evaluated by algorithm team
Final chosen gestures:
Take a Photo
CHALLENGE & SOLUTIONS
Challenge 1 - False Triggering
After gesture poses define, I put the gestures in robot idle states and started simplest testing.
What is False Triggering?
Like Alexa or Siri have false voice triggering when users did not call them, but they react. Gestures were also falsely triggered by mistakenly recognizing the random body language as the gesture commands and potentially starting to move suddenly!
False triggering is a very high-risk problem because it can cause security problems:
Smashes objects, potentially hurt people
Lost because followed wrong person…
KET DESIGN DECISION #1
I get inspiration from smart speakers, which always have a wake-up voice command like "Alexa", "Hey Siri ".
How's the weather?
After testing, the wake-up gesture is an effective solution for false triggering; it’s like a barrier to prevent command gestures from being recognized accidentally.
Wake-up gestures can also save CPU usage by minimizing counts, reducing delays and crashes.
Wake-up gesture choosing
A say-hi gesture.
* Accessibility considerations: for all one hand gesture, both right-hand and left-hand gestures are okay for recognization.
KET DESIGN DECISION #2
Visual Feedback for Holding Gestures
Another technical reason for the false triggering is the short recognition time required. Lead the users to hold the gesture for 2 seconds could avoiding false triggering.
Turn design ideas into visual interfaces, and I designed 2 visual feedback to guide users holding gestures:
Circle process bar
After testing, I chose the circle process bar feedback because testers know better about how long it will take. It's always good to set the right expectations for users.
Solutions for Feedback wasnot timely
CHALLENGE & SOLUTIONS
Challenge 2 -
Bridging the Gap between Design and Development
Why the developed result different from the designer's imagination? 😂
Delayed Feedback Causing Low Recognition Rate
After hand-off the design docs to the engineer, I found out that the developed version was not ideal enough due to the delayed feedback. Testers got so confused because when they raised the hand, the bar didn’t run; when they put the hand down, it didn’t stop.
Design in the Engineer's Shoes
After an in-depth talk with the engineer, he let me know that the technical definition he was making was recognizing each frame, not the seconds that I considered in the design. I found that the gap between us resulted in insufficient detail in the design hand-off.
Thinking in the engineer’s shoe, I redefined the start and end of recognition based on frame. And solved the delay feedback problem.
Take start recognition for example:
After continuously iterations and testings:
The general gesture recognition rate:
Take a Photo
Ready for Launch - Onboarding
For the goal of easy to learn, I designed gesture notification and tutorials as the new feature onboarding experience.
Mobile Notification & Tutorials
When the version including Gesture was shipped and installed by the user, the notification will be sent and lead the user to go through a feature introduction.
Interactive Robot Tutorial
Considered the robot’s moving and speaking capabilities, I designed an interactive tutorial to let Loomo teach users how to use gestures with its own tones.
The best way to learn is to practice! 🎓
Impact of Gesture
The Gesture feature of Loomo was launched in October 2018.
4.1 - 4.3
Loomo Application score:
6.2h - 7.35 h
Product weekly usage time:
5.8 - 8.2
Times of follow and take photo feature weekly used:
Gesture feature also received positive evaluations and sincere suggestions from the Loomo support website.
This is a precious opportunity to let me realize that in-depth communication between different team roles can profoundly change the design.
A poor experience may be caused by multiple reasons. It is equally essential for designers to troubleshooting and finding solutions.
Check More I Designed for Loomo
Over-the-Air (OTA) Update
Over-the-Air (OTA) Update
User testing &
User Testing & Product Testing