The story about me and LOOMO

LOOMO is a Mobile Robot Sidekick, both a self-balance vehicle and a personal assistant. LOOMO went on sale in the US market in April 2018. Since the launch, Loomo has been integrated into the lives of 30,000+ satisfied customers.

Enjoy sunlight with LOOMO
Service at the party
Interact with kids

 I joined the LOOMO team in May 2017, worked as the UX designer in a department of 30+ People of Segway Robotics, and went through the whole design and marketing process. I worked on both the Robot Product and the robot Controlling Application.

When designed each feature, I pay attention to Multi-devices Design (Robot system, iOS, Android), and lead a smooth experience between the application and the robot itself. 

While I worked on many features in the end-to-end  product design, I'd like to highlight the following features which I drove the design: 

Robot Status System
Gesture Interaction
Voice Auto-shot
Interactive Tutorial
Over-the-Air (OTA) Update
User testing &
Product testing

On this page, I'd like to mainly introduce the complete design process of the Gesture Interaction feature. Other feature design details could be found above.


Gesture Interaction


Quick view

Gesture interaction is a more natural, quick way to access outdoor features.


What's the problem?

LOOMO is a product that can be used both indoors and outdoors, although it already got Voice Interaction, it's hard for LOOMO to receive voice commands in a noisy outdoor environment

Why Gesture?

There were 2 reasons I thought gesture is a good fit to solve the problem:

1. Gesture is a natural interaction way, especially for LOOMO. Call LOOMO as a little buddy of users, just like when people are calling a dog with gestures.

2. Gesture as a Long-distance interaction fits the popular outdoor features scenarios: Follow Me, Take a Photo very well. 

Goals of Gesture

Natural, quick, safe gestures for 3 feature: 

  • Follow, Stop Follow, Take Photo

  • High accuracy of gesture recognition

  • Easy to learn


3 principles to choose gesture positions 


Fit to life scenario, match the meaning of commands


Take future gesture plan into consideraton

Final chosen gestures:

Start Follow

Stop Follow

Evaluated by algorithm team


Take a Photo


I started the simplest testing immediately after gesture poses define, put the gesture in Robot without feedbacks in this step.

Challenge 1 - False Triggering

What is False Triggering?

My boss once demonstrated LOOMO to a client. While the two were chatting and doing some random positions at the same time, LOOMO mistakenly recognized the action as the Follow gesture and start moving suddenly!

Why so serious?

algorithmic demo

I considered False Triggering as a very high-risk problem because it's not only:

  1. A Poor experience

But also cause          security problems:
   2. Smashes objects, potentially hurts people
   3. Lost because followed wrong person...

Solutions of - False Triggering

The wake-up gesture is like a barrier to avoid bad things from happen 

How's the weather?

1. Add a Waken Gesture

I get inspiration from smart speakers, which always has a wake-up voice command like "Alexa", "Hey Siri ".

I thought the wake-up gesture also could be a very effective way to avoid False Triggering because:


Take Photo


Take Photo


2. Require a length of time to recognize gestures

Waken gesture also saves CPU occupancy by letting it counting less, reduces lag, and crashes.

Waken gesture choosing

Accessibility considerations:

 for all one hand gesture, both right-hand and left-hand gestures are okay for recognization.

I researched the technical reasons for the False Triggering. The required recognition time was too short also cause the problem, let some random actions be judged as command gestures.

In this step, I designed 2 feedback for how to let user sknow the required time:


Bar in circle

After testing, I chose the bar in circle feedback, because clearly in this version, testers know better about how long it will take. It's always good to set the right expectations for users. 

Challenge 2 - Feedback wasnot timely - low recognization rate

Why the developed result different from your imagination as a Designer? 😂

I delivered the design docs to Engineer, but I found out the developed version was not ideal enough because the Feedback was not timely! Testers get so confused because when the delay: 

When they raise hand, the bar did not go, when they but down hand, it did not stop.  

low recognization rate

Solutions for Feedback wasnot timely

Think about design in the engineer's shoes, find the gap!

After a DEEP talk with the engineer. He let me know that the technical judgment he was making was based on the recognition of each FRAME, not the SECONDS that I considered in the design. I found this gap between us caused I was not able to design in-depth determination details.

Designer: Seconds?

Engineer: Frames?

Take start recognition for example:

Design for start and end of recognition gestures in an engineer's view. 


After continuously iterations and testings:  

The general recognition rate was :

And I'm happy to see the False Triggering Never happen again


Final Design

Start Follow

Stop Follow


Start Follow

Stop Follow

Take a Photo

Design for ship - Tutorials

Gesture as a new feature after product shipped, it was incomplete without notification and tutorials

APP Notification and Tutorial

When the version including Gesture was shipped and installed by the user, the notification will be sent and lead the user to go through a feature introduction.

Interactive Robot Tutorial

Considered the robot that can speak and move, I decided to design an interactive tutorial to let LOOMO teach the gesture by themselves. 🎓

The best way to learn is to practice!

Impact of Gesture

The Gesture feature of LOOMO was released in October 2018. 

LOOMO Application score:

Product weekly usage time:

Times of Follow and Take Photo feature weekly used:


6.2h - 7.35h

5.8 -8.2

It was a good reward for me to receive positive evaluations and sincere suggestions from the LOOMO support website.

  1.  This is a precious opportunity to let me realize that in-depth communication between different team roles can profoundly change the design.

  2. A poor experience may be caused by multiple reasons. It is equally essential for designers to troubleshooting and finding solutions.

Design by Lai Wei © 2021