loomo.png
Loomo3.png

World’s most popular personal robot of 2018 & 2019.

Home.GIF
iphone_edited.png
资源 54.png
Google-Play-and-Apple-App-Store-Logos-Tw
Google-Play-and-Apple-App-Store-Logos-Tw
Loomo3.png

Meet

 

Loomo is a mobile robot sidekick, both a self-balance transporter and a personal assistant.  Loomo went on sale in April 2018 worldwide and has been integrated into the lives of 30,000+ satisfied customers.

image%2029_edited.jpg

SELF-BALANCE TRANSPORTER

image%2030_edited.jpg

PERSONAL SMART ROBOT

  • YouTube

My Role:

Product Designer

Duration:

16 Months

Cross-Platform:

Robot, iOS, Android

 I joined the Loomo team in May 2017, worked as the Product designer in a team of 2 PMs, 3 Product designers, and 20 engineers. It’s my luck to join the team from an early stage, so I had the chance to went through the whole design, develop and marketing process of Loomo.
 

As a designer for smart products, I lead the design of multiple features in the end-to-end product design, across the robot gesture, voice interaction, and companion mobile application (iOS, Android) interfaces.

On this page, I'd like to mainly share how I brought the gesture interaction from ideation to launch and solved the outdoor interaction problem.
 

Gesture Interaction

A natural, quick way for outdoor interaction.
 

 

Gesture Quick View

start.gif
未标题-2.png
02.gif

The Problem

Loomo has using scenarios both indoors and outdoors. Although it already got voice interaction, it's hard for voice commands to work in a noisy outdoor environment.


As users, they need an interaction method to quickly access outdoor high-frequency features like follow, take photos.

 

04.gif

Why Gesture

dog2.png

There were 2 reasons I thought gesture is an excellent fit:

1. Loomo is designed as a little buddy of users. Just like when people call their dog with gestures, users could call Loomo in the same way.

​2. Gesture as a Long-distance interaction fits the high-frequency features like follow, take photos very well.

Goals of Gesture

  1. Natural, quick, safe gestures for 3 feature: Follow, Stop Follow, Take Photo

  2. High accuracy of gesture recognition

  3. Easy to learn

Pre-design: Define Gestures

3 principles to choose gesture positions

image 4.png
未标题-2.png
image 8.png

1.

Fit to life scenario, match the meaning of commands

2.

Take future gesture plan into consideration

3.

Evaluated by algorithm team

Final chosen gestures:

image 26.png
image 27.png
资源 55.png

Start Follow

资源 55.png

Stop Follow

image 25.png
资源 55.png

Take a Photo

IMG_4775_edited.jpg

CHALLENGE & SOLUTIONS
 

Challenge 1 - False Triggering

After gesture poses define, I put the gestures in robot idle states and started simplest testing.

 

What is False Triggering?

Like Alexa or Siri have false voice triggering when users did not call them, but they react. Gestures were also falsely triggered by mistakenly recognizing the random body language as the gesture commands and potentially starting to move suddenly!

Gesture_StartFollowing.png
wuchu.gif

False triggering is a very high-risk problem because it can cause security problems:

  1. Poor experience

  2. Smashes objects, potentially hurt people

  3. Lost because followed wrong person…

资源 59.png

KET DESIGN DECISION #1

Wake-up Gesture

I get inspiration from smart speakers, which always have a wake-up voice command like "Alexa", "Hey Siri ".

0b8b5bc099644a1d9299fd34fc26cc42.png
Alexa
How's the weather?

After testing, the wake-up gesture is an effective solution for false triggering; it’s like a barrier to prevent command gestures from being recognized accidentally.

Wake-up gestures can also save CPU usage by minimizing counts, reducing delays and crashes.

Take Photo

Follow

Follow

Group 20.png
Group 21.png

Take Photo

Waken

pexels-gabby-k-5063095.jpg

Wake-up gesture choosing

A say-hi gesture.

* Accessibility considerations: for all one hand gesture, both right-hand and left-hand gestures are okay for recognization.

KET DESIGN DECISION #2

Visual Feedback for Holding Gestures

Another technical reason for the false triggering is the short recognition time required. Lead the users to hold the gesture for 2 seconds could avoiding false triggering.​

Turn design ideas into visual interfaces, and I designed 2 visual feedback to guide users holding gestures:

v1.gif

1.

Loading animation

timer.gif

2.

Circle process bar

After testing, I chose the circle process bar feedback because testers know better about how long it will take. It's always good to set the right expectations for users.

Solutions for Feedback wasnot timely

CHALLENGE & SOLUTIONS
 

Challenge 2 -

Bridging the Gap between Design and Development

Why the developed result different from the designer's imagination? 😂

Delayed Feedback Causing Low Recognition Rate

After hand-off the design docs to the engineer, I found out that the developed version was not ideal enough due to the delayed feedback. Testers got so confused because when they raised the hand, the bar didn’t run; when they put the hand down, it didn’t stop.

late.gif
confused-1.png

SOLUTION

Design in the Engineer's Shoes

After an in-depth talk with the engineer, he let me know that the technical definition he was making was recognizing each frame, not the seconds that I considered in the design. I found that the gap between us resulted in insufficient detail in the design hand-off.

Frame.png
confused-1.png
Frame-1.png
confused-1.png

Designer: Seconds?

Engineer: Frames?

Thinking in the engineer’s shoe, I redefined the start and end of recognition based on frame. And solved the delay feedback problem.

Take start recognition for example:

未标题-2.png
Group 7.png
Group 25.png
Rectangle 46.png

Evaluation

After continuously iterations and testings:

92.6%

The general gesture recognition rate:

Never

False Triggering

happen again

 

Final Design

 
start.gif
未标题-2.png
02.gif
资源 66.png

Start Follow

stop.gif
未标题-2.png
03.gif
资源 65.png

Stop Follow

shot.gif
未标题-2.png
01.gif
资源 67.png

Auto-shot

Take  a Photo

Start Follow

Stop Follow

 

Ready for Launch - Onboarding

For the goal of easy to learn, I designed gesture notification and tutorials as the new feature onboarding experience.

Mobile Notification & Tutorials

When the version including Gesture was shipped and installed by the user, the notification will be sent and lead the user to go through a feature introduction.

Space Gray.png
资源 56.png

Interactive Robot Tutorial

Considered the robot’s moving and speaking capabilities, I designed an interactive tutorial to let Loomo teach users how to use gestures with its own tones.

The best way to learn is to practice! 🎓

apple2_edited_edited.jpg

Impact of Gesture

The Gesture feature of Loomo was launched in October 2018. 

4.1 - 4.3

Loomo Application score:

6.2h - 7.35 h

Product weekly usage time:

5.8 -  8.2

Times of follow and take photo feature weekly used:

 

Gesture feature also received positive evaluations and sincere suggestions from the Loomo support website.

资源 70.png
资源 71.png

Takeaways

  1.  This is a precious opportunity to let me realize that in-depth communication between different team roles can profoundly change the design.

  2. A poor experience may be caused by multiple reasons. It is equally essential for designers to troubleshooting and finding solutions.

Check More I Designed for Loomo

 
Voice Auto-shot
资源 4.png

Voice Auto-shot

Over-the-Air (OTA) Update
资源 2.png

Over-the-Air (OTA) Update

User testing &
Product testing
资源 3.png

User Testing & Product Testing