Are Robo-Butlers Really Handy?

As all of us are spending more time confined in our domiciles, robo-maids could not be a more timely concept. Last month, Sebastian Seung, President of Samsung Research, unveiled at (virtual) CES 2021, Handy – the autonomous butler. Seung described his innovation as “an extension of you in the kitchen, living room, and anywhere else you may need an extra hand in your home.” The highly-produced video was complete with examples of dish-loading, wine-pouring, and toy-picker-upping. What wasn’t shown were the numerous bloopers and behind-the-screen outtakes of the exhaustive programing hours to set up the demos.

In pulling back the curtain on deep tech in the home, I Zoomed Peter Burton, founder of General Manual Robotics (GMR). Dr. Burton, a math professor at the University of Texas, is currently in the early days of building his own robotic housekeeping platform. He shared with me his opinion of Samsung’s display, “I am extremely skeptical about the quality of the artificial intelligence behind the housekeeping robot in Samsung’s CES video. They show it performing an extremely wide variety of tasks, from pouring wine to loading the dishwasher to doing laundry. Developing an AI robust enough to go into an unknown consumer home and learn from them how they want to pick up toys is already a hard problem.” The professor is fully aware of the challenges of mechanically cleaning a room. As an entrepreneur and roboticist in his own right, he further commented, “I think it is likely that the robot in the Samsung video is using specifically coded computer vision and manipulation instructions to perform its tasks and would not be able to learn to do these tasks in an unseen home without further human intervention. “

Unlike Samsung, which aims to build a Swiss Army Knife type of robot functionality, Burton is singularly focused on building a “mechanical solution to pick up toys,” affectionately called TouchBot. He outlined to me how his approach differs from Samsung and others on the market: “We’re confident that our provisional patent application offers the most efficient way for a small robot enabled with perception, mobility and manipulation to learn from a consumer home environment well to satisfy them and alternative approaches have a large risk of drowning out any feasible amount of training data in the large size of the problem space.” The professor further delineated his methodology, “The essence of neural network architecture is to build a mathematical formalism that takes advantage of structure in the underlying data imposed by its human purpose. In our case, this means innovating in both convolutional and recurrent neural network design to incorporate the 3D geometry of the home environment and the distributing of user commands and tasks in time.”

Using a cognitive deep learning network, Dr. Burton aims to make the customer experience as seamless as working an Alexa device. “Our approach to the user experience is that we never want them to interact with the robot like a piece of technology that they have to figure out – the process of controlling it will consist of easy voice commands and simple gestures,” boasts the TouchBot’s creator. The novel conversational nature of TouchBot’s command structure is the key to its deep learning system. The founder walked me through the current examples:

• User command: ‘Put the red truck in the box’, gestures toward the box . 
  • Robot response (a): Assume there is only one red truck in the room. It will immediately comply by driving to the red truck, picking it up and putting it in the box. The robot will understand to look for toy trucks rather than scanning for images of 18-wheelers. 
  • Robot response (b): If there’s multiple red trucks, it will give a voice response asking whether to put all of them in the box.
• User command: ‘Where’s Jane’s favorite teddy bear?’
  • Robot response (a): Assume it’s heard a command ‘this is Jane’s favorite teddy bear’ in the past while the user is holding or gesturing to the teddy bear. The robot will locate the most recent place it saw the bear in a memory of the home and give a voice response indicating the location and offer to retrieve it.
  • Robot response (b): If it’s never heard an explicit command about Jane’s favorite teddy bear, it will know how to recognize images of teddy bears in general and suggest various ones in the home to the user.
• User command: ‘Put all toys in this room on the shelf every time Jane leaves them on the floor’
  • Robot response: It will be able to consistently apply this command now and in the future, clearing the toys from the floor whenever they are left and no one is directly using them.

Dr. Burton plans to expand the library of tasks to eventually include: “sorting clothes and putting them in the laundry; picking up small packages at the door; retrieving medication bottles and snacks; and playing simple tossing games with children or dogs.” His mission is to solve real human problems at a reasonable price. “We think of ‘good’ vs ‘bad’ uses of robotics in terms of the physical complexity of the hardware necessary to complete the task. For us, a good use of home robotics is one that involves simpler and cheaper hardware because as scientists we believe in solving easier problems first and there are numerous applications of simple and cheap hardware going unsolved in the consumer home due to a lack of quality AI,” exclaims the mathematician. The scientist points to Roomba as a model for “good” robot use cases. “Roomba solves a clear physical problem: vacuuming. Other mobile home robots like Jibo and Anki Vector have essentially been smart speakers on wheels and the only added customer benefit over static smart speakers seems to be a sense of novelty. We believe our robot can follow Roomba’s path rather than the numerous failures because it solves a clear physical problem that is not possible without a manipulator,” suggests Burton.

The technical challenges are not the only obstacles to automating housekeeping, cost is still the biggest barrier for most American families. While Touchbot is in the early days of prototyping, Burton hopes to begin marketing his device next year at a price point under a thousand dollars. According to Care.com that is the equivalent to 61 hours of housekeeping services to pick up a few toys. In 2019, only 10% of US families were reported to use a professional cleaning service. Pre-Pandemic, Home Cleaning Centers of America estimated that housekeeping services would topple $20 billion in the coming decade with the rising number of two-career households. However, Covid-19 has significantly impacted the industry with more people working from home and rising unemployment, making these services currently an inaccessible luxury.

I pressed the startup executive on the impact he thinks coronavirus will have on his invention. He professed that Covid has brought to light the value of homemaking. “Domestic labor is still fundamentally undervalued by standard marketing analysis because it exists largely outside the formal economy. However, the lesson of traditional home appliances indicates that once a mechanical solution to a domestic labor problem is on the market, it quickly becomes seen as part of the essential standard of living for middle-class Americans,” declared Burton. He continued to optimistically opine, “While I hope the pandemic will be over by the time our product is on the market, it once again emphasizes the ‘butterfly effect’ aspect of mass adoption whereby having the right timing is absolutely crucial. I think that being first to market in a space that is so empty of existing solutions will have this impact for us.”