About 1 million adults in the United States need someone to help them eat, according to .

It’s a time-consuming and often awkward task, one largely done out of necessity rather than choice.

Researchers at the 91爆料 are working on a robotic system that can help make it easier. After identifying different foods on a plate, the robot can strategize how to use a fork to pick up and deliver the desired bite to a person’s mouth.

The team published its results in a series of papers: One in IEEE Robotics and Automation Letters, while the other March 13 at the in South Korea.

“Being dependent on a caregiver to feed every bite every day takes away a person’s sense of independence,” said corresponding author , the Boeing Endowed Professor in the 91爆料’s Paul G. Allen School of Computer Science & Engineering. “Our goal with this project is to give people a bit more control over their lives.”

The idea was to develop an autonomous feeding system that would be attached to people’s wheelchairs and feed people whatever they wanted to eat.

“When we started the project we realized: There are so many ways that people can eat a piece of food depending on its size, shape or consistency. How do we start?” said co-author , a postdoctoral research associate in the Allen School. “So we set up an experiment to see how humans eat common foods like grapes and carrots.”

The researchers arranged plates with about a dozen different kinds of food, ranging in consistency from hard carrots to soft bananas. The plates also included foods like tomatoes and grapes, which have a tough skin and soft insides. Then the team gave volunteers a fork and asked them to pick up different pieces of food and feed them to a mannequin. The fork contained a sensor to measure how much force people used when they picked up food.

The volunteers used various strategies to pick up food with different consistencies. For example, people skewered soft items like bananas at an angle to keep them from slipping off the fork. For items like carrots and grapes, the volunteers tended to use wiggling motions to increase the force and spear each bite.

“People seemed to use different strategies not just based on the size and shape of the food but also how hard or soft it is. But do we actually need to do that?” Bhattacharjee said. “We decided to do an experiment with the robot where we had it skewer food until the fork reached a certain depth inside, regardless of the type of food.”

The robot used the same force-and-skewering strategy to try to pick up all the pieces of food, regardless of their consistency. It was able to pick up hard foods, but it struggled with soft foods and those with tough skins and soft insides. So robots, like humans, need to adjust how much force and angle they use to pick up different kinds of food.

The team also noted that the acts of picking up a piece of food and feeding it to someone are not independent of each other. Volunteers often would specifically orient a piece of food on the fork so that it could be eaten easily.

“You can pick up a carrot stick by skewering it in the center of the stick, but it will be difficult for a person to eat,” Bhattacharjee said. “On the other hand, if you pick it up on one of the ends and then tilt the carrot toward someone’s mouth, it’s easier to take a bite.”

To design a skewering and feeding strategy that changes based on the food item, the researchers combined two different algorithms. First they used an object-detection algorithm called RetinaNet, which scans the plate, identifies the types of food on it and places a frame around each item.

Then they developed SPNet, an algorithm that examines the type of food in a specific frame and tells the robot the best way to pick up the food. For example, SPNet tells the robot to skewer a strawberry or a slice of banana in the middle, and spear carrots at one of the two ends.

The team had the robot pick up pieces of food and feed them to volunteers using SPNet or a more uniform strategy: an approach that skewered the center of each food item regardless of what it was. SPNet’s varying strategies outperformed or performed the same as the uniform approach for all the food.

“Many engineering challenges are not picky about their solutions, but this research is very intimately connected with people,” Srinivasa said. “If we don’t take into account how easy it is for a person to take a bite, then people might not be able to use our system. There’s a universe of types of food out there, so our biggest challenge is to develop strategies that can deal with all of them.”

The team is currently working with the to get feedback from caregivers and patients in assisted living facilities on how to improve the system to match people’s needs.

“Ultimately our goal is for our robot to help people have their lunch or dinner on their own,” Srinivasa said. “But the point is not to replace caregivers: We want to empower them. With a robot to help, the caregiver can set up the plate, and then do something else while the person eats.”

Co-authors for the first paper include doctoral student and research scientist , both at the Allen School. Co-authors on the second paper include Daniel Gallenberger, a master’s student at Technische Universit盲t M眉nchen in Germany who completed this research while at the 91爆料, and , a research scientist at the Allen School. This work was first debuted at the recent , where it won a best demo award.

This research was funded by the National Institutes of Health, the National Science Foundation, the Office of Naval Research, the Robotics Collaborative Technology Alliance, Amazon and Honda.

###

For more information, contact Srinivasa at siddh@cs.washington.edu.

Grant numbers: R01EB019335, 1544797, 1637748

Children are curious because it helps them better understand their world. Now researchers are curious if the same is true for robots.

Honda Research Institute USA, Inc., today , called Curious Minded Machine, which will design a robot or system that learns continuously in a humanlike, curiosity-driven way. Curious robots would be lifelong learners that could expand their list of skills without any additional training.

The 91爆料 will lead one of three teams that will partner with the institute to explore the mechanisms behind curiosity and seek advances in artificial cognition. The 91爆料-led team will receive $2.7 million over the next three years to generate a mathematical model of curiosity.

“We wish to explore several questions in our work,” said team leader , who is the Boeing Endowed professor with the 91爆料’s Paul G. Allen School of Computer Science & Engineering. “What is curiosity? Can we build a rich mathematical model that makes a robot curious? Will a curious robot be accepted more? Will we be more tolerant of its mistakes?”

Two other Allen School researchers, assistant professor and professor , bring their experience studying human鈥搑obot interactions, including designing programmable robots that users can personalize for specific tasks and developing ways for robots to perceive objects in an environment. , an acting associate professor of psychology at the University of California, Santa Cruz, rounds out the team with expertise on the social science of how humans interact with robots.

“Our first step is to better understand curiosity in humans, starting from infants’ constant experimentation with their surroundings, to 4-year-olds asking why everything is the way it is, to adults’ interest in topics completely outside their professions,” Cakmak said. “Humans are intrinsically rewarded by new information even when that information is not necessarily applicable, but curiosity has long-term benefits. We would like to give robots similar benefits for being curious.”

After the researchers develop a model of curiosity, they hope to use it to create two separate robot prototypes: a social robot that interacts with people in an office building or a home, and an arm robot that could manipulate objects placed in front of it, like in an assembly line. The team argues that the same model could be used for both prototypes.

“Let’s say our model of curiosity rewards the robot for obtaining novel information that is irrelevant to its current task,” Cakmak said. “For the building robot this could manifest as taking a different path on the way back from a delivery. For the arm robot, this could result in the robot ‘playing’ with objects that are not part of its current task.”

Curious robots, the team said, are a step toward making robots that can perform constantly changing tasks. For example, future caregiver robots will need to be able to adjust their jobs to meet patients’ fluctuating needs.

“We think that curious robots will not only be better at their jobs, but they will appeal more to people,” Cakmak said. “Prior work also shows that robots can spark curiosity in people, which would be a wonderful side effect of curious robots.”

The 91爆料-led team’s efforts will dovetail with work by two other teams, led by the Massachusetts Institute of Technology and the University of Pennsylvania. These teams will address other areas, such as how robots perceive and interact with the world and how robots can predict upcoming actions. After the three-year program, Honda Research Initiative will combine the work from all the teams to form the foundation for a future curious-minded robot.

“Honda is a pioneer in robotics research,” Srinivasa said. “Their was a game-changer in locomotion, motion planning, control and AI. To date, it is one of the most impressive and intelligent robots I have ever seen. With this collaboration, we hope to bring about a similar wave of new excitement to human鈥搑obot interaction and the field of robotics.”

This fall, the 91爆料’s annual will feature three College of Engineering faculty whose research is accelerating positive impact here and around the world. Their lectures 鈥� on assistive robots, environmental equity and disaster relief 鈥� are free and open to the public, but seating is limited and .

Building a robot butler: Toward fluent human-robot interaction

The series kicks off Thursday, Oct. 11, in Kane Hall 130. , an associate professor in the Paul G. Allen School of Computer Science & Engineering, will be discussing his work on building caregiver robots. Robots with the capability to interact with humans as equals have potential to improve the daily lives of people who require assistive care, such as those with special needs. Learn how researchers are developing these robots using mathematical models and physics-based manipulation.

Updated 12/7/18 – video

Clearing the air: Environmental justice and air quality

On Tuesday, Oct. 30, in Kane Hall 130, civil and environmental engineering professor will talk about how air pollution is the leading environmental health risk in the U.S. 鈥� and is responsible for thousands of deaths each year. His research examines how air pollution impacts different groups and has revealed that, on average, people of color are exposed to more air pollution. Now he is testing solutions to reduce the exposure disparity.

Updated 12/7/18 – video

Meeting our global obligations: The Hurricane Maria energy & health project

The lecture series closes on Tuesday, Nov. 13, in Kane Hall 130 with chemical engineering associate professor , who uses nanotechnology for clean energy and healthcare applications. In September 2017, Hurricane Maria devastated Puerto Rico and left its residents without power, water and sanitation systems. A group led by Pozzo initiated a combined research and service project to assess the disaster鈥檚 impact on the health of rural residents. She will discuss how this project provided emergency clean energy that helped vulnerable people in this community.

Updated 12/7/18 – video

All lectures are free and start at 7:30 p.m. Advance registration, either or by calling 206-543-0540, is required. All lectures will be broadcast at a later date on 91爆料TV.

###

, professor of industrial and systems engineering, was honored for 鈥渓eadership in virtual and augmented reality鈥� and , professor in the Paul G. Allen School of Computer Science & Engineering, was recognized for 鈥渃ontributions to robotic manipulation and human-robot interaction.鈥�

The IEEE Fellow distinction is reserved for select members who exhibit an extraordinary record of accomplishments in any of the IEEE fields of interest, which include aerospace systems, biomedical engineering, computing, consumer electronics, energy, telecommunications and more. Nominated by peers and conferred by the IEEE Board of Directors, fellowship is considered both a prestigious honor and a noteworthy career achievement within the technical community. The total number selected in any one year does not exceed one-tenth of 1 percent of the Institute’s total voting membership.

Furness is a pioneer in human interface technology and grandfather of virtual reality. In addition to his ISE professorship, he holds adjunct professorships in electrical engineering, mechanical engineering and human-centered design and engineering. He is the founder of the聽聽(HIT Lab) at 91爆料 and sister HIT Labs at the University of Canterbury in Christchurch, New Zealand, and the University of Tasmania, in Australia. He is also the founder of the Virtual World Society, which is dedicated to bringing together hearts and minds through virtual reality to solve pervasive problems in the world.

Prior to joining the faculty at the 91爆料 in 1989, Furness served a combined 23 years as a U.S. Air Force officer and civilian scientist developing advanced cockpits and virtual interfaces for the Department of Defense. Furness lectures and speaks widely on virtual reality innovations and holds 21 patents in advanced sensor, display and interface technologies.

厂谤颈苍颈惫补蝉补听聽this past fall as the Boeing Endowed Professor from the faculty of Carnegie Mellon University, where he was a member of the Robotics Institute and founding director of the Personal Robotics Lab. He has made pioneering contributions to two fundamental areas of robotics, robotic manipulation and human-robot interaction (HRI), with the aim of enabling robots to perform complex tasks with and around people. A full-stack roboticist, Srinivasa has built several end-to-end systems that integrate perception, planning and control in the real world.

厂谤颈苍颈惫补蝉补鈥檚听聽in manipulation has enabled robots to push, pull and sweep objects under conditions of clutter and uncertainty through non-prehensile, physics-based interactions. He also is credited with having created the field of algorithmic HRI through his efforts to build the formal mathematical foundations of human-robot interaction. To that end, Srinivasa and his team built HERB, the Home Exploring Robot Butler, to serve as a realistic testbed for new algorithms enabling human-robot collaboration. In addition to his role in the lab, HERB has become an ambassador of sorts for Srinivasa and his team 鈥� and for the field of robotics, generally.