A Five-Year Collaboration Between Six Universities and Industry Partners Will Place Assistive Robots in Everyday Human Environments
The University of Texas at Austin will lead a new National Science Foundation Science and Technology Center (NSF-STC) to study what happens when assistive robots leave the lab and move into the real world to live and work alongside humans. The new center takes a values-driven approach, bringing together computer scientists and engineers with researchers in the social sciences and liberal arts to pursue leading-edge science through creative, cross-disciplinary collaboration.
The NSF-STCs undertake potentially groundbreaking investigations at the interfaces of multiple disciplines provide a rich environment for encouraging scientists, engineers, and educators to take risks in pursuing discoveries and new knowledge. The UT team is poised to meet this challenge head on with its forthcoming Center for Human and Robot Co-Adaptation, which will bring together 39 researchers from six universities and multiple industry partners to establish a new science of “human-robot co-adaptation” — how people and robots learn from one another and adjust their behavior over long periods in everyday settings that will include homes, hospitals and assisted-living residences.
Partner universities are the Massachusetts Institute of Technology, Yale University, Indiana University Bloomington, the University of Utah, and Tufts University — with UT Austin distinguished among the group as home to both the center’s Principal Investigator (PI) and one of its four co-PIs. Industry collaborators include Amazon, Apptronik, Diligent Robotics, Google DeepMind, Hello Robot, MassRobotics, NVIDIA and Robust AI.
The Human Environment with Robots Journey
Most robots today are designed for specific tasks, tested and deployed in controlled environments, and judged on task completion. The new center is after something deeper: understanding how robots and people might adapt to each other over months and years — anticipating shifting needs, learning new skills, unlearning outdated ones, and picking up on unspoken social rules.
For example, teaching a robot to set a table is the easy part. Getting it to set the table before being asked — and then including soup spoons because it knows you’re making chili — is the real challenge.
That same co-adaptability could make a crucial difference in settings such as rehabilitative care. An assistive robot could learn to adjust its reach based on a person’s physical limitations so that they can take a sip of water. As the robot adapts to the person, the patient learns what else to ask for — and how — gaining a new sense of independence.
“The next leap in robotics is not just getting robots to perform more tasks,” said Joydeep Biswas,an associate professor of computer science who will serve as the center’s director. “It is enabling robots to understand the people around them: their needs, preferences, values and constraints, while recognizing that people also adapt their behavior when they integrate robots into their environment.”
The five-year, $30 million award — one of three center grants recently announced by NSF — will unite researchers in artificial intelligence, robotics, cognitive and social science, and the
humanities to study how humans and robots can coexist, grounding the work in the Human Environment with Robots (HERO) Facility Network, a set of real-world deployment sites hosted by UT and its partners, including homes, cafés, a museum, a rehabilitation hospital and elder-care residences.
“From a behavioral science perspective, this project creates an unprecedented opportunity to collect large volumes of multimodal data from real-world human–robot interactions,” said co-PI Chen Yu, professor of psychology and director of the Center for Perceptual Systems who will serve as the HERO network director. The NSF center’s roots trace in part to a departmental initiative on natural behavior and computation, which brings together psychology faculty who use computational approaches to study human behavior in real-world settings. “Following people and robots over extended periods will allow us to study human behavior at a scale and ecological richness that is difficult to achieve in traditional laboratory experiments, revealing how communication, trust, routines, and expectations emerge and change through everyday interaction.”
Every step of the way, community members will have an opportunity to influence how, when and where robots get deployed, and help shape the development of the technology so it best meets the needs and values of their community. An internal ethics board will oversee the research and develop consent and opt-out procedures.
The center grew from a core project of Good Systems, UT’s research grand challenge focused on ethical artificial intelligence, and includes faculty members from UT’s School of Computing, College of Liberal Arts, Cockrell School of Engineering, Moody College of Communication and Texas Advanced Computing Center. Alongside Yu, COLA faculty include Sam Baker, associate professor of English, and Kaya de Barbaro and Nick Duran, both associate professors of psychology.
“In Good Systems, we learned our individual fields move faster and further when we take the time to understand how our colleagues in other disciplines actually think, and bring such understanding into our own work as well as into our collaborations,” said Baker, who will study how communities form expectations about robots living among them.
Industry partners will help ensure that the research is grounded in the most cutting-edge technologies available today. By leveraging NVIDIA’s accelerated computing, simulation and robotics platforms, researchers will develop and validate new robotic capabilities, conduct large-scale training and evaluation, and explore how AI can be deployed and tested on robotic systems. The work will also examine NVIDIA foundation models designed to advance human-robot collaboration and adaptation.
Beyond the research itself, the center will build open-source AI and robotics curricula, expand undergraduate research opportunities and collaborate with high school teachers to launch a Co-Adaptive Robotics for All Learners (CORAL) program.
“This initiative is an opportunity for engineers, computer scientists, social scientists and humanists to take risks together, to pursue answers none of them could reach alone,” said Frances Champagne, associate dean of research in the College of Liberal Arts. “That’s what UT Austin makes possible, and it’s especially true here in the college. Success will mean researchers and students who’ve learned to think across disciplinary lines, and communities who helped shape this technology around their own values from the start.”
