What Does It Take for a Robot to Talk to a Whole Room?

Can a robot engage in a conversation with an entire room, not just a single person? This was one of the fundamental questions explored during my week at the Imperial Robotics Summer School, held from July 20th to 24th at Imperial College London. The experience sharpened my thinking and encouraged me to approach robotics challenges from new perspectives that differ from my usual day-to-day work.

The summer school brought together emerging researchers, academics, and industry professionals from across the robotics community for an intensive, hands-on program. I gained a wealth of knowledge from expert-led lectures covering robot kinematics, dynamics, sensing, control, and robot learning, as well as specialized sessions on aerial robotics, robot intelligence, surgical robot vision, bio-inspired sensing, and personal assistive robotics.

One of the highlights was exploring Imperial's robotics labs, where I observed cutting-edge research in adaptive and intelligent robotics, aerial systems, robotic surgery, manipulation, and assistive technologies. Alongside researchers from the UK and beyond, we tackled real-world challenges involving quadruped locomotion, vision-based learning, tactile sensing, and healthcare applications.

Our group project centered on enabling a robot to interact with multiple individuals simultaneously, moving beyond the typical one-on-one interactions that most social robots are designed for. We aimed to detect the active speaker in real time, track each person's identity throughout the interaction, and generate natural gaze and head movements so the robot appears attentive to the entire group, not just one individual.

Through this project, I gained a comprehensive understanding of the entire pipeline from language to motion. I learned how a large language model (LLM) translates natural language commands into structured instructions, how these are mapped to high-level robot functions, and how inverse kinematics converts them into joint trajectories. Finally, I saw how these trajectories reach the actuators via the robot's control stack.

This hands-on experience illustrated the true complexity of multi-party human-robot interaction, emphasizing the need for interdisciplinary approaches. I left with a deeper appreciation for the collaborative effort required in robotics research.