CUHK’s newly-created embodied intelligence platform successfully completes the world’s first multi-task surgical automation tests on a live animal
A multidisciplinary research team from The Chinese University of Hong Kong (CUHK)’s Faculty of Medicine (CU Medicine) and Faculty of Engineering (CU Engineering) has developed new artificial intelligence (AI)-powered surgical robot automation techniques, successfully completing the world’s first multi-task surgical automation tests on a live animal. The research has been published in the prestigious multidisciplinary research journal Science Robotics.
A multidisciplinary research team from CU Medicine and CU Engineering has developed new AI-powered surgical robot automation techniques, successfully completing the world’s first multi-task surgical automation tests on a live animal. (from left) Professor Dou QI, Dr Yip Hon-chi, Professor Samuel Au Kwok-wai and Professor Philip Chiu Wai-yan
Embodied intelligence technology leads to breakthroughs in surgical robot automation
Surgical robots have performed millions of minimally invasive procedures worldwide. Autonomy is envisaged for next-generation surgical robots, enhancing operational efficiency and consistency, while alleviating pressure on medical resources.
Professor Dou Qi, Assistant Professor from CUHK’s Department of Computer Science and Engineering, who led the study, said: “Traditional surgical automation approaches often relied on additional sensors or predefined models, which limited their clinical applicability. We used innovative AI techniques to create a brand-new embodied intelligence framework for surgical robot automation, contributing a data-driven and purely vision-based solution that is the first of its kind globally.”
Professor Dou Qi
This surgical embodied intelligence framework can analyse endoscopic images in real time, without additional sensors. The framework integrates advanced visual foundation models, reinforcement learning and visual servoing techniques to achieve accurate, efficient and safe automation of various surgical tasks. Its foundation-model-based visual perception allows it to robustly perform surgical scene understanding and depth estimation in practice. The reinforcement learning-based control policy was trained using SurRoL, an embodied AI simulator that the team developed, and the simulation-trained policy can be directly deployed in real-world robots via zero-shot sim-to-real transfer. In this research, the developed AI system has been seamlessly integrated into the Sentire® Surgical System which has distinctive AI-readiness and AI-friendly characteristics. This data-driven paradigm eliminates task-specific engineering, providing a general-purpose solution for versatile surgical autonomy through embodied AI, accelerating the translation from concept to pre-clinical testing.
In vivo testing validates AI-powered multi-task autonomy and human-robot collaboration
The research team conducted in vivo testing of the AI system using a live animal model that replicated clinical surgical conditions. The system successfully performed multiple autonomous surgical tasks, including tissue retraction, gauze picking and blood vessel clipping – actions that surgeons regularly perform during operations.
Dr Yip Hon-chi, Assistant Professor from Department of Surgery at CU Medicine, who led the animal testing, said: “This represents a breakthrough in AI-powered surgical robot automation, validated across diverse tasks and environmental conditions. Our system demonstrates remarkable generalisability, maintaining stable performance despite environmental changes such as different tissue appearances and varying lighting conditions.”
Dr Yip Hon-chi
The technology has the potential to enable the automated robotic arm to function as a surgeon’s third hand, providing assistance during complex procedures. By automating routine tasks with an AI assistant, the system can potentially significantly reduce surgeon workload, improve overall surgical efficiency and shorten procedure time for patients.
InnoHK Multi-Scale Medical Robotics Center (MRC) provides an international platform for high-impact research
The InnoHK Multi-Scale Medical Robotics Center (MRC) played a pivotal role in this groundbreaking research. SurRoL was developed through a strategic collaboration between CUHK and Johns Hopkins University (JHU) in the United States, fostered by the MRC’s international network. The research team open-sourced the surgical embodied AI software infrastructure to the global surgical robotics research community in 2021, and it has since been adopted by numerous prestigious research institutions worldwide.
Professor Samuel Au Kwok-wai, Co-director of MRC and Professor from CUHK’s Department of Mechanical and Automation Engineering, said: “This work exemplifies the exceptional innovations that can emerge from international collaborations cultivated by the MRC. The research has achieved pioneering advancements in AI-powered surgical robot automation,”
The live animal experiments were conducted in the MRC’s hybrid operating room, which provided professional support for pre-clinical evaluation. This environment allowed the surgeon to rigorously test the newly developed AI algorithms under conditions that closely resemble actual surgical settings. Professor Philip Chiu Wai-yan, Co-director of MRC and Dean of CU Medicine, said: “The MRC creates a unique synergy of engineering innovation and surgical expertise, significantly accelerating the journey from laboratory concepts to pre-clinical studies. This engineer-clinician collaborative research showcases the transformative potential of AI co-pilots in robotic surgery, positioning CUHK at the forefront of the global advancement of surgeon-AI-robot partnerships.”
The work was supported by the InnoHK initiative of the Hong Kong government’s Innovation and Technology Commission, the Hong Kong Research Grants Council, and the National Natural Science Foundation of China.
Video: In vivo testing
Video: Presentation and demonstration
