635 World’s First Live AI-Assisted Augmented Reality Robotic Colon Surgery with Instrument De-occlusion

Dr. Thalia Petropoulou

In robotic colorectal surgery, augmented reality 3D overlays can enhance orientation and safety. Yet a major limitation has been instrument occlusion—the overlay is blocked whenever surgical tools appear in the field, making it difficult to visualize anatomy around critical vessels and tumors. To overcome this, we developed an AI-assisted augmented reality system with real-time instrument de-occlusion. Using deep learning, the system automatically segments instruments and foreign bodies from the endoscopic image, ensuring that the 3D model of the patient’s anatomy remains continuously visible. The innovation lies in merging artificial intelligence with live AR guidance, allowing surgeons to maintain full awareness of both instruments and anatomy simultaneously. This represents a world-first application in robotic colon surgery, offering improved safety, teaching potential, and a pathway toward personalized, AI-enhanced operative care.
Procedural Innovation, Gastrointestinal Surgery, AI Augmented Reality, 3D Models

See more at: https://vattikutifoundation.com/videos/