661 Surgical Copilot: A Vision-Language Model Understanding Prostatectomies

Dr. Francois Remy

Being a surgeon requires expertise, awareness, and calm decision-making, yet trainees often lack immediate senior guidance. While AI copilots exist in other fields, surgery has lagged because so much context is visual. We introduce the Surgical AI Copilot for Robot-Assisted Prostatectomies, a vision-language model trained on annotated prostatectomy frames enriched with spatial, temporal, and depth information. From this, we generated over one million question-answer pairs to teach our system to understand surgical scenes. Running on secure edge hardware in the OR, it answers questions in real time without sending data to the cloud. Tested offline and live during surgery, it achieved up to 83% accuracy and 77% correct answers in real procedures, showing promise for education, workflow support, and surgical safety.
Technological Innovation, Artificial Intelligence, Robot-Assisted Radical Prostatectomy

See more at: https://vattikutifoundation.com/videos/