Augmented reality in endoscopy and surgery

Minimally invasive surgery represents a major evolution in surgical techniques, bringing many benefits to the patient: less post-operative pain, reduced hospitalization time and a quicker return to activity. However, this type of surgery also poses new challenges for the surgeon, the main ones being the loss of gesture sensitivity and direct vision. It is these limitations that computer science and robotics propose to overcome by adding assistance to gestures based on the use of medical imaging. By combining the disciplines of radiology and surgery, surgery becomes a hybrid. Before the operation, it is possible to know the precise anatomy of each patient thanks to personalized 3D modeling. This digital clone of the patient can be used to monitor the evolution of a pathology or to define an optimal therapeutic strategy. The surgeon can train without risk to the patient. During surgery, this 3D model can be used to guide the surgeon using augmented reality. This technique consists of superimposing a 3D model of the patient's organs and pathologies onto the operating video, making the patient virtually transparent. Thanks to intra-operative medical imaging, the new hybrid operating theatres will make these techniques increasingly reliable, enabling certain surgical gestures to be automated by surgical robots. This revolution in progress is illustrated by the numerous results obtained by teams from the IRCAD, the hospital and the University of Strasbourg, grouped together within the IHU de Strasbourg, including the first robotized partial liver resection guided by augmented reality, carried out in September 2012.