AI System for Automatic Robotic Microsurgery

This study introduces a comprehensive framework aimed at advancing artificial intelligence and robotics technology within the context of cancer diagnosis and treatment. The framework consists of three key components. Firstly, we employ an unsupervised learning method to enhance Optical Coherence Tomography (OCT) images. Subsequently, the improved OCT images serve as a foundation for cancer detection and classification. Lastly, the automatic computer-assisted laser microsurgery system (Auto-CALM) ablates precisely target area defined by the surgeon. Component 1: OCT Image Enhancement Optical coherence tomography (OCT) is a vital imaging technology for visualizing living tissues, but image quality is often compromised by speckle noise and motion blur. To address this, an unsupervised learning method, termed the one-step enhancer (OSE), is proposed. OSE utilizes a generative adversarial network (GAN) to denoise and deblur OCT images in a single step. The network employs encoders to disentangle raw images into content, blur, and noise domains, facilitating the generation of high-quality images. The proposed method demonstrates effectiveness in denoising and deblurring, offering enhanced visual experiences for clinicians and supporting the development of autonomous OCT-guided surgical robotic systems. Component 2: Melanoma Cell Classification with Raman Spectroscopy and CGAN The cancer classification framework was proposed to detect and classify cancer based on enhanced OCT images. Due to a lack of OCT data related to cancer, this study explores the synergy between Raman spectroscopy and Conditional Generative Adversarial Networks (CGAN). The research specifically aims to enhance the accuracy of classifying different cell types, including melanoma cells, normal melanocytes, normal skin fibroblasts, and tumor-associated fibroblasts. CGAN is employed for data augmentation to address the challenge posed by limited samples. The experimental results demonstrate significant performance improvements in deep learning models for tasks involving spectroscopic data after the implementation of data augmentation. This innovative approach paves the way for advancements in cell classification and detection, holding potential applications across various biomedical fields. Component 3: Automatic Laser Surgery with Auto-CALM The third component introduces the Automatic Computer-Assisted Laser Microsurgery System (Auto-CALM), a novel controller for real-time dynamic laser ablation. Auto-CALM enables precise ablation of defined areas, compensating for tissue motions and deformations. The system incorporates target tracking, laser tracking, and an ablation control algorithm. Target tracking is achieved through improved optical flow and scaling strategies, while laser tracking utilizes a pretrained Segment Anything Model. The ablation algorithm generates trajectories based on dynamically updated laser and target positions, ensuring motion compensation and enhancing system accuracy. Validation experiments demonstrate Auto-CALM's effectiveness, achieving a Dice Similarity Coefficient of 95.49\% under challenging conditions. The platform exhibits potential for accurate tissue ablation in clinical settings, with future studies aimed at translation to clinical use through ex-vivo and in-vivo investigations.