An AI-based machine vision system for monitoring in vitro diagnostic devices: system design and algorithm evaluation

This thesis focuses on the development of Artificial Intelligence techniques for a Machine Vision System designed to improve the efficiency and reliability of In Vitro Diagnostic medical devices. The project originates from a real-world industrial need: the company holding the system’s patent sought to reduce operational errors and prevent damage to the medical devices that automate the analysis of biological fluids. The work was carried out as part of an industrial PhD program, and the doctoral project was developed in collaboration with MACS s.r.l., the company that owns the patent. The proposed vision system is integrated into a laboratory diagnostic device and concentrates on monitoring the component responsible for fluid handling, the pipette tip, or simply tip. The system provides direct and comprehensive visual feedback at two critical moments during device operation: the gripping of the tip by the IVD instrument, and the aspiration or dispensing of fluids by the tip itself. To achieve this, three distinct monitoring tasks were developed. The first checks whether the tip has been correctly picked up by the device. The second ensures that the tip being used has the correct size. The third estimates the amount of liquid aspirated or dispensed by the tip. To implement these functions, we adopted a hybrid approach that combines classical image processing techniques with Machine Learning and Deep Learning models. While machine vision systems powered by artificial intelligence have been applied in various fields such as manufacturing, agriculture, and healthcare, their use for this specific type of diagnostic device remains unexplored in the current literature. Different solutions were designed depending on the task. For the tip presence and size verification, we evaluated and compared several Support Vector Machines with different kernel functions and carefully selected features. For the size classification task, in addition to support vector machines, we also developed a Convolutional Neural Network to directly compare the performance of deep learning and traditional machine learning models. For the estimation of liquid volume, we implemented and compared both Convolutional Neural Networks and Vision Transformers. The results were promising across all tasks, showing high accuracy and strong generalization performance. Given the growing importance of interpretability in deep learning, we further analysed the models using various explainability techniques to gain insights into their decision-making processes. Finally, since all models are intended for real-time application within the diagnostic workflow, we verified that they met the required processing times and were successfully integrated into the system’s control software. This work demonstrates that vision-based solutions driven by artificial intelligence can significantly enhance the reliability and automation of laboratory diagnostic tools, offering a foundation for the next generation of intelligent medical devices.