The goal of this project is to perform multimodal Geometric Deep Learning for Diagnosing COVID-19 Pneumonia from both Chest CT-Scan and X-Ray Images and compare the results with that of only using deep learning methods.
Due to the COVID-19 Pandemic, doctors need to make medical decisions for their patients based on many examinations including CT-Scans and X-rays. Many current deep learning methods such as transfer learning have been used in several researches and focuses on only a single modality of biomarkers such as CT-Scan or X-Ray alone for diagnosing Pneumonia.
In this research we want to combine both CT-Scan and X-Ray of the chest using geometric methods to improve the classification accuracy since recent study has shown that use different biomarkers may provide complementary information for detecting COVID-19 Pneumonia. Specifically, we propose to integrate geometrically two different methods, U-Net and geometric learning, using an open-source dataset of 2500 CT-Scan images and 2500 X-ray images for classifying CT-Scan images and X-ray images into two classes: normal and COVID-19 Pneumonia.
Professor Weiqing Gu has developed some cutting-edge Geometric Learning Methods which are effective and efficient when they are applied in Medical diagnosis. Unlike deep learning methods, Geometric Learning methods are intuitive and explainable, and they integrate AI with medicine without requiring a medical doctor getting another Ph.D. degree in Machine Learning.
In previous summer researches, Dr. Gu has applied Geometric Learning in cancer diagnosis and recently she and her team has applied U-Net based model together with some geometric analysis to automatically segment the infected regions to find the volume of the infection in the lungs. Their method was evaluated on two datasets comprised of 20 COVID-19 positive patients. The segmentation model achieves a DICE score of 0.8722 on testing data, suggesting promising segmentation efficiency. When they were conduct this research more than 1.5 year ago, not much positive COVID-19 data was available. Now there are much more data available including per patient having both X-ray and CT scan data.
For this summer, Dr. Gu and her team plan to further investigate the power in using geometric data analytic methods when integrating it with U-Net model and want to extend the methods to a multimodal geometric fusion method using Chest CT-Scan and X-Ray Images.
Once the chest scans are taken and the infectious areas are automatically detected, it can be sent to the radiologist. This allows the radiologist efficiently to examine the scans remotely and decide the plan of treatment. In this case, same time is saving lives!