publications

2025

1. arXiv

   Integration of nested cross-validation, automated hyperparameter optimization, high-performance computing to reduce and quantify the variance of test performance estimation of deep learning models

   Paul Calle, Averi Bates, Justin C. Reynolds, and 10 more authors

   2025

   Abs Bib HTML Code

   The variability and biases in the real-world performance benchmarking of deep learning models for medical imaging compromise their trustworthiness for real-world deployment. The common approach of holding out a single fixed test set fails to quantify the variance in the estimation of test performance metrics. This study introduces NACHOS (Nested and Automated Cross-validation and Hyperparameter Optimization using Supercomputing) to reduce and quantify the variance of test performance metrics of deep learning models. NACHOS integrates Nested Cross-Validation (NCV) and Automated Hyperparameter Optimization (AHPO) within a parallelized high-performance computing (HPC) framework. NACHOS was demonstrated on a chest X-ray repository and an Optical Coherence Tomography (OCT) dataset under multiple data partitioning schemes. Beyond performance estimation, DACHOS (Deployment with Automated Cross-validation and Hyperparameter Optimization using Supercomputing) is introduced to leverage AHPO and cross-validation to build the final model on the full dataset, improving expected deployment performance. The findings underscore the importance of NCV in quantifying and reducing estimation variance, AHPO in optimizing hyperparameters consistently across test folds, and HPC in ensuring computational feasibility. By integrating these methodologies, NACHOS and DACHOS provide a scalable, reproducible, and trustworthy framework for DL model evaluation and deployment in medical imaging.

   @misc{calle_2025,
     author = {Calle, Paul and Bates, Averi and Reynolds, Justin C. and Liu, Yunlong and Cui, Hoayang and Ly, Sinaro and Wang, Chen and Zhang, Qinghao and de Armendi, Alberto J. and Shettar, Shashank S. and Fung, Kar-Ming and Tang, Qinggong and Pan, Chongle},
     title = {Integration of nested cross-validation, automated hyperparameter optimization, high-performance computing to reduce and quantify the variance of test performance estimation of deep learning models},
     publisher = {arXiv},
     year = {2025},
     type = {preprint},
     url = {https://doi.org/10.48550/arXiv.2503.08589},
   }

2024

1. ACM CHI

   Towards AI-Driven Healthcare: Systematic Optimization, Linguistic Analysis, and Clinicians’ Evaluation of Large Language Models for Smoking Cessation Interventions

   Paul Calle*, Ruosi Shao*, Yunlong Liu, and 5 more authors

   2024

   Abs Bib HTML

   Creating intervention messages for smoking cessation is a laborintensive process. Advances in Large Language Models (LLMs) offer a promising alternative for automated message generation. Two critical questions remain: 1) How to optimize LLMs to mimic human expert writing, and 2) Do LLM-generated messages meet clinical standards? We systematically examined the message generation and evaluation processes through three studies investigating prompt engineering (Study 1), decoding optimization (Study 2), and expert review (Study 3). We employed computational linguistic analysis in LLM assessment and established a comprehensive evaluation framework, incorporating automated metrics, linguistic attributes, and expert evaluations. Certified tobacco treatment specialists assessed the quality, accuracy, credibility, and persuasiveness of LLM-generated messages, using expert-written messages as the benchmark. Results indicate that larger LLMs, including ChatGPT, OPT-13B, and OPT-30B, can effectively emulate expert writing to generate well-written, accurate, and persuasive messages, thereby demonstrating the capability of LLMs in augmenting clinical practices of smoking cessation interventions.

   @misc{calle_2024,
     author = {Calle*, Paul and Shao*, Ruosi and Liu, Yunlong and Hébert, Emily T. and Kendzor, Darla and Neil, Jordan and Businelle, Michael and Pan, Chongle},
     title = {Towards AI-Driven Healthcare: Systematic Optimization, Linguistic Analysis, and Clinicians’ Evaluation of Large Language Models for Smoking Cessation Interventions},
     publisher = {Association for Computing Machinery},
     year = {2024},
     type = {Conference Paper},
     url = {https://doi.org/10.1145/3613904.3641965},
   }

2. Comms. Eng.

   Automatic renal carcinoma biopsy guidance using forward-viewing endoscopic optical coherence tomography and deep learning

   Chen Wang, Haoyang Cui, Qinghao Zhang, and 11 more authors

   Communications Engineering, 2024

   Abs Bib HTML

   Percutaneous renal biopsy is commonly used for kidney cancer diagnosis. However, the biopsy procedure remains challenging in sampling accuracy. Here we introduce a forward-viewing optical coherence tomography probe for differentiating tumor and normal tissues, aiming at precise biopsy guidance. Totally, ten human kidney samples, nine of which had malignant renal carcinoma and one had benign oncocytoma, were used for system evaluation. Based on their distinct imaging features, carcinoma could be efficiently distinguished from normal renal tissues. Additionally, oncocytoma could be differentiated from carcinoma. We developed convolutional neural networks for tissue recognition. Compared to the conventional attenuation coefficient method, convolutional neural network models provided more accurate carcinoma predictions. These models reached a tissue recognition accuracy of 99.1% on a hold-out set of four kidney samples. Furthermore, they could efficiently distinguish oncocytoma from carcinoma. In conclusion, our convolutional neural network-aided endoscopic imaging platform could enhance carcinoma diagnosis during percutaneous renal biopsy procedures.

   @article{wang2024automatic,
     title = {Automatic renal carcinoma biopsy guidance using forward-viewing endoscopic optical coherence tomography and deep learning},
     author = {Wang, Chen and Cui, Haoyang and Zhang, Qinghao and Calle, Paul and Yan, Yuyang and Yan, Feng and Fung, Kar-Ming and Patel, Sanjay G and Yu, Zhongxin and Duguay, Sean and Vanlandingham, William and Jain, Ajay and and Pan, Chongle and Tang, Qinggong},
     journal = {Communications Engineering},
     volume = {3},
     number = {1},
     pages = {107},
     year = {2024},
     publisher = {Nature Publishing Group UK London},
     url = {https://doi.org/10.1038/s44172-024-00254-9},
   }

3. J. Biophotonics

   Enhancing epidural needle guidance using a polarization-sensitive optical coherence tomography probe with convolutional neural networks

   Chen Wang, Yunlong Liu, Paul Calle, and 9 more authors

   2024

   Abs Bib HTML Code

   Abstract Epidural anesthesia helps manage pain during different surgeries. Nonetheless, the precise placement of the epidural needle remains a challenge. In this study, we developed a probe based on polarization-sensitive optical coherence tomography (PS-OCT) to enhance the epidural anesthesia needle placement. The probe was tested on six porcine spinal samples. The multimodal imaging guidance used the OCT intensity mode and three distinct PS-OCT modes: (1) phase retardation, (2) optic axis, and (3) degree of polarization uniformity (DOPU). Each mode enabled the classification of different epidural tissues through distinct imaging characteristics. To further streamline the tissue recognition procedure, convolutional neural network (CNN) were used to autonomously identify the tissue types within the probe’s field of view. ResNet50 models were developed for all four imaging modes. DOPU imaging was found to provide the highest cross-testing accuracy of 91.53%. These results showed the improved precision by PS-OCT in guiding epidural anesthesia needle placement.

   @article{chen_2024,
     author = {Wang, Chen and Liu, Yunlong and Calle, Paul and Li, Xinwei and Liu, Ronghao and Zhang, Qinghao and Yan, Feng and Fung, Kar-ming and Conner, Andrew K. and Chen, Sixia and Pan, Chongle and Tang, Qinggong},
     title = {Enhancing epidural needle guidance using a polarization-sensitive optical coherence tomography probe with convolutional neural networks},
     journal = {Journal of Biophotonics},
     volume = {17},
     number = {2},
     pages = {e202300330},
     issn = {1864-063X},
     doi = {https://doi.org/10.1002/jbio.202300330},
     url = {https://onlinelibrary.wiley.com/doi/abs/10.1002/jbio.202300330},
     year = {2024},
     type = {Journal Article},
   }

2022

1. Sci. Rep.

   Epidural anesthesia needle guidance by forward-view endoscopic optical coherence tomography and deep learning

   Chen Wang*, Paul Calle*, Justin C. Reynolds, and 10 more authors

   Scientific Reports, 2022

   Abs Bib HTML Code

   Epidural anesthesia requires injection of anesthetic into the epidural space in the spine. Accurate placement of the epidural needle is a major challenge. To address this, we developed a forward-view endoscopic optical coherence tomography (OCT) system for real-time imaging of the tissue in front of the needle tip during the puncture. We tested this OCT system in porcine backbones and developed a set of deep learning models to automatically process the imaging data for needle localization. A series of binary classification models were developed to recognize the five layers of the backbone, including fat, interspinous ligament, ligamentum flavum, epidural space, and spinal cord. The classification models provided an average classification accuracy of 96.65%. During puncture, it is important to maintain a safe distance between the needle tip and the dura mater. Regression models were developed to estimate that distance based on the OCT imaging data. Based on the Inception architecture, our models achieved a mean absolute percentage error of 3.05% ± 0.55%. Overall, our results validated the technical feasibility of using this novel imaging strategy to automatically recognize different tissue structures and measure the distances ahead of the needle tip during the epidural needle placement.

   @article{chen2022b,
     author = {Wang*, Chen and Calle*, Paul and Reynolds, Justin C. and Ton, Sam and Yan, Feng and Donaldson, Anthony M. and Ladymon, Avery D. and Roberts, Pamela R. and de Armendi, Alberto J. and Fung, Kar-ming and Shettar, Shashank S. and Pan, Chongle and Tang, Qinggong},
     title = {Epidural anesthesia needle guidance by forward-view endoscopic optical coherence tomography and deep learning},
     journal = {Scientific Reports},
     volume = {12},
     number = {1},
     pages = {9057},
     issn = {2045-2322},
     doi = {10.1038/s41598-022-12950-7},
     url = {https://doi.org/10.1038/s41598-022-12950-7},
     year = {2022},
   }

2. J. Biophotonics

   Computer-aided Veress needle guidance using endoscopic optical coherence tomography and convolutional neural networks

   Chen Wang, Justin C. Reynolds, Paul Calle, and 9 more authors

   Journal of Biophotonics, 2022

   Abs Bib

   Abstract During laparoscopic surgery, the Veress needle is commonly used in pneumoperitoneum establishment. Precise placement of the Veress needle is still a challenge for the surgeon. In this study, a computer-aided endoscopic optical coherence tomography (OCT) system was developed to effectively and safely guide Veress needle insertion. This endoscopic system was tested by imaging subcutaneous fat, muscle, abdominal space, and the small intestine from swine samples to simulate the surgical process, including the situation with small intestine injury. Each tissue layer was visualized in OCT images with unique features and subsequently used to develop a system for automatic localization of the Veress needle tip by identifying tissue layers (or spaces) and estimating the needle-to-tissue distance. We used convolutional neural networks (CNNs) in automatic tissue classification and distance estimation. The average testing accuracy in tissue classification was 98.53 ± 0.39%, and the average testing relative error in distance estimation reached 4.42 ± 0.56% (36.09 ± 4.92 μm).

   @article{chen2022a,
     author = {Wang, Chen and Reynolds, Justin C. and Calle, Paul and Ladymon, Avery D. and Yan, Feng and Yan, Yuyang and Ton, Sam and Fung, Kar-ming and Patel, Sanjay G. and Yu, Zhongxin and Pan, Chongle and Tang, Qinggong},
     title = {Computer-aided Veress needle guidance using endoscopic optical coherence tomography and convolutional neural networks},
     journal = {Journal of Biophotonics},
     volume = {15},
     number = {5},
     pages = {e202100347},
     issn = {1864-063X},
     doi = {https://doi.org/10.1002/jbio.202100347},
     url = {https://onlinelibrary.wiley.com/doi/abs/10.1002/jbio.202100347},
     year = {2022}
   }

3. J. Biophotonics

   Inside Cover

   Chen Wang, Justin C. Reynolds, Paul Calle, and 9 more authors

   2022

   Abs

   A computer-aided endoscopic optical coherence tomography (OCT) device was developed to guide the Veress needle insertion into the abdominal cavity. Four tissue layers which Veress needle went through were successfully visualized. Convolutional neural network (CNN) was used to automatically recognize the tissues and estimate the distance between needle tip and small intestine. Promising prediction results (98.53±0.39% for tissue recognition accuracy and 4.42±0.56% for distance estimation relative error) were obtained. Further details can be found in the article by Chen Wang, Justin C. Reynolds, Paul Calle, Avery D. Ladymon, Feng Yan, Yuyang Yan, Sam Ton, Kar-ming Fung, Sanjay G. Patel, Zhongxin Yu, Chongle Pan, and Qinggong Tang (e202100347)

2021

1. BOE

   Deep-learning-aided forward optical coherence tomography endoscope for percutaneous nephrostomy guidance

   Chen Wang*, Paul Calle*, Nu Bao Tran Ton, and 8 more authors

   Biomedical Optics Express, 2021

   Abs Bib HTML Code

   Percutaneous renal access is the critical initial step in many medical settings. In order to obtain the best surgical outcome with minimum patient morbidity, an improved method for access to the renal calyx is needed. In our study, we built a forward-view optical coherence tomography (OCT) endoscopic system for percutaneous nephrostomy (PCN) guidance. Porcine kidneys were imaged in our experiment to demonstrate the feasibility of the imaging system. Three tissue types of porcine kidneys (renal cortex, medulla, and calyx) can be clearly distinguished due to the morphological and tissue differences from the OCT endoscopic images. To further improve the guidance efficacy and reduce the learning burden of the clinical doctors, a deep-learning-based computer aided diagnosis platform was developed to automatically classify the OCT images by the renal tissue types. Convolutional neural networks (CNN) were developed with labeled OCT images based on the ResNet34, MobileNetv2 and ResNet50 architectures. Nested cross-validation and testing was used to benchmark the classification performance with uncertainty quantification over 10 kidneys, which demonstrated robust performance over substantial biological variability among kidneys. ResNet50-based CNN models achieved an average classification accuracy of 82.6%±3.0%. The classification precisions were 79%±4% for cortex, 85%±6% for medulla, and 91%±5% for calyx and the classification recalls were 68%±11% for cortex, 91%±4% for medulla, and 89%±3% for calyx. Interpretation of the CNN predictions showed the discriminative characteristics in the OCT images of the three renal tissue types. The results validated the technical feasibility of using this novel imaging platform to automatically recognize the images of renal tissue structures ahead of the PCN needle in PCN surgery.

   @article{chen2021,
     author = {Wang*, Chen and Calle*, Paul and Tran Ton, Nu Bao and Zhang, Zuyuan and Yan, Feng and Donaldson, Anthony M. and Bradley, Nathan A. and Yu, Zhongxin and Fung, Kar-ming and Pan, Chongle and Tang, Qinggong},
     title = {Deep-learning-aided forward optical coherence tomography endoscope for percutaneous nephrostomy guidance},
     journal = {Biomedical Optics Express},
     volume = {12},
     number = {4},
     pages = {2404-2418},
     keywords = {Endoscopic imaging
       In vivo imaging
       Laser scanning
       Medical imaging
       Optical coherence tomography
       Three dimensional imaging},
     doi = {10.1364/BOE.421299},
     year = {2021},
   }