Decision Support Algorithms in Real Action

Academic institutions proceed in developing machine algorithms for clinical decision support. The ongoing projects range from analyzing X-rays to measuring pediatric brain development. The collaborators emphasize the importance of good training data.  

According to Bhavik N. Patel, Stanford University Medical Center, Artificial Intelligence (AI) could be quickly implemented to simplify disease classification and workflow triage. “Large volume and correctly labeled data are the keys for well-working algorithms,” he said. Luckily, the Center of Artificial Intelligence in Medical Imaging (AIMI) at Stanford University is situated in close proximity to a hospital with as many as 1.5 million labeled imaging studies. Patel listed a number of ongoing collaborative projects.

Drawing near an automated workflow triage

The task was as simple as that: to differentiate between “normal” and “abnormal” lower extremity X-rays. Setting up an automated and reliable triage system on those would make the clinical workflow more effective.  

Pre-training the algorithm: Patel and colleagues tested three different convolutional neuronal networks (CNNs) for their performance, namely ResNet50, ResNet101 and DenseNet161. They used ImageNet und MURA for pre-training the CNNs. ImageNet is a huge source for non-medical images. The MURA dataset is a large dataset of musculoskeletal radiographs containing 40,895 images from 14,982 studies, where each study is manually labeled as either normal or abnormal. Patel subsequently compared the performance of the pre-trained CNNs among each other and with untrained CNNs. 

More than enough training data: Patel disposed of 93,500 X-ray labeled images of knees, hips, feet and ankles for further training of the CNNs. When feeding 20,000 of these images into the untrained and pre-trained networks, he observed a boost in performance from untrained to pre-trained CNNs. However, when expanding the set to 90,000 images, no pre-trained effect on the performance could be seen. “It seems that once you have enough training data, it does not really impact the result,” Patel concluded. Of the three trained CNNs, DenseNet161 turned out to deliver the best results.

More to come

Patel also started AI models for chest X-ray pathologies and detection of acute appendicitis. Another project is the prediction of the calcium score on coronary CT. “This works nearly perfect,” Patel said. Moreover, AI delivered results after 16 seconds, while the manual evaluation lasted four hours. 

Kristen Yeom, a colleague of Patel at Stanford University, added some promising AI projects in neuroradiology: 

1. Triage for cervical spine – identify acute findings
2. Predict fetal brain age
3. Identify abnormal brain development in babies and children
4. Automated ventricular segmentation of an MRI
5. Classification of aneurysms


Patel and Yeom are sure that research projects with AI will solve clinically important questions in the near future. They have started and tested several programs, which they aim to expand to a wider audience. “We need more well-labeled training data,” Patel said. Yeom emphasized that the AI models need to be working outside of the own institution. “Make sure that your model works in other centers,” Yeom added. Both recommend visiting the website of AIMI to get an overview on ongoing projects

Presentation Title: The Reality: Current Application of Machine Learning and Artificial Intelligence in Clinical Radiology and Research
Speaker: Bhavik N. Patel, Kristen Yeom, Stanford University Medical Center
Date: 2018-11-26
Session code: SPSI24B