Journal List > J Korean Soc Radiol > v.80(2) > 1141882

TOOLS
Kim: Imaging Informatics: A New Horizon for Radiologyin the Era of Artificial Intelligence, Big Data, and Data Science
Review Article

J Korean Soc Radiol 2019;80(2):176-201.

Published online: 20 January 2019

DOI: https://doi.org/10.3348/jksr.2019.80.2.176

Imaging Informatics: A New Horizon for Radiologyin the Era of Artificial Intelligence, Big Data, and Data Science

Jong Hyo Kim, PhD1, 2, 3, ∗

1Department of Radiology, Seoul National University College of Medicine, Seoul, Korea

2Department of Radiology, Seoul National University Hospital, Seoul, Korea

3Department of Transdisciplinary Studies, Granduate School of Convergence Science and Technology, Seoul National University, Seoul, Korea

∗Corresponding author Jong Hyo Kim, PhD Department of Radiology, Seoul National University Hospital, 101 Daehak-ro, Jongno-gu, Seoul 03080, Korea. Tel 82-2-2072-3600 Fax 82-2-747-5781

Received 12 February 2019       Accepted 20 March 2019

Copyright © 2019 The Korean Society of Radiology

This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/4.0) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

We are witnessing the big wave of Industrial Revolution 4.0, enabled by artificial intelligence (AI) and big data, which has shaken the entire industry and day-to-day life as well as rapidly changed the landscapes of related academic disciplines. After the introduction of genome sequencing and analysis technology, biology and medical sciences have been rapidly transforming into data science. Radiology is facing a challenging period of transformation into a data science. This review article draws attention to imaging informatics as a vehicle to open a new horizon and to drive to the future path for radiology in the AI and big data era. We introduce the basic concepts of imaging informatics and consider the informatics features of picture archiving and communication system and digital imaging and communications in medicine. We discuss the conceptsand differences of radiogenomics and radiomics, which are important specialties of imaging informatics. We introduce the basics of AI and its recent applications in radiology as well as requirements for the successful construction of big data for imaging informatics. We conclude by discussing unresolved issues, potential solutions, and directions for future developments.

References

1. Wikipedia. Informatics. Available at.https://en.wikipedia.org/wiki/Informatics. Accessed Jan 22,. 2019.
2. Korean Society of imaging Informatics in Medicine. Greeting. Available at.https://ksiim.org/site/introduce/greeting/new. Accessed Jan 22,. 2019.
3. Park EJ, Kim SI. The guideline for the standardization of the PACS Components.J Kor PACS Soc. 1998; 4:91–99.
4. Park HJ, Kim JH, Kim SI. An introduction of DICOM 3.0.J Kor PACS Soc. 1998; 4:145–166.
5. Mazurowski MA. Radiogenomics: what it is and why it is important.J Am Coll Radiol. 2015; 12:862–866.
6. Segal E, Sirlin CB, Ooi C, Adler AS, Gollub J, Chen X, et al. Decoding global gene expression programs in liver cancer by noninvasive imaging.Nat Biotechnol. 2007; 25:675–680.
7. Woo BY, Lee ME, Kim JH. Repeatability of gene set enrichment analysis in radiogenomics.J Kor Soc Imag Inf or Med. 2016; 22:29–37.
8. Lee ME, Kim JH. Opportunities and challenges in radiogenomics: imaging phenotype analysis for brain tumor. J Kor Soc Imag Infor Med. 2014; 20:19–26.
9. Diehn M, Nardini C, Wang DS, McGovern S, Jayaraman M, Liang Y, et al. Identification of noninvasive imaging surrogates for brain tumor gene-expression modules. Proc Natl Acad Sci U S A. 2008; 105:5213–5218.
crossref
10. Zinn PO, Mahajan B, Sathyan P, Singh SK, Majumder S, Jolesz FA, et al. Radiogenomic mapping of edema/cellular invasion MRI-phenotypes in glioblastoma multiforme.PLoS One. 2011; 6:e25451.
11. Yamamoto S, Maki DD, Korn RL, Kuo MD. Radiogenomic analysis of breast cancer using MRI: a preliminary study to define the landscape. AJR Am J Roentgenol. 2012; 199:654–663.
crossref
12. Gevaert O, Xu J, Hoang CD, Leung AN, Xu Y, Quon A, et al. Non-small cell lung cancer: identifying prognostic imaging biomarkers by leveraging public gene expression microarray data–methods and preliminary results. Radiology. 2012; 264:387–396.
crossref
13. Karlo CA, Di Paolo PL, Chaim J, Hakimi AA, Ostrovnaya I, Russo P, et al. Radiogenomics of clear cell renal cell carcinoma: associations between CT imaging features and mutations.Radiology. 2014; 270:464–471.
14. Jamshidi N, Diehn M, Bredel M, Kuo MD. Illuminating radiogenomic characteristics of glioblastoma multi-forme through integration of MR imaging, messenger RNA expression, and DNA copy number variation.Radiology. 2014; 270:1–2.
15. Yamamoto S, Korn RL, Oklu R, Migdal C, Gotway MB, Weiss GJ, et al. ALK molecular phenotype in non-small cell lung cancer: CT radiogenomic characterization.Rad/iiology. 2014; 272:568–576.
16. Yamamoto S, Han W, Kim Y, Du L, Jamshidi N, Huang D, et al. Breast cancer: radiogenomic biomarker reveals associations among dynamic contrast-enhanced MR imaging, long noncoding RNA, and metastasis. Radiology. 2015; 275:384–392.
crossref
17. Cancer Imaging Archive. TCIA collections. Available at.http://www.cancerimagingarchive.net/. Accessed Jan 22,. 2019.
18. Gillies RJ, Kinahan PE, Hricak H. Radiomics: images are more than pictures, they are data.Radiology. 2016; 278:563–577.
19. Lee M, Woo B, Kuo MD, Jamshidi N, Kim JH. Quality of radiomic features in glioblastoma multiforme: impact of semiautomated tumor segmentation software. Korean J Radiol. 2017; 18:498–509.
crossref
20. Balagurunathan Y, Gu Y, Wang H, Kumar V, Grove O, Hawkins S, et al. Reproducibility and prognosis of quantitative features extracted from CT images. Transl Oncol. 2014; 7:72–87.
crossref
21. Kim H, Park CM, Lee SM, Lee HJ, Goo JM. A comparison of two commercial volumetry software programs in the analysis of pulmonary groundglass nodules: segmentation capability and measurement accuracy. Kor ean J Radiol. 2013; 14:683–691.
crossref
22. Ryoo I, Choi SH, Kim JH, Sohn CH, Kim SC, Shin HS, et al. Cerebral blood volume calculated by dynamic susceptibility contrast-enhanced perfusion MR imaging: preliminary correlation study with glioblastoma genetic profiles.PLoS One. 2013; 8:e71704.
23. Egger J, Kapur T, Fedorov A, Pieper S, Miller JV, Veeraraghavan H, et el. GBM volumetry using the 3D Slicer medical image computing platform. Sci Rep. 2013; 3:1364.
24. Zhu Y, Young GS, Xue Z, Huang RY, You H, Setayesh K, et al. Semi-automatic segmentation software for quantitative clinical brain glioblastoma evaluation.Acad Radiol. 2012; 19:977–985.
25. Kim JW, Kim JH. Review of evaluation metrics for 3D medical image segmentation.J Kor Soc Imag Infor Med. 2017; 23:14–20.
26. Lao J, Chen Y, Li ZC, Li Q, Zhang J, Liu J, et al. A deep learning-based radiomics model for prediction of survival in glioblastoma multiforme. Sci Rep. 2017; 7:10353.
crossref
27. Limkin EJ, Sun R, Dercle L, Zacharaki EI, Robert C, Reuzé S, et al. Promises and challenges for the implementation of computational medical imaging (radiomics) in oncology.Ann Oncol. 2017; 28:1191–1206.
28. Lambin P, Rios-Velazquez E, Leijenaar R, Carvalho S, Van Stiphout RG, Granton P, et al. Radiomics: extracting more information from medical images using advanced feature analysis.Eur J Cancer. 2012; 48:441–446.
29. Aerts HJ, Velazquez ER, Leijenaar RT, Parmar C, Grossmann P, Carvalho S, et al. Decoding tumour phenotype by noninvasive imaging using a quantitative radiomics approach.Nat Commun. 2014; 5:4006.
30. Coroller TP, Grossmann P, Hou Y, Rios Velazquez E, Leijenaar RT, Hermann G, et al. CT-based radiomic signature predicts distant metastasis in lung adenocarcinoma. Radiother Oncol. 2015; 114:345–350.
crossref
31. Hawkins S, Wang H, Liu Y, Garcia A, Stringfield O, Krewer H, et al. Predicting malignant nodules from screening CT scans.J Thorac Oncol. 2016; 11:2120–2128.
32. Wu W, Parmar C, Grossmann P, Quackenbush J, Lambin P, Bussink J, et al. Exploratory study to identify radiomics classifiers for lung cancer histology.Front Oncol. 2016; 6:71.
33. Kickingereder P, Burth S, Wick A, Götz M, Eidel O, Schlemmer HP, et al. Radiomic profiling of glioblastoma: identifying an imaging predictor of patient survival with improved performance over established clinical and radiologic risk models. Radiology. 2016; 280:880–889.
crossref
34. Huang YQ, Liang CH, He L, Tian J, Liang CS, Chen X, et al. Development and validation of a radiomics nomogram for preoperative prediction of lymph node metastasis in colorectal cancer.J Clin Oncol. 2016; 34:2157–2164.
35. Aerts HJ, Grossmann P, Tan Y, Oxnard GR, Rizvi N, Schwartz LH, et al. Defining a radiomic response phenotype: a pilot study using targeted therapy in NSCLC. Sci Rep. 2016; 6:33860.
crossref
36. Michoux N, Van den Broeck S, Lacoste L, Fellah L, Galant C, Berlière M, et al. Texture analysis on MR images helps predicting non-response to NAC in breast cancer.BMC Cancer. 2015; 15:574.
37. Nie K, Shi L, Chen Q, Hu X, Jabbour SK, Yue N, et al. Rectal cancer: assessment of neoadjuvant chemoradiation outcome based on radiomics of multiparametric MRI.Clin Cancer Res. 2016; 22:5256–5264.
38. Fehr D, Veeraraghavan H, Wibmer A, Gondo T, Matsumoto K, Vargas HA, et al. Automatic classification of prostate cancer Gleason scores from multiparametric magnetic resonance images. Proc Natl Acad Sci U S A. 2015; 112:E6265–E6273.
crossref
39. Li H, Zhu Y, Burnside ES, Drukker K, Hoadley KA, Fan C, et al. MR imaging radiomics signatures for predicting the risk of breast cancer recurrence as given by research versions of MammaPrint, Oncotype DX, and PAM50 gene assays. Radiology. 2016; 281:382–391.
crossref
40. Lee JG, Jun S, Cho YW, Lee H, Kim GB, Seo JB, et al. Deep learning in medical imaging: general overview. Korean J Radiol. 2017; 18:570–584.
crossref
41. Alom MZ, Taha TM, Yakopcic C, Westberg S, Sidike P, Nasrin MS, et al. The history began from alexnet: a comprehensive survey on deep learning approaches.arXiv preprint. 2018; arXiv:1803. .01164.
42. Sahiner B, Pezeshk A, Hadjiiski LM, Wang X, Drukker K, Cha KH, et al. Deep learning in medical imaging and radiation therapy.Med Phys. 2019; 46:e1–e36.
43. Hu P, Wu F, Peng J, Bao Y, Chen F, Kong D. Automatic abdominal multiorgan segmentation using deep convolutional neural network and time-implicit level sets.Int J Comput Assist Radiol Surg. 2017; 12:399–411.
44. Dolz J, Desrosiers C, Ben Ayed I. 3D fully convolutional networks for subcortical segmentation in MRI: A large-scale study. NeuroImage. 2018; 170:456–470.
crossref
45. Cheng R, Roth HR, Lay N, Lu L, Turkbey B, Gandler W, et al. Automatic magnetic resonance prostate segmentation by deep learning with holistically nested networks.J Med Imaging (Bellingham). 2017; 4:041302.
46. Wang S, Zhou M, Liu Z, Liu Z, Gu D, Zang Y, et al. Central focused convolutional neural networks: developing a data-driven model for lung nodule segmentation.Med Image Anal. 2017; 40:172–183.
47. Alex V, Vaidhya K, Thirunavukkarasu S, Kesavadas C, Krishnamurthi G. Semisupervised learning using denoising autoencoders for brain lesion detection and segmentation.J Med Imag/iing (Bellingham). 2017; 4:041311.
48. Zhang R, Huang L, Xia W, Zhang B, Qiu B, Gao X. Multiple supervised residual network for osteosarcoma segmentation in CT images. Comput Med Imaging Graph. 2018; 63:1–8. ˇ.
49. Payer C, S tern D, Bischof H, Urschler M. Regressing heatmaps for multiple landmark localization using CNNs. In International Conference on Medical Image Computing and Computer-Assisted Intervention. Cham:. Springer;2006. p. 230–238.
50. Zhang J, Liu M, Shen D. Detecting anatomical landmarks from limited medical imaging data using two-stage task-oriented deep neural networks.IEEE Trans Image Process. 2017; 26:4753–4764.
51. Harrison AP, Xu Z, George K, Lu L, Summers RM, Mollura DJ.Progressive and multi-path holistically nested neural networks for pathological lung segmentation from CT images. In International Conference on Medi-cal Image Computing and Computer-Assisted Intervention. Cham:. Springer;2006. p. 621–629.
52. Rajpurkar P, Irvin J, Ball RL, Zhu K, Yang B, Mehta H, et al. Deep learning for chest radiograph diagnosis: a retrospective comparison of the CheXNeXt algorithm to practicing radiologists.PLoS Med. 2018; 15:e1002686.
53. Nam JG, Park S, Hwang EJ, Lee JH, Jin KN, Lim KY, et al. Development and validation of deep learning-based automatic detection algorithm for malignant pulmonary nodules on chest radiographs.Radiology. 2019; 290:218–228.
54. Wang Y, Zhou Y, Tang P, Shen W, Fishman EK, Yuille AL. Training multiorgan segmentation networks with sample selection by relaxed upper confident bound. arXiv preprint. 2018; arXiv:1804. .02595.
crossref
55. Hosny A, Parmar C, Coroller TP, Grossmann P, Zeleznik R, Kumar A, et al. Deep learning for lung cancer prognostication: a retrospective multicohort radiomics study.PLoS Med. 2018; 15:e1002711.
56. Wang H, Xu Z, Fujita H, Liu S. Towards felicitous decision making: an overview on challenges and trends of Big Data. Inf Sci. 2016; 367–368:747–765.
crossref
57. Jain A. Healthcare data analytics: the 5 Vs of big data. Available at.https://www.ibm.com/blogs/watson-health/the-5-vs-of-big-data/. Accessed Feb 10,. 2019.
58. Neugebauer R. Trends in industrie 4.0. Avaiable at.http://www.fraunhofer.jp/content/dam/japan/ja/docu-ments/media/publication/Trends-in-Industrie-40.pdf. Accessed Feb 10,. 2019.
59. Donoho D. High-dimensional data analysis: the curses and blessings of dimensionality in Mathematical Challenges of the 21st Century. 2000. Available at:. http://statweb.stanford.edu/∼donoho/Lectures/AMS2000/Curses.pdf. Accessed Feb 10,. 2019.
60. Daniel N. 5 fixes for your failing big data initiatives. Available at.https://www.forbes.com/sites/danielnewman/2018/06/28/5-fixes-for-your-failing-big-data-initiatives/#57c08a845a0b. Accessed Feb 10,. 2019.
61. Morris MA, Saboury B, Burkett B, Gao J, Siegel EL. Reinventing radiology: big data and the future of medical imaging.J Thorac Imaging. 2018; 33:4–16.
62. ICD10Data. ICD-10. Available at.https://www.icd10data.com/ICD10CM/Codes. Accessed Feb 10,. 2019.
63. Wikipedia. SNOMED CT.kr. Available at.https://en.wikipedia.org/wiki/SNOMED_CT. Accessed Feb 10,. 2019.
64. RSNA Informatics. RadLex. Available at.http://www.radlex.org/. Accessed Feb 10,. 2019.
65. Jamshidi N, Jonasch E, Zapala M, Korn RL, Aganovic L, Zhao H, et al. The radiogenomic risk score: construction of a prognostic quantitative, noninvasive image-based molecular assay for renal cell carcinoma.Radiol-ogy. 2015; 277:114–123.
66. Traverso A, Wee L, Dekker A, Gillies R. Repeatability and reproducibility of radiomic features: a systematic review. Int J Radiat Oncol Biol Phys. 2018; 102:1143–1158.
crossref
67. Wikipedia. Explainable artificial intelligence. Available at.https://en.wikipedia.org/wiki/Explainable_Artifi-cial_Intelligence. Accessed Jan 22,. 2019.
68. Abajian AC, Levy M, Rubin DL. Informatics in radiology: improving clinical work flow through an AIM database: a sample web-based lesion tracking application. Radiographics. 2012; 32:1543–1552.
crossref

Fig. 1.
The figure illustrates a typical analytics procedure in radiogenomics research.
jksr-80-176f1.tif
Fig. 2.
The figure shows an example of analysis procedure in radiomics research. Adapted from Lao et al. Sci Rep 2017;7:10353 (26).
jksr-80-176f2.tif
Fig. 3.
The figure shows examples of segmentation methods. (A) manual, (B) point click, (C) box draw, and (D) sketch draw. Adapted from Lee et al. J Kor Soc Imag Infor Med 2014;20:19–26 (8).
jksr-80-176f3.tif
Fig. 4.
The diagram shows an example of normalized dynamic range for features extracted from segmented tumor volumes. Adapted from Lee et al. Korean J Radiol 2017;18:498–509 (19).
jksr-80-176f4.tif
Fig. 5.
The figure shows an example of feature clustergram derived from feature cross correlation matrix. Adapt-ed from Lee et al. Korean J Radiol 2017; 18:498–509 (19).
jksr-80-176f5.tif
Fig. 6.
The figure illustrates basic building blocks and architectures in deep learning (A–F). Adapted from Alom et al. arXiv preprint 2018 arXiv:1803.01164 (41).
jksr-80-176f6.tif
Fig. 7.
The figure shows examples of deep learning applications. A. Segmentation of multiple abdominal organs in abdominal CT. Adapted from Wang et al. arXiv preprint 2018;arXiv:1804.02595 (54). B. Prognosis map of pulmonary nodules in chest CT. Adapted from Hosny et al. PLoS Med 2018;15:e1002711 (55).
jksr-80-176f7.tif
Fig. 8.
The figure shows an example list of curation document for the National Lung Screening Trial dataset. Patient, exam, lesion levels, curation dictionary, and data are provided.
jksr-80-176f8.tif
Table 1.
Examples of Imaging Phenotype Traits for Radiogenomic Analysis
Trait Name Description Scoring System
Location Location of most tumor mass Central = medial to midclavicular line 1 = central, 2 = peripheral
Pleural tail Pleural tail, defined as linear opacity extending from the mass to the pleura 1 = absent, 2 = present
Pleural effusion Severity of pleural effusion 0 = none, 1 = small, 2 = moderate, 3 = large
Dominant opacity of mass Characteristic of dominant opacity 1 = solid, 2 = ground glass, 3 = mixture, 4 = cavitary
Margin shape Mass margin shape 1 = circumscribed, 2 = lobulated, 3 = speculated, 4 = irregular
Overall shape Overall shape of mass 1 = round, 2 = oval, 3 = notched, 4 = irregular
Calcification Presence of calcification 1 = absent, 2 = present
Necrosis Percentage of necrosis within tumor 0 = none, 1 ≤ 25%, 2 ≤ 50%, 3 ≤ 75%, 4 = 100%
Tumor tissue interface Characteristic of tumor tissue interphase 1 = absent, 2 = present
Tumor capsule Presence of tumor capsule 1 = absent, 2 = present
Halo Presence of halo around mass 1 = absent, 2 = present
Table 2.
Selected Papers Published at an Early Stage of Radiogenomics Development
References Title Modality Feature Type ROI Definition Subjects (n) Features (n)
Segal et al. 2007 (6) Decoding global gene expression programs in liver cancer by noninvasive imaging CT Semantic N/A 28 138
Diehn et al. 2008 (9) Identification of noninvasive imaging surrogates for brain tumor gene-expression modules MRI Semantic N/A 25 10
Zinn et al. 2011 (10) Radiogenomic mapping of edema/ cellular invasion MRI-phenotypes in GBM MRI Computational features Manual 78 3
Yamamoto et al. 2012 (11) Radiogenomic analysis of breast cancer using MRI MRI Semantic + quantitative Manual 10 26
Gevaert et al. 2012 (12) NSCLC: identifying prognostic imaging biomarkers by leveraging public gene expression microarray data CT Semantic + computational Manual 26 180
Karlo et al. 2014 (13) Radiogenomics of clear cell renal cell carcinoma:associations between CT imaging features and mutations CT Semantic + quantitative Manual 223 8
Jamshidi et al. 2014 (14) Illuminating radiogenomic characteristics of GBM MRI Semantic N/A 23 6
Yamamoto et al. 2014 (15) ALK molecular phenotype in NSCLC: CT radio-genomic characterization CT Semantic N/A 172 24
Yamamoto et al. 2015 (16) Breast cancer: radiogenomic biomarker reveals associations among dynamic contrast-enhanced MR imaging, long noncoding RNA, and metastasis MRI Computational Auto 61 47

GBM = glioblastoma multiforme, N/A = not available, NSCLC = non-small cell lung cancer, ROI = region of interest

Table 3.
Reliability of Segmentation Methods in Tumor Segmentation
References Title Modality Methods Result (%)
Balaqurunathan et al. 2014 (20) Reproducibility and prognosis of quantitative features extracted from CT Images CT Manual vs. point click Similarity: 78
Kim et al. 2013 (21) A comparison of two commercial volumetry software programs in the analysis of pulmonary ground-glass nodules CT Manual vs. point click Failure: 10 Error: 11–28
Ryoo et al. 2013 (22) Cerebral blood volume analysis in glioblastoma using dynamic susceptibility contrast-enhanced perfusion MRI MRI Manual vs. box draw Variability: 16–37
2013 (22) Egger et al. 2013 (23) susceptibility contrast-enhanced perfusion MRI GBM volumetry using the 3D slicer medical image computing platform MRI Manual vs. grow-cut Rater similarity: 88 Error: 12–36
Zhu et al. 2012 (24) Semi-automatic segmentation software for quantitative clinical brain glioblastoma evaluation MRI Manual vs. semiauto Error: 10–35

GBM = glioblastoma multiforme

Table 4.
Selected Studies on Radiomics Research in Diagnosis, Tumor Staging and Prognosis, and Treatment Response Prediction
References Tumor Type Modality Training/ Validation Sets Features Qualification and Model Learning Significance
Diagnosis
Hawkins et al. 2016 (31) Lung cancer (NLST) CT T: 312 V: 188 219 3-D features (size, shape, location, and textures) CNN, random forest Malignancy prediction AUC 79%
Wu et al. 2016 (32) Lung cancer CE CT T: 198 V: 152 textures) 440 features (intensity, shape, texture) Correlation-based feature elimination and univariate feature selection. 3 classification Histology prediction AUC 0.72
Fehr et al. 2015 (38) Prostate cancer MRI T: 147 1st and 2nd order statistics Oversampling approach, support vector machine Gleason score prediction accuracy 93%
Tumor staging and prognosis
Aerts et al. 2014 (29) NSCLC and HNSCC CE CT T: 31, 21, 422 V: 225, 136, 95, 89 440 features (intensity, shape, texture, wavelet) Stability testing, unsupervised clustering; Friedman test Overall survival CI 0.65 (NSCLC), CI 0.69 (HNSCC)
Kickingereder et al. 2016 (33) GBM CE MRI T: 112 V: 60 4842 total 17 1st order features, 9 volume and shape features, 162 texture features Supervised principal component analysis, Cox proportional hazard models, Integrated Brier scores Prediction of treatment outcome to antiangiogenic therapy PFS (p < 0.03) and OS (p < 0.001)
Huang et al. 2016 (34) Colorectal cancerCE CT T: 326 V: 200 150 texture features, 24 signature LASSO; Multi-variate binary logistic regression, nomograms and calibration plots Predicts lymph node metastases (CI = 0.78),
Li et al. 2016 (39) Breast Cancer (TCIA) CE MRI T: 84 38 features (morphology, enhancement texture, kinetic curve features) and calibration plots Leave one-case-out cross-validation analysis with logistic regression Predict the recurrence risk as assessed by oncotype Dx, or mammaprint (AUCs 0.55–0.88)
features) Treatment response prediction
Treatment respon Aerts et al. 2016 (35) nse prediction Lung adenocarcinomas HR CT T: 47 V: 31 183 initial features, 11 independent features Spearman rank statistic, intraclass correlation coefficient Predict response to gefitinib at baseline (AUC 0.67) and change in pre-and post treatment (AUC 0.74–0.91)
Michoux et al. 2015 (36) Breast cancer CE MRI T: 69 20 texture, 3 kinetic, BI-RADS and biologic parameters Logistic regression model, k-means clustering, leave-one-out cross validation treatment (AUC 0.74–0.91) Predict response to neoadjuvant chemotherapy (accuracy 68%),
Nie et al. 2016 (37) Rectal cancer CE MRI T: 48 103 features (texture, shape, histogram) out cross validation Artificial neural network Reflect response to neoadjuvant therapy (AUC 0.71–0.79)

AUC = area under the curve, BI-RADS = Breast Imaging-Reporting and Data System, CE = contrast enhancement, CI = confidence interval, CNN = convolutional neural network, GBM = glioblastoma multiforme, HNSCC = head and neck squamous cell carcinoma, HR = hazard ratio, LASSO = Least Absolute Shrinkage and Selection Operator, NLST = National Lung Screening Trial, NSCLC = non-small cell lung cancer, OS = overall survival, PFS = progression-free survival, T = training set, TCIA = The Cancer Imaging Archive, V = validation set

Table 5.
Selected Studies for Deep Learning Applications in Medical Imaging
References Object Modality Training/Validation Sets Network Architecture Task Performance
Hu et al. 2017 (43) Liver, spleen, kidneys CT 140 scans 5-fold CV Custom CNN Organ segmentation Dice: 0.94–0.96
Dolz et al. 2018 (44) Brain substructures MRI T: 150 V: 947 patients FCN Organ segmentation Dice: 0.86–0.92
Cheng et al. 2017 (45) Prostate MRI 250 patients 5-fold CV HNN Organ segmentation Dice: 0.90
Wang et al. 2017 (46) Lung nodule CT T: 350 V: 493 nodules Custom CNN Lesion segmentation Dice: 0.82
Alex et al. 2017 (47) Brain tumor MRI HGG: 150/69 patients, LGG: 20/23 patients AE Lesion segmentation Dice HGG: 0.86 LGG: 0.82
Zhang et al. 2018 (48) Osteosarcoma CT T: 15 V: 8 patients ResNet-50 Lesion segmentation Dice: 0.89
Payer et al. 2016 (49) 37 hand landmarks X-ray 895 images 3-fold CV Custom CNN Landmark localization Error: 1.19 ± 1.14 mm
Zhang et al. 2017 (50) Brain landmarks MRI T: 350 V: 350 FCN Landmark localization Error: 2.94 ± 1.58 mm
Harrison et al. 2017 (51) Pathologic lung CT 929 scans 5-fold CV FCN Landmark localization Error: 0.76 ± 0.53 mm
Rajpurkar et al. 2018 (52) 14 pulmonary pathologies X-ray T: 112, 120 V: 420 images CNN Multiple pathology detection AUC: 0.70–0.92
Nam et al. 2019 (53) Pulmonary nodule X-ray 43292 images CNN Lesion detection AUC: 0.92–0.99

AE = auto-encoder, AUC = area under the curve, CNN = convolutional neural network, CV = cross validation, FCN = fully convolutional network, HGG = high grade glioma, LGG = low grade glioma, T = training set, V = validation set

Table 6.
TCIA Lung Cancer Imaging Data Sets
Name Concentration Modalities Patients Studies Series Images Comments
LIDC-IDRI Lesion/nodules in diagnostic and screening studies CT, DX, CR 1010 1308 1018 CT, 290 CR/DX 244527 Lesions are marked on the images after 2-phase image annotation process by 4 thoracic radiologists
LUNGCT-Diagnosis Lung adenocarcinoma CT 61 61 61 4682 Images obtained at initial diagnosis, along with survival data and TNM stage
NLST (limited access) Lung cancer screening CT 26254 73118 203099 21082502 Images from the NLST. CT images and lung biopsy pathology images are available
NRG-1308 (limited access) NSLC (stage II-IIIB, inoperable) CT, RTSTRUCT, RTPLAN,RTDOSE 12 12 85 1983 Images from trials to compare overall survival after photon vs. proton chemoradiation therapy for inoperable stage II-IIIB NSCLC
NSCLC radiogenomics NSLC (operable) PET, CT 26 52 128 36593 Images and gene expression microarray data of the tumor tissue samples
NSCLC-radiomics NSCLC CT, RTSTRUCT 422 422 740 51513 Pretreatment images, manually defined tumor volume, and clinical outcome (Nature Communications Lung1 Data Set)
NSCLC-radiomics genomics NSLC (operable) CT 89 89 89 13482 Set) Pretreatment images, gene expression data, and clinical information (Nature Communications Lung 3 data Set)
QIN LUNG CT NSCLC (mixed stage/ histology) CT 10 10 10 1174 Pretreatment images of mixed stage/histology NSCLC confirmed after treatment
RIDER lung CT NSCLC (tumor measurement variability) CT 32 46 63 15419 Same-day repeat CT scan images and lesion measurement data for NSCLC
RIDER lung PET-CT Lung cancer and synthetic reference model PET-CT CT, PET 244 275 1349 269511 Serial PET-CT imaging of lung cancer patients and a reference synthetic lung to demonstrate systemic variance
SPIE-AAPM lung CT challenge Lung nodule classification CT 70 70 70 22489 CT lung nodule images—training set and test set for SPIE Medical Imaging Conference Challenge
TCGA-LUAD Lung adenocarcinoma CT, PET, NM 69 152 624 48931 Images, clinical data, pathology, and genomic data. Images were obtained as part of routine care and are not acquired in a standardized manner
TCGA-LUSC Lung squamous cell carcinoma CT, PET, NM 37 74 279 36518 Images, clinical data, pathology, and genomic data. Images were obtained as part of routine care and are not acquired in a standardized manner

TCIA Lung Cancer Imaging Data sets: Descriptive features of all TCIA lung cancer image databases. Access is open to all databases except the NLST and NRG-1308 wherein access must be requested and permission granted. All imaging datafiles are DICOM format (62). PET indicates positron emission tomography. AAPM = American Association of Physicists in Medicine, IDRI = Image Database Resource Initiative, LIDC = The Lung Image Database Consortium, LUAD = Lung Adenocarcinoma, LUSC = Lung Squamous Cell Carcinoma, NLST = National Lung Screening Trial, QIN = Quantitative Imaging Network, RIDER = Reference Imaging Database to Evaluate Response, RTDOSE = RT Dose, RTPLAN = RT Plan, RTSTRUCT = radiotherapy structure set, SPIE = Society of Photo-optical Instrumentation Engineers, TCGA = The Cancer Genome Atlas, TCIA = The Cancer Imaging Archive

TOOLS
Similar articles