We are witnessing the big wave of Industrial Revolution 4.0, enabled by artificial intelligence (AI) and big data, which has shaken the entire industry and day-to-day life as well as rapidly changed the landscapes of related academic disciplines. After the introduction of genome sequencing and analysis technology, biology and medical sciences have been rapidly transforming into data science. Radiology is facing a challenging period of transformation into a data science. This review article draws attention to imaging informatics as a vehicle to open a new horizon and to drive to the future path for radiology in the AI and big data era. We introduce the basic concepts of imaging informatics and consider the informatics features of picture archiving and communication system and digital imaging and communications in medicine. We discuss the conceptsand differences of radiogenomics and radiomics, which are important specialties of imaging informatics. We introduce the basics of AI and its recent applications in radiology as well as requirements for the successful construction of big data for imaging informatics. We conclude by discussing unresolved issues, potential solutions, and directions for future developments.
The figure illustrates a typical analytics procedure in radiogenomics research.
The figure shows an example of analysis procedure in radiomics research. Adapted from Lao et al. Sci Rep 2017;7:10353 (26).
The figure shows examples of segmentation methods. (A) manual, (B) point click, (C) box draw, and (D) sketch draw. Adapted from Lee et al. J Kor Soc Imag Infor Med 2014;20:19–26 (8).
The diagram shows an example of normalized dynamic range for features extracted from segmented tumor volumes. Adapted from Lee et al. Korean J Radiol 2017;18:498–509 (19).
The figure shows an example of feature clustergram derived from feature cross correlation matrix. Adapt-ed from Lee et al. Korean J Radiol 2017; 18:498–509 (19).
The figure illustrates basic building blocks and architectures in deep learning (A–F). Adapted from Alom et al. arXiv preprint 2018 arXiv:1803.01164 (41).
The figure shows examples of deep learning applications. A. Segmentation of multiple abdominal organs in abdominal CT. Adapted from Wang et al. arXiv preprint 2018;arXiv:1804.02595 (54). B. Prognosis map of pulmonary nodules in chest CT. Adapted from Hosny et al. PLoS Med 2018;15:e1002711 (55).
The figure shows an example list of curation document for the National Lung Screening Trial dataset. Patient, exam, lesion levels, curation dictionary, and data are provided.
Examples of Imaging Phenotype Traits for Radiogenomic Analysis
Trait Name | Description | Scoring System |
---|---|---|
Location | Location of most tumor mass Central = medial to midclavicular line | 1 = central, 2 = peripheral |
Pleural tail | Pleural tail, defined as linear opacity extending from the mass to the pleura | 1 = absent, 2 = present |
Pleural effusion | Severity of pleural effusion | 0 = none, 1 = small, 2 = moderate, 3 = large |
Dominant opacity of mass | Characteristic of dominant opacity | 1 = solid, 2 = ground glass, 3 = mixture, 4 = cavitary |
Margin shape | Mass margin shape | 1 = circumscribed, 2 = lobulated, 3 = speculated, 4 = irregular |
Overall shape | Overall shape of mass | 1 = round, 2 = oval, 3 = notched, 4 = irregular |
Calcification | Presence of calcification | 1 = absent, 2 = present |
Necrosis | Percentage of necrosis within tumor | 0 = none, 1 ≤ 25%, 2 ≤ 50%, 3 ≤ 75%, 4 = 100% |
Tumor tissue interface | Characteristic of tumor tissue interphase | 1 = absent, 2 = present |
Tumor capsule | Presence of tumor capsule | 1 = absent, 2 = present |
Halo | Presence of halo around mass | 1 = absent, 2 = present |
Selected Papers Published at an Early Stage of Radiogenomics Development
References | Title | Modality | Feature Type | ROI Definition | Subjects ( |
Features ( |
---|---|---|---|---|---|---|
Segal et al. 2007 (6) | Decoding global gene expression programs in liver cancer by noninvasive imaging | CT | Semantic | N/A | 28 | 138 |
Diehn et al. 2008 (9) | Identification of noninvasive imaging surrogates for brain tumor gene-expression modules | MRI | Semantic | N/A | 25 | 10 |
Zinn et al. 2011 (10) | Radiogenomic mapping of edema/ cellular invasion MRI-phenotypes in GBM | MRI | Computational features | Manual | 78 | 3 |
Yamamoto et al. 2012 (11) | Radiogenomic analysis of breast cancer using MRI | MRI | Semantic + quantitative | Manual | 10 | 26 |
Gevaert et al. 2012 (12) | NSCLC: identifying prognostic imaging biomarkers by leveraging public gene expression microarray data | CT | Semantic + computational | Manual | 26 | 180 |
Karlo et al. 2014 (13) | Radiogenomics of clear cell renal cell carcinoma:associations between CT imaging features and mutations | CT | Semantic + quantitative | Manual | 223 | 8 |
Jamshidi et al. 2014 (14) | Illuminating radiogenomic characteristics of GBM | MRI | Semantic | N/A | 23 | 6 |
Yamamoto et al. 2014 (15) | ALK molecular phenotype in NSCLC: CT radio-genomic characterization | CT | Semantic | N/A | 172 | 24 |
Yamamoto et al. 2015 (16) | Breast cancer: radiogenomic biomarker reveals associations among dynamic contrast-enhanced MR imaging, long noncoding RNA, and metastasis | MRI | Computational | Auto | 61 | 47 |
GBM = glioblastoma multiforme, N/A = not available, NSCLC = non-small cell lung cancer, ROI = region of interest
Reliability of Segmentation Methods in Tumor Segmentation
References | Title | Modality | Methods | Result (%) |
---|---|---|---|---|
Selected Studies on Radiomics Research in Diagnosis, Tumor Staging and Prognosis, and Treatment Response Prediction
References | Tumor Type Modality | Training/ Validation Sets | Features | Qualification and Model Learning | Significance |
---|---|---|---|---|---|
Diagnosis | |||||
Hawkins et al. 2016 (31) | Lung cancer (NLST) CT | T: 312 V: 188 | 219 3-D features (size, shape, location, and textures) | CNN, random forest | Malignancy prediction AUC 79% |
Wu et al. 2016 (32) | Lung cancer CE CT | T: 198 V: 152 | textures) 440 features (intensity, shape, texture) | Correlation-based feature elimination and univariate feature selection. 3 classification | Histology prediction AUC 0.72 |
Fehr et al. 2015 (38) | Prostate cancer MRI | T: 147 | 1st and 2nd order statistics | Oversampling approach, support vector machine | Gleason score prediction accuracy 93% |
Tumor staging and prognosis | |||||
Aerts et al. 2014 (29) | NSCLC and HNSCC CE CT | T: 31, 21, 422 V: 225, 136, 95, 89 | 440 features (intensity, shape, texture, wavelet) | Stability testing, unsupervised clustering; Friedman test | Overall survival CI 0.65 (NSCLC), CI 0.69 (HNSCC) |
Kickingereder et al. 2016 (33) | GBM CE MRI | T: 112 V: 60 | 4842 total 17 1st order features, 9 volume and shape features, 162 texture features | Supervised principal component analysis, Cox proportional hazard models, Integrated Brier scores | Prediction of treatment outcome to antiangiogenic therapy PFS ( |
Huang et al. 2016 (34) | Colorectal cancerCE CT | T: 326 V: 200 | 150 texture features, 24 signature | LASSO; Multi-variate binary logistic regression, nomograms and calibration plots | Predicts lymph node metastases (CI = 0.78), |
Li et al. 2016 (39) | Breast Cancer (TCIA) CE MRI | T: 84 | 38 features (morphology, enhancement texture, kinetic curve features) | and calibration plots Leave one-case-out cross-validation analysis with logistic regression | Predict the recurrence risk as assessed by oncotype Dx, or mammaprint (AUCs 0.55–0.88) |
features) Treatment response prediction | |||||
Treatment respon Aerts et al. 2016 (35) | nse prediction Lung adenocarcinomas HR CT | T: 47 V: 31 | 183 initial features, 11 independent features | Spearman rank statistic, intraclass correlation coefficient | Predict response to gefitinib at baseline (AUC 0.67) and change in pre-and post treatment (AUC 0.74–0.91) |
Michoux et al. 2015 (36) | Breast cancer CE MRI | T: 69 | 20 texture, 3 kinetic, BI-RADS and biologic parameters | Logistic regression model, k-means clustering, leave-one-out cross validation | treatment (AUC 0.74–0.91) Predict response to neoadjuvant chemotherapy (accuracy 68%), |
Nie et al. 2016 (37) | Rectal cancer CE MRI | T: 48 | 103 features (texture, shape, histogram) | out cross validation Artificial neural network | Reflect response to neoadjuvant therapy (AUC 0.71–0.79) |
AUC = area under the curve, BI-RADS = Breast Imaging-Reporting and Data System, CE = contrast enhancement, CI = confidence interval, CNN = convolutional neural network, GBM = glioblastoma multiforme, HNSCC = head and neck squamous cell carcinoma, HR = hazard ratio, LASSO = Least Absolute Shrinkage and Selection Operator, NLST = National Lung Screening Trial, NSCLC = non-small cell lung cancer, OS = overall survival, PFS = progression-free survival, T = training set, TCIA = The Cancer Imaging Archive, V = validation set
Selected Studies for Deep Learning Applications in Medical Imaging
References | Object Modality | Training/Validation Sets | Network Architecture | Task | Performance |
---|---|---|---|---|---|
TCIA Lung Cancer Imaging Data Sets
Name | Concentration | Modalities | Patients | Studies | Series | Images | Comments |
---|---|---|---|---|---|---|---|