Ideal 3D images for CHD have an isotropic high spatial resolution, sharp contrast between blood and myocardium, homogeneous signal distribution across all cardiovascular structures, low image noise, and minimal artifacts. Therefore, obtaining all these features is the gold standard for 3D data acquisition in CHD. These features make all post-processing steps and advanced visualization techniques described in the following sections more accurate and less time-consuming.

CT is increasingly used for evaluating CHD because the imaging modality is exceedingly fast and can provide excellent image quality (56). In addition, the CT radiation dose can be minimized by combining dose reduction strategies while maintaining image quality (78). Through electrocardiography (ECG)-synchronized CT scanning, crisp cardiovascular images can be obtained without meaningful cardiac pulsation artifacts. Further, iterative reconstruction algorithms can be used to reduce image noise and may eliminate the need for additional smoothing processes. Moreover, model-based iterative reconstruction algorithms may increase image sharpness. Optimization of an intravenous injection protocol is crucial for achieving uniform cardiovascular enhancement without substantial perivenous streak artifacts. A user-friendly, vendor-specific pediatric cardiothoracic CT guideline was recently published and can be utilized to optimize the CT protocols of clinicians and institutions (9).

MRI does not use ionizing radiation, but comprehensive examination is limited by the lengthy scanning times. A detailed cardiovascular MRI guideline for CHD was published in 2013 (10). ‘Non-ECG-synchronized contrast-enhanced magnetic resonance angiography’ and ‘ECG-triggered navigator-gated T2-prepared balanced steady-state free precession imaging’ are two 3D MRI methods with almost isotropic spatial resolution used for evaluating CHD. However, the accurate evaluation of intracardiac structures, coronary arteries, and proximal great arteries is noticeably impeded by cardiac pulsation artifacts in the former, and the latter is also suboptimal in image quality due to different signal intensities, lower spatial resolution, and more artifacts (Fig. 1). Intravenous administration of gadofosveset trisodium (11) and ferumoxytol (12) was used to improve the image quality of cardiac MRI. However, these two MRI contrast agents were recently withdrawn due to poor sales and potential safety issues.

A variety of useful volumetric segmentation tools, such as threshold-based segmentation and 3D region-growing algorithms, are available for post-processing the 3D cardiac images of CHD. Data in 3D are often presented as a single set of volume-rendered images. Multiple sets of different volumes may be color-coded, and spatial relationships with adjacent structures, including the airways and the thoracic cage, can be illustrated (Fig. 2). Furthermore, 3D threshold-based segmentation is increasingly used for volume measurements of cardiac chambers for evaluating cardiac function in patients with CHD (131415) (Fig. 3). A new cinematic rendering technique was recently introduced for 3D photorealistic visualization of complex cardiovascular structures by enhancing depth perception (16). Cinematic rendering may also be used to illustrate intracardiac structures by adjusting the opacity to make enhanced blood in cardiac chambers completely transparent (Fig. 4). A combined segmentation approach with deep-learning algorithms may accelerate cardiovascular post-processing (17).

In augmented reality, virtual objects are overlaid on the real-world with spatial registration. In virtual reality, a user is immersed in an entirely artificial digital environment. Thus, augmented reality is clearly distinguished from virtual reality in terms of the presence of the ability to still visualize the real-world. On the other hand, the concept of mixed reality, defined as anywhere in the so-called “reality-virtual continuum,” ranging from the completely real to a completely virtual environment, was proposed in 1994 (18). Compared to augmented reality, a mixed reality not only overlays but anchors virtual objects to real-world objects and allows the user to interact with combined virtual and real objects. Notably, segmented and processed medical images are universally used in all three of these advanced visualization techniques (Fig. 5). Clinically, virtual 3D cardiac modeling has recently become regarded as an invaluable alternative to cadaveric heart specimens that are difficult to obtain and subject to tissue degradation over time (1920). These 3D digital models can be viewed within different visualization methods and printed as a 3D physical replica. However, it should be noted that virtual 3D modeling requires labor-intensive, time-consuming processing and may provide lower image quality than a typical volume rendering (2122).

Fundamental elements of augmented reality include real objects (such as patients), virtual objects processed from medical images, spatial registration, instrument tracking, and visualization. The combined visualization of spatially registered virtual and real objects in augmented reality allows clinicians to maintain focus and work without taking their eyes off their patients. Thus clinicians avoid distractions and can concentrate on the procedure, while maintaining their ability to detect unanticipated complications. Spatial registration has been mostly applied to rigid organs. Accurate registration is difficult to achieve in soft or moving organs due to extensive elastic deformation, which is why applying augmented reality to CHD has been extraordinarily cumbersome. A tracking device is necessary for surgical or interventional navigation. Currently, optical, or electromagnetic tracking systems are used, and each of these has its benefits and limitations (123). In order to overlay virtual objects on real objects, a variety of display methods, such as conventional display monitors, head-mounted displays, projection displays, and transparent screens, may be used. Visualization using augmented reality is insufficient for expressing depth perception. This shortcoming could be improved by seamlessly switching between augmented reality and virtual reality (24). The main barriers to the extensive use of augmented reality in medicine include the initial setup costs, additional time required to prepare virtual data, and unreliable accuracy in spatial registration. On October 24, 2018, the United States Food and Drug Administration cleared the first mixed reality medical solution, HoloLens® (Microsoft, Redmond, WA, USA), for use in pre-operative surgical planning. The technology can overlay two-dimensional, 3D, and four-dimensional images of patients onto the patient’s body and provides true 3D depth perception. Holography, a 3D image formed by the interference of light beams from a laser or other coherent light source, recently became available for providing an intuitive and interactive real-time display of virtual objects in augmented or mixed reality without the need for any head-mounted devices by using a product called Holoscope® (RealView Imaging, Yokneam, Israel) (25).

In CHD, augmented reality can be used for education, pre-procedural planning and simulation, and procedural guidance. Augmented reality is particularly helpful for minimally invasive cardiac intervention or surgery. It was demonstrated that augmented reality-enhanced transesophageal echocardiography could help to determine the optimal annuloplasty ring size (particularly for surgeons with limited experience), and for percutaneous mitral valve repair (26). Another study demonstrated that augmented reality using ECG-gated CT data facilitated transcatheter heart valve implantation during valve-in-valve procedures while using less contrast agent (27). In a patient with congenitally corrected transposition of the great arteries, augmented reality was used to guide transcatheter pacemaker implantation (28). Mixed reality using HoloLens® has helped surgeons understand the complex morphologies of CHD (29).

In virtual reality, 3D cardiac models should be generated first from medical images. Dynamic cardiac models may be produced from a static model using non-linear registration for virtual reality (30). For interactive visualization and manipulation of virtual objects, virtual reality equipment—such as headsets, controllers, and base stations to detect the spatial locations of the user and the controllers—is necessary. The users may lose their sense of orientation when virtual objects are scaled up and rotated extensively. Furthermore, the entire virtual environment of virtual reality (completely separated from the real environment) is regarded as having narrow medical applicability due to patient safety concerns.

Until now, the medical use of virtual reality in the field of CHD has been quite limited. In 2001, virtual reality was first applied to interactive, real-time visualization of cardiac morphology (31). Its complementary role was recently demonstrated in two infants with CHD (32).

Charles Hull introduced the first 3D printing technology in 1986. Since then, technical developments in medical 3D printing have evolved continually. The workflow for 3D printing comprises several sequential stages: 1) 3D data acquisition, 2) segmentation, 3) conversion of a Digital Imaging and Communication in Medicine (DICOM) file to a 3D mesh file format, 4) computer-aided design (CAD), and 5) 3D printing (Fig. 6). Acquisition of high-resolution isotropic 3D image data with excellent image quality is a mandatory prerequisite for creating a quality 3D printed model. In this regard, cardiac CT is the most commonly used 3D medical imaging modality for CHD 3D printing (2). Less frequently, 3D cardiac MRI (33), 3D echocardiography (34), or rotational angiography (35) have been used. Models used for CHD 3D printing such as a blood pool (a cast model) or wall (a hollow model with 1–2 mm minimum wall thickness) of cardiovascular structures must be segmented using dedicated software tools developed for generating Standard Tessellation Language (STL) files or various alternate file types such as object file (OBJ), 3D manufacturing format (3MF), virtual reality modeling language (VRML). Mesh generation for the polygon structure is necessary to use CAD. There are several software programs ranging from expensive fully-featured commercial options such as Mimics (Materialise, Leuven, Belgium) and D2P (3D Systems, Rock Hill, SC, USA), mid-range versions such as Medip (MEDICALIP, Seoul, Korea), and open-source freeware platforms such as Slicer (http://www.slicer.org) and itk-SNAP (http://www.itksnap.org). Thresholding, region growing, and manual editing are used for segmentation. Exceptional expertise in the field of CHD is required for accurate and relevant segmentation. Even for top experts, segmentation is time-consuming (usually 2–3 hours) depending on the image quality of the original 3D medical imaging data (23).

Consequently, the importance of the original data cannot be overemphasized for obtaining quality STL files and printed models as well as for reducing image processing time. Newer software tools using semiautomatic segmentation may reduce time and improve reproducibility (36), but further refinement (e.g., wrapping or smoothing) using CAD software is required because the segmented STL images are not optimized for 3D printing in most cases. The process of ‘wrapping’ creates a watertight 3D surface by filling any holes, and the ‘smoothing’ process is often needed to make the 3D surface regular and even. Moreover, CAD software can be used to add artificial tissue such as a ventricular septal defect patch to a patient-specific 3D cardiac model (37)

The refined STL files are loaded into a 3D printer, then printing is started with appropriately selected materials. A variety of 3D printing technologies are available in cardiovascular medicine, including fused deposition modeling, selective laser sintering, stereolithography, material jetting, and injection molding; the details have been already described elsewhere (4383940). A wide range of printing materials may be selected, including plastic (resin), silicone, nylon, and metal, as needed, and are primarily divided into rigid vs. soft rubber-like materials, opaque vs. transparent, and single-color vs. multi-color (4383940414243). However, soft materials are still quite different from real cardiovascular tissues. Therefore, surgeons find the models more difficult to suture because they are easily torn or cut (244) (Fig. 7). A new flexible material (Agilus30 white, Stratasys, Rehovot, Israel), which is less prone to tearing, was recently introduced in November 2018, to compensate for these shortcomings. After a model is printed, a cleaning or finishing process is required, and sufficient time should be spent finishing the newly printed material. The time required to make the final product (as much as 24 hours), the cost of printing one unit (1000–3500 US dollars) as well as the quality of the printed models are quite different depending on the selected printer, printing material, model size, and complexity of cardiac morphology (243345464748). Recently, the time required for detailed segmentation was markedly reduced by using direct conversion from a segmented DICOM file to an STL file, available in a few clinical high-performance 3D workstations for CT and MRI (Fig. 8).

Binder et al. (49) first applied 3D printing from echocardiographic data to cardiac structures in 2000. In the last two decades, 3D printing has increasingly been used for diagnosis, pre-procedural planning and simulation, and different levels of training and education for a variety of simple and complex CHDs (23233343745505152). It is not surprising that 3D printing accurately replicates 3D digital models and 3D source images to a submillimeter level when considering the mechanical precision of the technology (323352). Thus, most errors in 3D printing are introduced during segmentation or via CAD.

Further medical value of 3D printing for CHD has been evaluated predominantly through subjective assessments and questionnaires without correlation to outcome measures mostly in case series or small studies (475152), and even in a prospective multi-center study (30). The added value should be more meaningful and more evident in complex cardiac defects than in uncomplicated cardiac defects. The required quality of 3D printed cardiac models is not the same in all medical applications. For example, the quality of 3D printed cardiac models for hands-on surgical simulation should be sufficiently high to mimic genuine cardiovascular tissue as much as possible, while such high-quality models may not be necessary for education and communication. In urgent clinical cases, the time required for the whole 3D printing process should be strictly controlled so that the resulting device is completed promptly (e.g., within 2–3 business days).

CT or MRI may be used to create a 3D printed cardiac valve model if imaging is acquired when the cardiac valves show minimal motions (Fig. 9). Alternatively, artificial valves created using CAD may be added to a patient-specific 3D cardiac model (2). Because CT or MRI cannot provide high-quality images of delicate and fast-moving cardiac values, 3D echocardiography is used to create a 3D printed cardiac valve models (43). Hybrid 3D printing, which integrates the strengths of 3D echocardiography to define the cardiac valves and the strengths of CT or MRI to define other cardiovascular structures, has been attempted (5354). Patient-specific cardiovascular phantoms using 3D printing may be designed for device testing or advanced hemodynamic evaluation (55).