PDF
ePub
Citation
Print
Introduction
Three-dimensional (3D) surface models of human organs can be used for interactive simulation; the hollow models, with small file sizes, can be opened, rotated, and modified in real time even on-line [1]. The surface models have been stereoscopically drawn from knowledge of anatomy. However, impartial models might be constructed by surface reconstruction after stacking serial outlines of the real human structures. Computerized tomographs (CTs) or magnetic resonance images (MRIs) are not sufficient for obtaining serial outlines of structures such as small individual muscles. In our laboratory, high-quality sectioned images were generated from a male cadaver head. In the sectioned images, the borders of structures were delineated to produce outlined images. Subsequently, the outlines could be filled with white or different colors [2,3]. The resultant three sets of images, namely outlined images, white-filled images, and color-filled images, are referred to as segmented images. Based on these segmented images, an effective surface reconstruction technique has been developed using conventional software programs without extra programming [4-7].
The objective of the present research was to ascertain the usefulness of the outlined images, white-filled images, and color-filled images with respect to surface reconstruction. In addition, whether the three types of segmented images could be converted from one to another was evaluated.
Materials and Methods
Head MRIs were obtained from a Korean male cadaver (67-years-old). The head was then separated from the cadaver to be embedded and horizontally milled at 0.1 mm intervals. All sectioned surfaces were digitalized to create sectioned images (0.1 mm-sized pixels; 48 bits color; tagged image file format [TIFF]) [2].
From a total of 2,343 sectioned images, we chose 234 images, taken at 1 mm intervals, to outline the head structures. The 91 structures of interest included brain structures, bones, muscles, arteries, and veins in the head [8]. On Photoshop CS3 ver. 10 (Adobe Systems, Inc., San Jose, CA, USA), outlines of each structure were drawn and placed among the structure's own layers of the Photoshop document (PSD) file to prepare 234 outlined images (Fig. 1A) [3].
From the outlined images, white-filled images and color-filled images were constructed (Fig. 1). The three types of segmented images were converted from one type to the other. From every category of segmented image, 3D surface reconstruction was performed to build the surface models (Fig. 2).
Conversion of outlined images into white-filled images (Photoshop)
Among the outlines of the 91 head structures, only outlines of a specific structure were remained using layers in the PSD files. The outlines were then filled with white to produce white-filled images (2 bits color; TIFF files) of a structure [3]. Unlike the outlined images, the white-filled images excluded the sectioned images (Fig. 1B). In the case of skin, 234 serial white-filled images were constructed and moved to the skin folder while for the cerebrum, 113 serial images were constructed. Folders for all 91 structures were created with the white-filled images of every individual structure [9].
Conversion of outlined images into color-filled images (Photoshop)
The outlines of every structure in the PSD files were filled with a specific color by taking advantage of the structure's layers, and the sectioned images were deleted (Fig. 1C). Diverse colors for individual structures were previously determined. The order of some layers was arranged to prevent external large structures from hiding internal small structures. As a result, 234 color-filled images (16 bits color; TIFF files) from 91 structures were obtained [3].
Conversion of white-filled images into color-filled images (Photoshop; After Effects)
In the white-filled images (TIFF files) of the structures, the white was changed into a previously determined color for the structure, and the background was made transparent by saving as a portable network graphic (PNG) file on Photoshop [10].
On the After Effects CS5 (Adobe Systems), video editing program, 91 empty layers with 234 frames was constructed. The timeline panel of the After Effects showed the layers and frames in the rows and columns, respectively. One frame was adjusted to be one second, so that the 234 frames were represented in three minutes and 54 seconds. The color-filled outlines of each structure (PNG files) were placed on the corresponding layer and the corresponding frame until all outlines of every structure were in place. The timeline panel then showed the entire distribution of the existing outlines (frames) according to the structures (layers); this allowed any mistakenly omitted outlines to be easily detected so that they could be corrected. The timeline panel also helped plan surface reconstruction of the individual structures, the next step in the process (Fig. 3). Each frame including several layers of the structures was finally converted into TIFF file that was the color-filled image (Fig. 1C).
Conversion of color-filled images into white-filled images (Photoshop)
At the top left side of the color-filled images (TIFF files), a tiny square was painted with the same color as the target structure. This top left square was selected using the 'magic wand' tool without the 'contiguous' option. As a result, not only the top left square, but also the target structure was simultaneously selected. After excluding the top left square, the structure was filled with white. Then, the 'inverse selection' was chosen to fill the entire outside of the structure with white (Fig. 1B). Taking all color-filled images together, the procedure was executed using the 'batch' function to produce a whole set of white-filled images (TIFF files) of the structure at one time and this process was carried out repeatedly for all structures [7].
Conversion of white-filled images into outlined images (Photoshop)
In the white-filled images (TIFF files) of each structure, the outlines were extracted using the 'magic wand' tool and then positioned on the corresponding layers of the PSD files. The procedure was repeated for the other structures. Lastly, the sectioned images could be retrieved to obtain the outlined images (Fig. 1A) [9].
Conversion of color-filled images into outlined images (Photoshop)
This procedure was the same as above for the outlined images converted from the white-filled images. The only difference is that a color-filled image has various structures while a white-filled image has only one structure.
Surface reconstruction of outlined images (Photoshop; Maya; Rhinoceros)
With Photoshop, the serial outlined images (PSD files) of a structure were converted into vectorized outlines that were then stacked on Maya ver. 2009 (Autodesk, Inc., San Rafael, CA, USA). Using Rhinoceros ver. 3.0 (McNeel North America, Seattle, WA, USA), gaps between the stacked outlines were filled with polygons to build a surface model. The stacked outlines were deleted to smooth the surface model, and the number of polygons was appropriately reduced on Maya (Fig. 4A). The refined surface model was saved as a Maya binary (MB) file [11-13].
For a supplement, volume models were constructed from the outlined images containing the sectioned images (Fig. 1A). Using Photoshop, the outlines were used to prepare the sectioned images of a structure with the outside deleted. With 3D-DOCTOR ver. 4 (Able software, Corp., Lexington, MA, USA), the sectioned images were piled and volume-reconstructed to build a volume model of the structure. The volume model showed the sectional planes as well as the surface with real body color (Fig. 4B, C). The volume models could be greatly improved using computer programming language [14].
Surface reconstruction of white-filled images (Combustion; 3D-DOCTOR; Maya)
Surface reconstruction could be automatically carried out only when neighboring white-filled outlines of a structure successively overlapped. Therefore, in the cases that were not overlapped (e.g., superficial temporal artery), the white-filled outlines were interpolated on Combustion 2008 (Autodesk, Inc.) to make them overlap [15, 16].
The white-filled outlines of a structure from the TIFF files were sequentially stacked using 3D-DOCTOR. The white-filled outlines were expanded to the next images to build combined volume models. From these, a surface 3D model was extracted to be saved as a stereolithography (STL) file. On Maya, after eliminating the outlines, polygons were reduced in number to be saved as an MB file (Fig. 4A) [13, 16].
Surface reconstruction of color-filled images (3D-DOCTOR; Maya)
For the color-filled images (TIFF files), serial outlines of individual structures were grouped by the 'auto segment' command of 3D-DOCTOR. Surface reconstruction of all structures was then achieved in the same manner with 3D-DOCTOR as previously described. Consequently, a surface model integrating several structures at once was built. With Maya, using the 'separate' command, the surface model was divided into a number of models (MB files) of diverse structures, each of which was also refined by erasing the outlines. However, if the color-filled images contained structures without overlapping outlines, the structures should be separated and interpolated prior to the surface reconstruction.
Treatment of the surface models (Maya)
In a newly created MB file, the surface model of each structure was placed in its own layer to produce assembled surface models, which allowed selective display. The surface models built with Rhinoceros preserved the information on the original size and location of the structures. However, the surface models constructed using 3D-DOCTOR did not keep this information. Therefore, the separate models had to be properly placed in the stacked outlines of all structures prior to assembling [13]. The surface models were painted with different colors (Fig. 4A).
With the Maya software, the surface models of the chosen structures were displayed with their transparency adjusted to show various scenes (Fig. 5). The 3D surface models were superimposed over the various corresponding two-dimensional (2D) images of the same subject. For the 2D images, the following could be alternatively exhibited: horizontal, coronal, sagittal planes of the sectioned images, color-filled images, and MRIs (Fig. 6).
Results
The surface models of the structures, which were constructed from the outlined, white-filled, color-filled images, did not show any different features from one another despite different reconstruction processes. The surface models corresponded to the volume models from the same subject (Fig. 4A, B). Although the resultant models from the outlined, white-filled, color-filled images were similar, handlings of the three types of segmented images on software were quite different in terms of easiness (Table 1).
The outlined images were advantageous from these points of views. The outlined images could be easily produced from the sectioned images. The outlined images were derived directly from the sectioned images. However, the white-filled images and color-filled images were obtained by way of the outlined images (Fig. 2). The outlined images could be easily converted into the other segmented images. The outlined images were automatically converted into the successive white-filled and color-filled images by matching layers in PSD files [3]. However, the reverse was a relatively labor-consuming process that required such maneuvers as extraction of the outlines (Fig. 2). The outlined images could be easily transformed into volume models. The outlined images, conveying sectioned images, were the only resource for both the surface and volume reconstruction (Fig. 4).
Both the white-filled images and color-filled images proved to be beneficial in the following respects. The white-filled image and color-filled images could be easily transformed into surface models. The white-filled images were desirable for the convenient surface reconstruction of discrete structures. In the cases without overlapping of the serial images, due to a large interval (1 mm instead of 0.2 mm), the white-filled images were easily interpolated for overlapping and this accelerated automatization of the reconstruction [15, 16]. The color-filled images were effective in simultaneous surface reconstruction of many adjacent organs. However, outlined images were inconvenient for surface reconstruction [4].
The white-filled and color-filled images could be easily distributed and stored. The white-filled images with just two bits of color had the smallest file size in spite of large number of files in the 91 folders. The color-filled images had a small file size as well. By contrast, the outlined images were large files due to the highly organized PSD files, even when the sectioned images were discarded. The small file size was favored for distribution and storage by both providers and users [9].
The color-filled images had an additional merit. By utilizing the color-filled images, outlining could be easily verified. Serial color-filled images (TIFF files) could be continuously observed in standard image viewers to recognize all outlined structures. This was helpful for planning surface reconstruction of many structures. Regarding the PSD or PNG files, that are the precursors of color-filled images, the timeline panel of the After Effects program, displaying the distribution of the outlined structures at a glance, was useful (Fig. 3). However, the white-filled images of each structure were incapable of simultaneous visualization of neighboring structures.
Discussion
Surface models of anatomic structures can be made through segmentation and surface reconstruction process. In our recent report, surface reconstruction was dealt with [16], whereas in this research, we intensively considered capabilities of segmented images. The outlined images, white-filled images, and color-filled images were used as the resources for the construction of 3D surface models. Among the three kinds of images, there was no absolute fit for the all procedures for the surface reconstruction (Table 1). Each investigator must determine which type of image suits their needs.
First, the outlined images from the three types might be chosen to construct both surface and volume models. From the outlined images containing sectioned images, a certain structure's unique set of sectioned images can be generated, which can be used as the material for volume reconstruction. The volume models, with the corresponding surface models can be used as educational tools (Fig. 4). The outlined images have the additional advantage of being capable of surface reconstruction, whether or not the outlines between adjacent cross-sectional planes overlapped [4, 16, 17]. The constructed surface models maintain the structures' information regarding location. Moreover, the outlined images can easily be changed to the white-filled and color-filled images (Table 1).
Second, the white-filled images might be preferred in cases where the surface models were constructed separately. The white-filled images of each structure were already arranged in a folder, which permitted straight surface reconstruction of the structure. Similarly, interpolation of the images can be readily performed [15]. Furthermore, the white-filled images with their small file size can be conveniently distributed even on-line (Table 1) [9]. Easy distribution is meaningful because the segmented images, unlike the sectioned images, could be frequently updated and shared by users [10, 13, 15].
Third, the color-filled images might be useful for building multiple neighboring surface models at once. Simultaneous recon struction is possible using the newly developed technology in the present study. Similar to the white-filled images, the color-filled images had a small file size, which assisted wide use. One additional benefit of the color-filled images is the facilitation of debugging the delineation and planning the surface reconstruction of numerous structures (Fig. 3) [7]. The color-filled images could be used for identification of outlined structures in the corresponding sectioned images (Table 1). After programming, the names of the structures could be displayed using the structure's RGB triplet [10].
An investigator, that selects one among the three sets of segmented images, can easily switch it to another. For example, a user who downloads the color-filled images for his/her original use might have to convert the color-filled images to white-filled images for another purpose. The well automated conversion method, suggested in this study, makes the user able to adapt to a variety of situations. The investigator, aware of the methods, could independently obtain any wanted 2D images and 3D models. The method suggested in this paper seems to be the best in terms of flexibility between the image data (Fig. 2).
The above-mentioned segmented images are meaningful not only for the surface reconstruction but also for the production of animation. When an animated 2D movie is developed on the computer, graphic objects (e.g., leading and supporting characters) could be regarded as the segmented images. Just as the surface model requires serial gathering of the segmented images that are contour lines, the animation requires serial display of the segmented images that are frames. Both the surface reconstruction and video production might use the outlined, white-filled, and color-filled images alternating among them (Fig. 2).
The similarity of surface reconstruction and animation production led to the diverse use of the software packages in the present study. The After Effects and Combustion programs were originally developed as video editing software, equipped with the tools to handle frames. In the After Effects program, the timeline panel, which provides a glance at the layers (structures) and frames (extent of structures' outlines), is used to detect any mistakes and to plan the surface reconstruction (Fig. 3). With the Combustion program, the motion tracking function, which creates the intervening frames for smoothing the animation, is used to interpolate between the non-overlapping adjacent white-filled images [15]. The beneficial use of the software is based on the idea of regarding the 'time (frames)' of animation as the 'level (contour lines)' of the 3D models.
Maya is a powerful program for both surface reconstruction and animation production [18, 19]. The final product of the present study is the 91 surface models in an MB file. The models, placed in the proper layers, can be promptly processed in Maya for making animations such as brain operation.
The detailed surface models are accompanied by the MRIs, sectioned images, and three types of segmented images from a male cadaver. Corresponding images could be constructed in various combinations (Fig. 6). Referring to the sectioned images with high resolution and real body color, the segmented images and surface models of other structures could be elaborated by other investigators [4, 5, 15, 20]. If necessary, 3D surface models of structures not outlined can be stereoscopically drawn in addition to the constructed surface models [13, 21]. Without doubt, it would be beneficial to share all image data among different users.
The 2D images and 3D models of the male head are expected to contribute to the medical simulation of head and neck surgery and neurosurgery (Figs. 5, 6). In particular, the virtual simulator would likely be more valuable if equipped with a haptic device to allow users to experience the physical properties of each anatomic structure [12, 22].
This study has determined the value of the outlined images, white-filled images, and color-filled images with regard to surface reconstruction. The concept of three segmented images can be applied to building various volume and surface models from patient's serial images (e.g., CTs and MRIs). The 3D models, accompanied by the corresponding 2D images, which would be helpful in diagnosis and surgery planning.
XML Download