To obtain a 3-D polygonized model with a better visualization, segmentation was performed using the 3D Slicer version 4.10.2 (http://www.slicer.org).

For the vertebrae, 3-D data were obtained separately from each of the five lumbar vertebrae, the sacrum, and the pelvic bones. In addition, 3-D polygonal shapes of the intervertebral discs between each vertebra were obtained. In the case of the skull, 3-D data as a single bone was obtained, on the theory that the area to be treated will not be greatly affected by the movement of other bones. The whole-body contour was also obtained by leaving all the remaining values except for air. Although each polygonized data point was segmented separately, their spatial positional references were maintained to obtain a combined 3-D structure that could be reassembled.

The structure obtained had approximately 60,000 to 120,000 polygons per bone. However, when the number of polygons increases, all vertices, edges, and faces require computing resources in the program. Equipment that implements virtual reality needs to perform these calculations about 60 times per second, so in general, if the total number of polygons exceeds 1 million, performance may be affected. Therefore, there was a risk of performance degradation when used directly in a program. Therefore, using MeshMixer version 3.5 (Autodesk Inc., San Rafael, CA), a model was used that reduced the number of polygons by 5%.

For image acquisition, it was necessary to load the DICOM file format in the form of binary data that could be manipulated directly in the virtual space. In the case of CT, the DICOM file contains information such as the spatial location, size, and direction of each datum; therefore, it is highly effective if it can be used directly rather than transforming it into another form. It was necessary to place both the data and virtual needle directly in the virtual space to generate a virtual X-ray quickly and accurately. For this, existing libraries were sought out; however, none met the requirements. Therefore, a DICOM file handler compatible with the UNITY engine had to be created. A program was created that could read and manipulate DICOM files in the UNITY version 4.2 (Unity Technologies, San Francisco, CA) environment. Since the Grassroots DICOM version 3.0 (http://gdcm.sourceforge.net) library was not compatible with UNITY, CMake version 3.119.1 (Kitware, Inc., Clifton Park, NY), Simplified Wrapper and Interface Generator version 4.0.2 (http://www.swig.org/) were used to set an environment that could read and write DICOM data in a virtual environment for the UNITY and Visual Studio 2019 version 16.10.3 (Microsoft, Redmond, WA). In the development environment created, DICOM data were read as binary data and were classified with tags such as Image Type (Tag# 0008,0008), Image Orientation (Tag# 0020, 0037), Instance Number (Tag# 0020, 0013), Image Position (Tag# 0020, 0032), Pixel Spacing (Tag# 0028, 0030), Windows Center (Tag# 0028,1050), Windows Width (Tag# 0028,1051), Rescale Intercept (Tag# 0028, 1052), Rescale Slope (Tag# 0028, 1053), Pixel Data (Tag# 7FE0,0010), etc., and stored into a separate series of arrays.

The Hounsfield unit (HU) value of each image was obtained by adjusting the pixel data value of the extracted data with the rescale intercept and rescale slope values. The CT image was obtained by assembling the pixel colors and adjusting the obtained HU values using the Windows center and Windows width values. The images were positioned in the virtual space using the values of image orientation, image position, and pixel spacing (Fig. 1A). When only HU values between 200 and 400 (adjusted according to bone density) were visualized, a 3-D bone-like image was obtained (Fig. 1B).

The spatial coordinates of the four corners of the obtained images were stored in arrays. An X-ray generator and a detector were set in a virtual space to reproduce the actual procedure using the C-arm. The spatial coordinates of the four corners of the detector were divided by the desired resolution (x × y) to set targets for a virtual X-ray that started from the generator, as shown in Fig. 2A (resolution 5 × 5) and Fig. 2B (resolution 10 × 10). The generator and detector were programmed such that the position and direction could be adjusted similar to the actual use of the C-arm. Using a keyboard, horizontal direction keys were set to adjust the caudad to cephalad tilting, and vertical direction keys were set to adjust the left-to-right rotation. The point of the CT slice images where the collision occurred was calculated using the ratio of the dot product of each X-ray with the reference point of the obtained images. After the corresponding HU values were obtained, the sum of these values was calculated. By dividing this value by the total number of CT slices (the number of the slices ranged from 200 to 600 depending on the patient), the degree of absorption of X-rays was calculated, and virtual X-rays were obtained by arranging them sequentially by converting the values into pixel values (Fig. 2C [resolution 128 × 128] and Fig. 2D [resolution 512 × 512]).

The previously generated 3-D polygonal data and the bone shapes obtained by manipulating the DICOM data in UNITY were precisely positioned in identical spatial coordinates to increase the performance and convenience of the pre-procedural simulation (Fig. 3A and B). A virtual needle was then created to simulate the procedure. The needle was designed to have a shape and size similar to that of the actual needle (Fig. 3C), and colliders were placed in the handle and the needle shaft (Fig. 3D). When the virtual X-ray made contact with these colliders, a value of 2,000 HU was added to the final image so that the needle is expressed as a metallic material on the virtual X-ray image (Fig. 3E and F).

The needle positioning was programmed to move to the target point by holding the approximate position with a mouse and fine-tuning it using keyboard input. In addition, the virtual camera and C-arm were programmed to move and rotate in detail using keyboard input. When the angle of the C-arm was adjusted, the degrees of tilting and rotation were displayed as angles. By another keyboard input, a virtual X-ray could be generated when the desired position of the needle has been obtained in virtual space. Through this simulation, the 3-D positional relationship between the needle and the patient’s anatomy according to the generated X-ray image could be explored in detail.

An 84-year-old woman visited the hospital with a complaint of chronic back pain that radiated to the lateral side of her left leg. Lumbar magnetic resonance imaging (MRI) revealed severe deformity and left foraminal stenosis at the L5/S1 level, which could be responsible for the pain (Fig. 4A). Therefore, transforaminal epidural injection was attempted at the left L5/S1 intervertebral foramen. At the first trial, the authors failed to place the needle in the desired position despite several attempts because her L5/S1 intervertebral foramen was severely narrowed due to deformities and osteophytes. Therefore, a pre-procedural virtual simulation was planned to determine the placement of the needle in the correct position. The simulation was performed using the virtual 3-D data of the patient. After positioning the needle at the optimal position in the anterior epidural space (Fig. 4B), the virtual C-arm was adjusted to the angle at which the needle was viewed as a point (Fig. 4C). Further, several other views were simulated, including the anteroposterior view (Fig. 4D). The spatial relationship between the needle and the spinal anatomy of the patient could be compared in the virtual space. In addition, a 2-D X-ray image could be simulated, which should be obtained in advance for a successful procedure by moving the needle forward while keeping the needle visible as a point. When the patient revisited the hospital, a similar image was obtained using the angle obtained through the simulation, and matched the relative position of the vertebral body and the articular processes by finely adjusting the angle and distance of the actual C-arm, as seen in Fig. 4C. The authors successfully positioned the needle as a pinpoint (Fig. 4E). The needle location was further confirmed through the antero-posterior view (Fig. 4F), and it was found that the contrast medium had appropriately spread into the anterior epidural space.

A 77-year-old man visited the hospital complaining of right pelvic pain due to lung cancer and pelvic bone metastasis. Cancer metastases was observed in the right L5 vertebra and sacral ala of the patient, along with bone marrow edema in the sacroiliac joint in his MRI (Fig. 5A). To ease the severe pain reported by the patient, a superior hypogastric plexus block was planned, which is a challenging procedure since the needle must travel a long distance to the ventral lumbosacral area by avoiding the osteophytes of the transverse process and vertebral body. He had cancer metastasis at the L5 vertebra and sacrum, and subsequent degenerative changes with osteophytes made it more difficult to get the needle to the target. In addition, the transdiscal approach could not be attempted due to the narrow intervertebral disc height of the L5/S1.

We carefully planned a pre-procedural virtual simulation using virtual 3-D data (Fig. 5B). The postero-lateral approach was used and the appropriate angle and direction of the needle that reached the lower L5 anterolateral surface of the vertebral body from the entry point starting near the L4 vertebral body was investigated. The relative position of the C-arm was recorded when the needle was viewed as a point (Fig. 5C). The antero-posterior view of the virtual X-ray was saved, as shown in Fig. 5D. During the next visit of the patient, the C-arm angle was adjusted to obtain the same view as shown in Fig. 5C and inserted the actual 25-gauge needle as a point without difficulty (Fig. 5E). The location of the needle was confirmed with the contrast medium in the anteroposterior view as shown in Fig. 5F.

A 69-year-old woman visited the hospital complaining of pain in the left cheek that had begun suddenly and felt like stabbing and electric shock (as described by the patient). A pain a score of 8–9 was assigned by the patient on the numerical rating scale. The pain lasted for 1–2 minutes and became worse with cold stimuli and on eating. A diagnosis of trigeminal neuralgia was established and a Gasserian ganglion block was planned using an anterior approach to relieve the symptoms of the patient. However, in this procedure it is often challenging to identify the target point (i.e., the foramen ovale) even with aid of fluoroscopic guidance, and due to her degenerative changes near the target, there was a fear of injury to important nearby structures. A pre-procedural virtual simulation was attempted to understand the complex structures of the skull. The skull bone was embodied in a virtual simulation using head CT. After accurately positioning the needle in the foramen ovale (Fig. 6A and B), the virtual C-arm was adjusted to view the needles as a point and saved the lateral view (Fig. 6C and D).

Subsequently, when the patient visited the hospital, the foramen ovale was approached as implemented in the virtual simulation, and the procedure was performed easily without complications (Fig. 6E and F).