Virtual Reality Technology and 3D Reconstruction in Hepatectomy

Abstract

Three dimensional (3D) reconstruction can realize the accurate preoperative diagnosis of liver and intrahepatic tumors, and can also help to complete the preoperative operation planning. 3D visual surgical navigation technology has been applied in liver surgery. Virtual reality (VR) technology is still in the primary stage of research and development. It can only simulate the operation of 3D reconstruction model of liver, but cannot fully show the actual operation scene. With the development of computer technology and the further maturity of 3D reconstruction and VR technology, their application in liver surgery is showing a bright future.

Keywords

Liver, Virtual reality technology, 3D reconstruction, Hepatectomy

Abbreviations

3D: Three Dimensional; 2D: Two Dimensional; CT: Computed Tomography; MRI: Magnetic Resonance Imaging; VR: Virtual Reality

Introduction

Three dimensional (3D) reconstruction technology is a kind of mathematical model established by computer for 3D objects. The model can be operated, processed and analyzed in the computer environment. It is also the key technology in virtual reality (VR) technology. After 3D reconstruction of the two-dimensional information obtained from the routine CT/MRI examination of the liver by the software, surgeons can examine the volume of the liver, anatomical variation, the course of the intrahepatic vessels, the location of the tumor and its relationship with the surrounding vessels more intuitively, vividly and from multiple angles [1-3]. Preoperative 3D reconstruction and VR technology can realize the measurement of liver volume and the implementation of simulated hepatectomy, which can further clarify the scope of surgical resection and ensure the residual liver volume and function to meet the needs of patients after operation.

The virtual operation and image navigation before and during the operation can also prevent the injury of important blood vessels and bile ducts in the liver during the operation, significantly shorten the operation time, reduce the bleeding during the operation, and reduce the occurrence of complications such as liver dysfunction, bile leakage and bleeding after hepatectomy [4-6]. 3D reconstruction and VR technology can help surgeons to optimize the operation plan, predict the possible accidents in the operation in advance, and ensure the radical resection of liver tumor.

Preoperative Planning and Virtual Surgery

The anatomical structure of the liver is complex, and there are many variations in the intrahepatic vascular and biliary system. Hepatectomy is still a kind of high-risk operation, with high incidence of complications and high mortality. At present, the diagnosis of intrahepatic lesions and the design of surgical scheme before operation mainly rely on the 3D reconstruction of continuous 2D images of patients by surgeons brain. The 3D reconstruction by human brain needs skilled film reading experience and a certain accumulation of surgical experience, which can be formed after long-term specialized training. Human brain reconstruction image is temporary and unstable, which is difficult to present to the whole treatment team. The reconstruction results of each surgeon are different from the actual situation, which is significantly related to the experience level of the operator. However, 3D reconstruction software can be used to reconstruct liver CT or MRI image data. This kind of software system can fully understand the course and variation of intrahepatic vessels, and judge the spatial relationship between tumor and them.

The individualized 3D reconstruction model of liver based on thin-layer CT or MRI data can make surgeons understand the location of tumor and its adjacent relationship with intrahepatic vessels more intuitively, clearly and multi dimensionally. The 3D model established by 3D reconstruction technology can also be used for free combination or appropriate transparency of liver, tumor and vessels according to different needs. By enlarging, shrinking and rotating, surgeons can observe and analyze the distribution of intrahepatic vessels, understand the spatial relationship between them and tumors, and measure the distance between the tumors and related vessels. 3D reconstruction technology can be used to plan and simulate the operation before operation, and can implement intraoperative guidance, which is conducive to shorten the operation time and improve the operation safety. At the same time, 3D reconstruction technology can automatically segment the liver and accurately locate the location of the liver segment or lobe where the tumor is located. Virtual liver resection technology can be used to calculate the volume of liver resection and residual liver. The 3D reconstruction model has the characteristics of individualization, and the reconstructed image can be presented to the treatment team permanently [7-9].

The factors that need to be considered in the process of liver surgery mainly include the following points: (1) Location of tumor and the venous return system of remaining liver, and (2) blood supply of tumor, hepatic artery and portal vein supply branches, and (3) sufficient resection margin, and (4) sufficient residual liver volume. The 3D reconstruction model can display the branches of hepatic artery and portal vein which supply tumor, and it can display the corresponding backflow hepatic vein. It also can help surgeons to judge whether to take regular or irregular hepatectomy, and determine the scope of resection before operation.

In addition to providing accurate 3D images of liver, tumor and intrahepatic vascular system, the virtual operation function of 3D reconstruction technology can assist surgeons to design operation planning. Virtual surgery is based on the 3D reconstruction of intrahepatic duct system and tumor anatomical structure, and the interactive platform is established by software to simulate the operation procedure. Virtual surgery can show the important pipeline structure that may be encountered in the real operation process in advance, so that the operator can make necessary preventive measures in advance. By calculating the volume of the liver to be resected and the volume of the residual liver, risk of postoperative liver dysfunction can be judged, which is conducive to the development of individualized operation plan, reducing the risk of damage to important liver structures and reducing the risk of postoperative liver dysfunction [10-12].

The biggest advantage of virtual surgery is its arbitrariness and repeatability. The complex and dangerous situations that may appear in the actual operation procedure can be found in advance by imitating the actual procedure before surgery. If the set resection level may damage the important bile ducts and blood vessels in the liver, the operator can adjust and modify the resection level accordingly. After simulating different surgical schemes and comparing them, a reasonable individualized operation plan is worked out. The operator can take necessary preventive measures in advance to ensure the safety of the actual operation. For living donor liver transplantation, virtual surgery can optimize the operation level and determine the best scope of donor liver resection by simulating the operation for many times, which can reduce the risk of damage to residual liver vessels and bile ducts, and retain the blood circulation of donor liver and residual liver to the greatest extent.

Virtual reality technology and intraoperative navigation

With the development of 3D reconstruction and VR technology, intraoperative navigation system has been widely used in plastic surgery, neurosurgery and maxillofacial surgery [13-15]. The application of VR navigation system in liver surgery has also been carried out. The application of intraoperative navigation combined with intraoperative ultrasound guidance in open hepatectomy has been reported. VR technology can use the image data of preoperative planning for real-time operation, guide the resection process for the operator, and make the liver resection operation more safe and convenient.

Navigation by VR technology in liver surgery is mainly realized by 3D reconstruction and virtual reality interaction technology. Surgeons can bring the virtual 3D model of the liver or 3D printed liver model into the operating room, and provide indirect real-time guidance for the operator during the operation. With the guidance of these models, the operator can quickly identify and locate the key parts, accurately locate the tumor and determine the surgical resection plane. It can also guide the operator to pay attention to the anatomy of important vessels and tumor resection in real time. Liver resection assisted by 3D reconstruction technology can effectively reduce the injury and complication rate, so as to improve the safety of the operation. The virtual 3D model of liver is transparent, but surgeons can't understand the various pipes that will be encountered in real time during real liver surgery, which will increase the risk of accidental bleeding and bile duct injury. The development of virtual reality interaction technology provides liver surgeons with a pair of 'perspective eyes' which can penetrate the liver.

Virtual reality interaction technology can fuse the 3D liver model reconstructed before operation with the real liver registration of patients during operation. The operator can see the tumor and pipeline system in real time by wearing a special mask and glasses, without the need for doctors to switch the field of vision between the image display and the actual operation area. The use of this technology can handle the doctor's hand eye uncoordinated problem, provide real-time 3D intraoperative navigation for liver resection surgery, so as to make doctors know the important blood vessels or bile ducts that will be encountered in advance, and ensure the operation safety to the maximum extent finally. But in the actual liver surgery process, usually need to free the liver or because of the operator's operation and patient's respiratory movement, which will affect the virtual reality matching problems. To solve these problems, more appropriate liver biomechanical model building algorithm, more accurate image segmentation algorithm and navigation registration algorithm are needed to solve these problems [16-19].

In recent years, laparoscopic and robotic hepatectomy have been carried out in more and more units. There is a common defect between laparoscopic and robotic surgery, that is, the loss of direct visual and tactile feedback to surgeons. The introduction of VR technology into laparoscopic and robotic hepatectomy can solve this problem to a certain extent. Taking robot liver surgery as an example, the robot operation system adds a computer interface between the operator and the patient, which provides a good platform for the application of virtual reality technology. By installing endoscope camera and other auxiliary equipment on the robot, real-time three-dimensional image and virtual model can be constructed at the same time of operation, so as to guide the operation more accurately. Some researchers have superimposed virtual reality technology into the main control system of Da Vinci surgical robot, so as to achieve the best tool coordination scheme [20-22].

VR technology helps robot to identify all intrahepatic duct structures accurately and quickly, which greatly improves the success rate of robot liver resection. The combination of VR technology and robotic surgery operating system will be a trend in the future. If VR technology wants to play a greater role in liver surgery navigation, it needs to rely on the further development of biomechanical modeling, display sensing technology and target recognition algorithm.

Conclusion

3D reconstruction technology can achieve accurate preoperative diagnosis of liver and intrahepatic tumors, and can also assist in the completion of preoperative surgical planning, which is of great help for accurate hepatectomy. 3D visualization of surgical navigation technology has been applied in liver surgery. Virtual surgery technology is still in the primary stage of research and development. It can only simulate the operation of 3D reconstruction model of liver, but cannot fully show the actual operation scene.

The application of VR technology in liver surgery can enable us to obtain more intuitive and specific 3D model, more comprehensive image information, more easily understood teaching methods and doctor-patient communication means, which will help us more accurately guide the implementation and teaching training of liver surgery. With the further cross integration of computer science, liver surgery and medical imaging, VR technology will be better used in liver surgery navigation and virtual surgery system construction, which will bring great progress to liver surgery. With the further development of 3D reconstruction and VR technology, its application in liver surgery is also showing a bright future.

Competing Interests

The authors have declared that no competing interest exists.

Acknowledgement

We are grateful to Zhao Zhiming for critical comments related to the manuscript.

References

1. Tang R, Ma LF, Rong ZX, et al. (2018) Augmented reality technology for preoperative planning and intraoperative navigation during hepatobiliary surgery: A review of current methods. Hepatobiliary Pancreat Dis Int 17: 101-112.
2. Yeo CT, MacDonald A, Ungi T, et al. (2018) Utility of 3D reconstruction of 2D liver computed tomography/magnetic resonance images as a surgical planning tool for residents in liver resection surgery. J Surg Educ 75: 792-797.
3. Su L, Dong Q, Zhang H, et al. (2016) Clinical application of a three-dimensional imaging technique in infants and young children with complex liver tumors. Pediatr Surg Int 32: 387-395.
4. Zheng YX, Yu DF, Zhao JG, et al. (2016) 3D printout models vs. 3D-rendered images: Which is better for preoperative planning? J Surg Educ 73: 518-523.
5. Zhang WW, Wang HG, Shi XJ, et al. (2016) Significance of three-dimensional reconstruction as a method of preoperative planning of laparoscopic radiofrequency ablation. Zhonghua Wai Ke Za Zhi 54: 692-699.
6. Debbaut C, Segers P, Cornillie P, et al. (2014) Analyzing the human liver vascular architecture by combining vascular corrosion casting and micro-CT scanning: A feasibility study. J Anat 224: 509-517.
7. Conversano F, Franchini R, Demitri C, et al. (2011) Hepatic vessel segmentation for 3D planning of liver surgery experimental evaluation of a new fully automatic algorithm. Acad Radiol 18: 461-470.
8. Herfarth C, Lamadé W, Fischer L, et al. (2002) The effect of virtual reality and training on liver operation planning. Swiss Surg 8: 67-73.
9. Sauer IM, Queisner M, Tang P, et al. (2017) Mixed reality in visceral surgery: Development of a suitable workflow and evaluation of intraoperative use⁃ Ann Surg 266: 706-712.
10. Uchida Y, Taura K, Nakao M, et al. (2019) A clinical pilot study of resection process map: A novel virtual hepatectomy software to visualize the resection process, case series. Int J Surg 71: 36-40.
11. Takamoto T, Sano K, Hashimoto T, et al. (2018) Practical contribution of virtual hepatectomy for colorectal liver metastases: A propensity-matched analysis of clinical outcome. J Gastrointest Surg 22: 2037-2044.
12. Matsuki R, Mise Y, Saiura A, et al. (2016) Parenchymal-sparing hepatectomy for deep-placed colorectal liver metastases. Surgery 160: 1256-1263.
13. Mise Y, Hasegawa K, Satou S, et al. (2018) How has virtual hepatectomy changed the practice of liver surgery?: Experience of 1194 virtual hepatectomy before liver resection and living donor liver transplantation. Ann Surg 268: 127-133.
14. Takamoto T, Mise Y, Satou S, et al. (2018) Feasibility of intraoperative navigation for liver resection using real-time virtual sonography with novel automatic registration system. World J Surg 42: 841-848.
15. Oshiro Y, Ohkohchi N (2017) Three-dimensional liver surgery simulation: Computer-assisted surgical planning with three-dimensional simulation software and three-dimensional printing. Tissue Eng Part A 23: 474-480.
16. Soler L, Nicolau S, Pessaux P, et al. (2014) Real-time 3D image reconstruction guidance in liver resection surgery. Hepatobiliary Surg Nutr 3: 73-81.
17. Benckert C, Bruns C (2014) The surgeon’s contribution to image-guided oncology. Viszeralmedizin 30: 232-236.
18. Witowski J, Wake N, Grochowska A, et al. (2019) Investigating accuracy of 3D printed liver models with computed tomography. Quant Imaging Med Surg 9: 43-52.
19. Buchs NC, Volonte F, Pugin F, et al. (2013) Augmented environments for the targeting of hepatic lesions during image-guided robotic liver surgery. J Surg Res 184: 825-831.
20. Chen G, Yang SZ, Wu GQ, et al. (2009) Development and clinical application of 3D operative planning system of live in virtual reality environments. Zhonghua Wai Ke Za Zhi 47: 1620-1623.
21. Chen G, Li XC, Wu GQ, et al. (2010) The use of virtual reality for the functional simulation of hepatic tumors (case control study). Int J Surg 8: 72-78.
22. Mise Y, Tani K, Aoki T, et al. (2013) Virtual liver resection: Computer-assisted operation planning using a three-dimensional liver representation. J Hepatobiliary Pancreat Sci 20: 157-164.