Multi-phase computed tomography images are widely applied to doctors’ routine interpretation of organic lesions or before human abdominal surgery. However, because of the lack of intuitive and three-dimensional (3D) observations, human factors lead to 80% of the clinical errors. A primary malignant tumor resection system based on a virtual reality (VR) helmet and a force-feedback device combined with a 3D printing model is proposed in this paper. First, we used the thin-plate spline (TPS) deformation method to register the different phase images. In the case of a metastatic tumor, a spherical scoring filter (SSF) model was built for searching the tumor pattern with edge detection and subtraction processing, from which the initial tumors were selected on the basis of the calculated score. For hepatocellular carcinoma cases, candidates were extracted as the areas without edges by using edge detection filters on the subtraction image between the equilibrium and arterial phase images. Finally, the false positive (FP) candidate was eliminated before obtaining the 3D shape of the liver tumor in the expansion process, thus providing an accurate volume of lesions for the surgical plan for the liver resection. The results showed that our virtual surgery system enabled the user to simulate the use of a scalpel for cutting and removing the organ labels; 3D display on a VR helmet with a deformable effect; and touching organs with a force-feedback device. Our virtual surgery tools could simulate all of the effects of the doctors’ operations and make clinical practice more efficient.
El-SeragH.B. and MasonA.C., Rising incidence of hepatocellular carcinoma in the United States, N Engl J Med340 (1999), 745–750.
2.
OkumuraE., SanadaS., SuzukiM. and MatsuiO., A computer-aided temporal and dynamic subtraction technique of the liver for detection of small hepatocellular carcinomas on abdominal CT images, Phys Med Biol51 (2006), 4759–4771.
3.
NakagawaJ., ShimizuA. and KobatakeH., Development of an automated extraction method for liver tumors in three-dimensional multiphase multislice images, Systems and Computers in Japan36 (2005), 43–54.
4.
HayashiY., DeguchiD., KitasakaY., Mori;K., MekadaY. and SuenagaY., Detection of liver cancer regions from dynamic CT images, International Journal of Computer Assisted Radiology and Surgery. LemkeH.U., InamuraK., DoiK., VannierM.W. and FarmanA.G. (Eds.), 1(1) (2006), 524–525.
5.
TajimaT., ZhangX., KitagawaT., KanematsuM., ZhouX., HaraT., FujitaH., YokoyamaR. and KondoH., and H Hoshi, Computer-aided detection (CAD) of hepatocellular carcinoma on multiphase CT images, Proc of SPIE Medical Imaging 2007: Computer-Aided Diagnosis6514 (2007), 65142Q-1-10.
6.
FreimanM., EdreiY. and GrossE., Liver metastases early detection using fMRI based statistical model, Proc of the 5th IEEE Int Symp on Biomedical Imaging: From Nano to Macro, 2008, pp. 584–587.
7.
ZhangX., FurukawaT., FujitaH., KanematsuM., ZhouX. and HaraT., Detection of metastatic liver tumor in multiphase CT images by using a spherical gray-level differentiation searching filter, SPIE Medical Imaging: Computer-aided Diagnosis7963 (2011), 79632K–79632K-8.
8.
RiversJ., RueckertD., SonodaLI., HayesC., HillD., LeachM.O. and HawkesD.J., Non-rigid registration using free-form deformations: Application to breast MR images, IEEE Trans on Medical Imaging18(8) (1999), 712–721.
9.
ZhangX., TanX., GaoX., WuD., ZhouX. and FujitaH., Non-rigid registration of multi-phase liver CT data using fully automated landmark detection and TPS deformation, Cluster Computing-The Journal of Networks, Software Tools and Applications (2018). https://org/10.1007/s10586-018-2567-3.
10.
SutherlandI.E., The ultimate display. Multimedia, From Wagner to virtual reality, 1965.
11.
CosmanP.H., CreganP.C. and MartinC.J., Virtual reality simulators: Current status in acquisition and assessment of surgical skills, ANZ Journal of Surgery72 (2002), 30–34.
12.
HalvorsenF., ElleO.J. and FosseE., Minimally invasive therapy & allied technologies: MITAT: Official journal of the Society for Minimally Invasive Therapy,14, Simulators in Surgery14 (2005), 214–223.
13.
BasdoganC., SedefM., HardersM. and WesargS., VR-based simulators for training in minimally invasive surgery, IEEE Computer Graphics and Applications27 (2007), 54–66.
14.
TangW. and WanT.R., Constraint-based soft tissue simulation for virtual surgical training, IEEE Transactions on Biomedical Engineering61 (2014), 2698–2706.
15.
SpicerM.A., VelsenM., CaffreyJ.P. and ApuzzoM.L., Virtual reality neurosurgery: A simulator blueprint, Neurosurgery54 (2004), 783–797.
16.
VitielloV., LeeS.L., CundyT.P. and YangG.Z., Emerging robotic platforms for minimally invasive surgery, IEEE Reviews in Biomedical Engineering6 (2013), 111–126.
17.
HassanT., HameedA., NisarS., KamalN. and HasanO., Al-zahrawi: A telesurgical robotic system for minimal invasive surgery, IEEE Systems Journal10 (2016), 1035–1045.
18.
AiY., PanB., NiuG., FuY. and WangS., Master-slave control technology of isomeric surgical robot for minimally invasive surgery, Proc of 2016 IEEE International Conference on Robotics and Biomimetics2016, pp. 2134–2139.
19.
MohanA. and ProctorM., Virtual reality–a ‘play station’ of the future. A review of virtual reality and orthopaedics, Acta Orthopaedica Belgica72 (2006), 659–663.
20.
MaberyJ.D., ReinigK.D. and CannonW.D., Virtual reality in orthopaedics: Is it a reality?Clinical Orthopaedics and Related Research468 (2010), 2586–2591.
21.
KingS.B., When medicine meets virtual reality, JACC Cardiovascular Interventions3 (2010), 691–692.
22.
PiquerasM., MarcoE. and CollM., Effectiveness of an interactive virtual telerehabilitation system in patients after total knee arthoplasty: A randomized controlled trial, Journal of Rehabilitation Medicine45 (2013), 392–396.
23.
LeiboldN. and SchwarzL., Virtual simulations: A creative, evidence-based approach to develop and educate nurses, Creative Nursing23 (2017), 29–34.
24.
PillaiA.K. and KhuranaProf.A., Design and development of security framework for data storage in public cloud model, International Journal of Innovative Research in Computer and Communication Engineering (IJIRCCE)5(11) (2017).
25.
NicolauS., SolerL. and MutterD., Augmented reality in laparoscopic surgical oncology, Surgical Oncology20 (2011), 189–201.
26.
SeitelM., Maier-HeinL. and RietdorfU., Towards a mixed reality environment for preoperative planning of cardiac surgery, Studies in Health Technology and Informatics142 (2009), 307–309.
27.
DelpS.L., LoanJ.P. and HoyM.G., An interactive graphics-based model of the lower extremity to study orthopaedic surgical, IEEE Transactions on Biomedical Engineering37 (1990), 757–766.
28.
CourtecuisseH., HungH. and AllardJ., GPU-based real-time soft tissue deformation with cutting and haptic feedback, Progress in Biophysics and Molecular Biology103 (2010), 159–168.
29.
PontoK., ShinH.J., KohlmannJ. and GleicherM., Online real-time presentation of virtual experiences for external viewers, Proc ACM Symp Virtual Real Softw Technol (2012). 10.1145/2407336.2407346
30.
LivatinoS., PaolisL.T.D., AgostinoM.D’, ZoccoA., AgrimiA., SantisA.D., BrunoL.V. and LapresaM., Stereoscopic visualization and 3-D technologies in medical endoscopic teleoperation, IEEE Transactions on Industrial Electronics62 (2015), 525–535.
31.
BeylT., SchreiterL. and NicolaiP., 3D perception technologies for surgical operating theatres, Studies in Health Technology and Informatics220 (2016), 45–50.
32.
GerlachS., KuhlemannI. and JauerP., Robotic ultrasound-guided SBRT of the prostate: Feasibility with respect to plan quality, International Journal of Computer Assisted Radiology and Surgery12 (2017), 149–159.
33.
KimY.J., ParkS.W., YeomH.G., BangM.S., KimJ.S., ChungC.K. and KimS., A study on a robot arm driven by three-dimensional trajectories predicted from non-invasive neural signals, Biomedical Engineering Online14 (2015), 81.
34.
WangP., BeckerA. and JonesI., A virtual reality surgery simulation of cutting and retraction in neurosurgery with force-feedback, Computer Methods and Programs in Biomedicine84 (2006), 11–18.
35.
RohrK., StiehlH.S., SprengelR., BuzugT.M. and WeeseJ., Landmark-based elastic registration using approximating thin-plate splines, IEEE Transactions on Medical Imaging20 (2002), 526–534.
36.
PrakashA. and RavichandranC.S., Power search algorithm for economic load dispatch problems considering valve point loading effects, International Journal of Innovative Research in Computer and Communication Engineering (IJIRCCE)5(12) (2017).
37.
BooksteinF.L., Principal warps: Thin-plate splines and the decomposition of deformations, IEEE Trans on Pattern Analysis and Machine Intelligence11 (1989), 567–585.
38.
BaeK., GigerM. and ChenC., Automatic segmentation of liver structure in CT images, Med Phys20 (1993), 71–78.
39.
HitosugiT., ShimizuA., TamuraM. and KobatakeH., Development of a liver extraction method using a level set method and its performance evaluation, Journal of Computer Aided Diagnosis of Medical Images7 (2003), 85–88 (in Japanese).
40.
GaoL., HeathD., KuszykB. and FishmanE., Automatic liver segmentation technique for three-dimensional visualization of CT data, Radiology201 (1996), 359–364.
41.
MasumotoJ., HoriM. and SatoY., Automated liver segmentation using multislice CT images, IEICE84(2) (2001), 2150–2161. (in Japanese).
42.
TiwariS. and Shukla Dr.N., Copyrights protection techniques: A survey, International Journal of Innovative Research in Computer and Communication Engineering (IJIRCCE)6(1) (2018).
43.
ParkH., BlandP.H. and MeyerC.R., Construction of an abdominal probabilistic atlas and its application in segmentation, IEEE Trans on Med Imag22(4) (2003), 483–492.
44.
ZhouX., TakayamaR., WangS., HaraT. and FujitaH., Deep learning of the sectional appearances of 3D CT images for anatomical structure segmentation based on an FCN voting method, Med Phys44 (2017), 5221–5233.
45.
ZhangX., TajimaT., KitagawaT., KanematsuM., ZhouX., HaraT., FujitaH., YokoyamaR., KondoH. and HoshiH., Segmentation of liver region with tumorous tissues, Proc of SPIE Medical Imaging 2007: Image Processing6512 (2007), 651235-1-651235-9.
46.
MarrD. and HildrethE., Theory of edge detection, Proc Royal Society of London207 (1980), 187–217.
47.
ZhangZ., ZhangX., WuD., ZhangY. and DengJ., Automatic extraction of primary malignant liver tumor for surgical liver resection, Journal of Investigative Medicine61 (2013), 17.
48.
ZhangX., FurukawaT., FujitaH., KanematsuM., ZhouX. and HaraT., Detection of metastatic liver tumor in multiphase CT images by using a spherical gray-level differentiation searching filter, SPIE on Medical Imaging: Computer-Aided Diagnosis (2011), 7963–7991.
49.
HaralickR.M. and ShanmugamK.I., Texture features for image classification, IEEE Trans Sys Man Cybern3(6) (1973), 610–621.
50.
ZhangX., GaoX., LiuB.J., MaK., YanW., LongL.L., YuhongH. and FujitaH., Effective staging of fibrosis by the selected texture features of liver: Which one is better, CT or MR imaging?Comput Med Imag Grap46 (2015), 227–236.
51.
KawajiriS., ZhouX., ZhangX., HaraT., FujitaH., YokoyamaR., KondoH., KanematsuM. and HoshiH., Automated segmentation of hepatic vessels in non-contrast X-ray CT images, Radiological Physics and Technology1 (2008), 214–222.
52.
HoppeH., DeRoseT., DuchampT., McDonaldJ. and StuetzleW., Mesh optimization, Proceedings of the ACM SIGGRAPH ’ 93, 1993, pp. 19–26.
53.
ZhangX., ZhouX., HaraT. and FujitaH., Non-invasive assessment of hepatic fibrosis by elastic measurement of liver using magnetic resonance tagging images, Applied Sciences8 (2018), 437. doi: 10.3390/app8030437, online publication
54.
ZhangX., LiB. and WuD., A web based 3D model construction tool for abdominal organs segmentation, Basic & Clinical Pharmacology & Toxicology124 (2018), 26.
